[maemo-developers] Follow up from QA meeting on IRC

[From: Dave Neary]

Hi,

Edward Page wrote:
> To help remind people that there is a checklist and whats on it,
> should the rating page link to or include the criteria?
> 
> I see there were no notes on the algorithm.  A threshold of 10 was
> annoying as a developer.  As a tester, a threshold of 10 made me feel
> more comfortable not doing a full blown /opt check or power management
> check because of 10 people I could hope someone else would do it and I
> could worry about other issues like application stability.  With a
> smaller threshold I would feel more of a burden to do all of the steps
> which would discourage me.
> 
> So I guess I'll share my idea.  To me, it seems that one tester would
> probably be enough for /opt, power management, etc.  If the categories
> were broken out, these could just require a net of +1 karma with a
> required comment to describe steps and results regardless of whether
> they gave an up or down.  Net +1 is in case others disagree, they can
> vote it down.  Required comments either way are to make people feel
> comfortable that it was tested properly and not just someone saying
> "it works for me" and voting it up.

Ed's point definitely resonates with me. The great thing about QA is
that you can crowd-source it effectively if you don't require much of
the user/tester. It seems like the Maemo QA process is more
developer-focussed than user-focussed at the moment, and is as such
pushing a lot of the responsibility for the QA process to the user.

This seems like an ideal opportunity to lower the barrier to
participation to tiny levels, but only if it is

1. easy to give a +1/-1
2. We don't require intimate knowledge of the Maemo community for
feedback (I'm thinking of the checklist, what "optification" means, etc)
3. We require enough feedback that most of the code paths in the
application will be tested before we OK an application

Lowering the threshold to 5 is implicitly saying "we're not getting
feedback quickly enough", which in turn is saying "the feedback process
is overly cumbersome for a casual user". It seems to me that that's the
problem, rather than the contents of the checklist or the threshold in
place. If giving feedback was trivially easy (as it is, for example, in
Android Market) you would be getting hundreds of votes when new versions
of applications are released, as people installed & used them.

So - how can we make giving the feedback and voting on applications easier?

Cheers,
Dave.

-- 
maemo.org docsmaster
Email: dneary at maemo.org
Jabber: bolsh at jabber.org