[maemo-developers] How to use extras-testing correctly?

[From: Fred Lefévère-Laoide]

And you're lucky if you get votes !

I can imagine nobody cares about pwsafe (except 2 people) but does that 
mean that if you get no vote in extras-testing you'd better forget about 
your app and start something new ?

I know you can't force people to vote or test ... But it means that if 
an app has a very limited public it doesn't get out of extras-testing ?

Fred


Henrik Hedberg a écrit :
>     Extras-testing QA is not working as it is implemented now! There are
> two main issues:
> 
> * Comments are stored into a wrong place. Those belong to Bugzilla! It 
> is double effort for a developer to track two different places or to
> transfer reports into bug tracking system manually.
> 
> * Developers are giving karma based on their subjective thinking instead
> of agreed (?) QA requirements.
> 
>     Let's analyse karma and comments that Mauku has got:
> 
> * Two testers of five either add a new bug report or search the existing 
> bug reports before entering a comment. Good for those two, bad for the rest.
> 
> * Negative karma given at 2009-09-24 04:52 UTC and related comment 
> written at 2009-09-24 04:55 UTC. Is there any real reason to give 
> negative karma based on the comment? Tester either wants some new 
> features (definitely not based on QA requirements) or some minor user 
> interface modifications (not a show stopper).
> 
> * All testers report at least one issue that is not actually related to 
> the application itself but the underlying library. I understand that it 
> is hard for end-user to see the difference, but what happens when the 
> library is updated? These issues are fixed, and so is the application 
> also, but the negative karma stays there.
> 
>     My suggestion here is that:
> 
> * Negative karma can be given _only_ if it based on the agreed QA 
> requirements.
> 
> * The package page should have a link to a bug tracker.
> 
> * Negative karma can be given _only_ with a link to a bug tracker having 
> a bug report about the show stopper. It may be either a new bug report 
> written by the tester or an old open bug report just referred in the 
> comment.
> 
> * Negative karma is automatically removed when the related bug report is 
> closed (fixed or other way resolved).
> 
>     Graham Cobb wrote:
> 
>> Testers should be testing against the agreed
>> requirements only.  The subjective element should be eliminated as
>> much as possible - we are using human testers because it is
>> impossible to do this testing manually, not because we expect
>> different opinions.  We are using more than one tester just so that a
>> problem doesn't slip through because one tester missed it, not
>> because we expect voting.
> 
>     I strongly agree!
> 
>> By all means add a comment if you are unhappy with the UI but it
>> should get a +1 as long as it doesn't conflict with the requirements.
>> Ideally every -1 should require a comment saying which of the QA
>> requirements is violated.
> 
>     I strongly agree. Actually, I would remove the word "ideally". It is 
> a must! And with the relationship to bug tracker I described above.
> 
>     BR,
> 
>     Henrik
>