[maemo-developers] How to use extras-testing correctly?

[From: Graham Cobb]

My long reply was lost when my N900 crashed while writing it.  So this will have to do.

I strongly disagree.  Testers should be testing against the agreed requirements only.  The subjective element should be eliminated as much as possible - we are using human testers because it is impossible to do this testing manually, not because we expect different opinions.  We are using more than one tester just so that a problem doesn't slip through because one tester missed it, not because we expect voting.

By all means add a comment if you are unhappy with the UI but it should get a +1 as long as it doesn't conflict with the requirements.  Ideally every -1 should require a comment saying which of the QA requirements is violated.

Graham

--
Sent from Nokia Nseries mobile computer
----- Original message -----
> Am Mittwoch, den 23.09.2009, 11:07 +0100 schrieb Andrew Flegg:
> > It's easy to find problems, but our basic QA procedure started off as
> > "installs cleanly, uninstalls cleanly, doesn't waste battery, has
> > human readable description, has nice icons" and now "uses /opt and
> > doesn't waste disk space if appropriate". These are detailed at:
> >
> >        http://wiki.maemo.org/Extras-testing#Quality_criteria
> >
> > Should someone be pressing thumbs up if a package meets those
> > requirements and has no blocking/crashing bugs? That doesn't seem to
> > be happening - is it because no-one's testing, or because no-one feels
> > clear as to what they should be doing?
>
> I must admit that so far I've considered giving a Thumbs Up or Thumbs
> Down to be a purely subjective decision based on personal impression.
> I for myself e.g. would give a Thumbs Down for a very small, working app
> if the main menu is not fremantle'ized (as the maintainer does not have
> a large codebase to maintain this should not be too hard), but that
> would not be much of an issue to me for a large complicated app (lots of
> other more important stuff to maintain and work correctly).
>
> That's why others can still vote +1 if I said -1 and the other way round
> - I assume everybody has his/her own ideas about thresholds and
> expectations and that's exactly why there's more than one person allowed
> && required to vote.
>
> Just my current point of view,
> andre
> --
> Andre Klapper (maemo.org bugmaster)
>
> _______________________________________________
> maemo-developers mailing list
> maemo-developers at maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers