[maemo-developers] QA process = bug fixing disincentive?

[From: Martin Grimme]

Hi,

resetting Karma on a new version leads to one very bad issue, IMHO:

Developers of packages with some Karma will hold back bugfix-updates
until the unfixed version has reached extras.

This should be avoided.


Martin


2009/11/1, Henrik Hedberg <henrik.hedberg at innologies.fi>:
> Igor.Stoppa at nokia.com wrote:
>
>> I think the problem here is that some braindead system has been
>> introduced,
>> which doesn't account for the actual work being done.
>
>     And what is the biggest mistake here is that the new system has been
> put into production before testing it at all.
>
>     Someone just came up with an idea of crowd sourcing the QA and used
> random generator to set the parameters. Soon it was in real use before
> any experiences of its functionaly had got.
>
>     I would have understood if the parameters had been really low (say,
> 1 day and 1 thumbs up) at the beginning, or there would have been a
> separate repository (say, extras-selection) to test this new
> functionality. The parameters could have been changed according to
> success of the system and in parallel with the amount of testers (and
> devices available).
>
>     The sad contradiction here is that developers are expected to
> produce high quality outcomes and their results are put under QA, put
> the maemo.org processes and tools are not! Those should have been
> tested, for example, with a smaller group of developers before putting
> into production system.
>
>     The maemo-extras testing marathon did amazing job. Thanks to
> everybody who participated! However, it cannot be a permanent way to
> overcome one of the biggest problems with the new system.
>
>     I am not totally against the new system, and I do recognize the
> importance of quality assurance. I just hope that we can learn from the
> past and react rapidly. Please, do not refer to the better future and
> the possibility to have more users and testers later. Things should work
> now!
>
>     Here are my suggestions now:
>
> 1) Fine tune the parameters: say, 5 days and 5 votes. These can be
> changed later when the system is working (has enough testers).
>
> 2) Change the system so that user packages that are depended by another
> user package are promoted automatically when the actual user package is
> promoted (like non-user packages are promoted currently). For example,
> when an user is testing Mauku, she is implicitely testing also the
> microfeed package [1]).
>
> 3) Find a way to overcome the limitations when an upgraded package is an
> important security fix.
>
>     And here are my older suggestions [2], which, I think, are still valid:
>
> * Negative karma can be given _only_ if it based on the agreed QA
> requirements. (Testers are still giving karma based on their subjective
> thinking instead of QA requirements.)
>
> * The package page should have a link to a bug tracker and the link must
> be used! (Comments are stored into a wrong place currently. It is double
> effort for a developer to track the packages interface in addition to
> bugs.maemo.org.)
>
> * Negative karma can be given _only_ with a link to a bug tracker having
> a bug report about the show stopper. It may be either a new bug report
> written by the tester or an old open bug report just referred in the
> comment.
>
> * Negative karma is automatically removed when the related bug report is
> closed (fixed or other way resolved). (Currently, there is no way to
> remove the negative karma (thumbs down) if the bug is fixed. Please,
> note that the bug may be in the library code, and the bug in the package
> is thus actually fixed when the library is fixed. So, there would not be
> any need to update the application package every time.)
>
>     BR,
>
>     Henrik
>
>     P.S. I do not necessarily see that more testers will make things
> easier and more workable. The more testers there are, the more
> subjective evaluations we will get. If one tester just do not like the
> graphics of an application he may give thumbs down, and even four other
> testers giving thumbs up are needed to fix that misjudgement. I really
> would like to see a discussion about the responsibilities of testers.
> Should there be a mechanism to give negative karma to testers who are
> not following the QA rules, or even way to ban them?
>
> [1] http://talk.maemo.org/showthread.php?p=362575#post362575
>
> [2]
> http://lists.maemo.org/pipermail/maemo-developers/2009-September/020921.html
>
> --
>     Henrik Hedberg  -  http://www.henrikhedberg.net/
> _______________________________________________
> maemo-developers mailing list
> maemo-developers at maemo.org
> https://lists.maemo.org/mailman/listinfo/maemo-developers
>