[maemo-developers] QA process = bug fixing disincentive?

[From: Marius Vollmer]

ext Attila Csipa <maemo at csipa.in.rs> writes:

> Yes, there is definitely a sense of throwing out the baby with the bathwater 
> here - as is, with a sufficiently mature app, NOT applying simple fixes will get 
> the app to the user quicker, and applying the fixes will keep the app AWAY from 
> the users.

I am not involved with all of this, so my opinions are very suspect, and
I haven't even read the QA process, but here goes anyway:

Shouldn't it be so that a package collects karma over its whole
lifetime, starting out at zero and then gaining from version to version
as it matures?

A high karma could mean that a package has many users/testers and that
we are confident that bugs are found quickly, that the developers have a
good track record of not screwing up and have demonstrated that they
understand the ins and outs of the Maemo platform (like power
management, package icons, etc), and other things that remain true even
when a package is changed.

The fitness of a concrete version of a package would be expressed as our
"confidence that it contains no critical bugs".  That confidence is
proportional to

  - time in extras-testing without critical bugs being reported, and

  - confidence of the developer that the change doesn't break anything,
    weighted by

  - the karma that the package/developer has gathered with the previous
    versions.

Then, we strike a balance between the confidence of having no critical
bugs and the urgency of the update.  Urgent updates such as security
fixes can be risked with lower confidence.

Thus, a package with zero karma and a very confident developer might
still spent 10 days in extras-testing.  Once the package has gained 10
karma points, it can leave extras-testing after two days, say, unless
the developer explicitly asks for more test exposure.