[maemo-developers] QA from extras-devel to extras-testing

[From: Quim Gil]

Hi,

Let me start again because I don't think this downstream-upstream
relationship is relevant to define hhis extras QA process.

Proof: if an application in extras has been demoted because of a severe
bug in a library it's the maintainer of that app the main responsible of
finding a solution. Going upstream might be the safest bet but probably
not the fastest. He will decide in any case. If a package is used by
many apps then the problem is common to many maintainers and, again,
they will decide whether to fix the bug themselves and file the fix
upstream or report upstream and fix the problem altogether.

In any case it is not the business of the maemo.org extras QA process to
deal with the bugfix itself. This process should make sure the quality
of the apps offered to end users is good, promoting what is good and
demoting what is bad.

Goals

- We need to know when an application in extras-devel is ready to be
tested by power users (betatesters).

- We need to know when an application in extras-testing is ready for end
users.

- We need to have a community process to report severe bugs and demote
an application if a confirmed severe bug is not being addressed.


Assumptions:

- We have extras-devel, extras-testing and extras.

- The jump from extras-devel to extras-testing is initiated manually by
the maintainer of an application once he thinks it's free of major
issues and would be stopped by some kind of objective criteria applied
to automated testing with no human testing: installs/de-installs,
package info complete...

- The jump from extras-testing to extras is made automatically after n
days when human-based criteria are met: no crashes or severe bugs
detected, n votes for promotion. Betatesters have specific tools
installed in their devices helping them to detect and report crashes and
 power management issues.

- Betatesters can also flag an application that is buggy enough to be
demoted. Only the relevant packages should be demoted, as one library
might be shared by many apps. They would be all demoted if the library
is demoted so we want to do this only when it is proven that the severe
bug is in the library. This means that the demotion needs to be done
manually by people really knowing what they are doing.

- We are looking for end user perceived quality and therefore we need to
handle end user bug reports. We can't expect a user to know what exact
package or library is causing the problem, the bugs will be filed
against applications found buggy.

-- 
Quim Gil
open source advocate
Maemo Software @ Nokia