[maemo-developers] QA process = bug fixing disincentive?

[From: Andrew Flegg]

On Wed, Nov 4, 2009 at 09:03,  <tero.kojo at nokia.com> wrote:
>
> Two days later I notice a blinking orange light in my status bar. I see
> a new version of the application. I install, I check what has changed
> (minor or major?), I run my tests and thumb it up again.

Aside: how do you check what has changed?

> I didn't loose too many minutes or hours of my life, no one got killed
> or injured, and the punitive damages to all parties appear to be zero.
> Life's good and I get a cup of coffee.

Trouble is, developers (myself and others) are saying the current
process of resetting to zero on every single promotion is incompatible
with "release early and release often". The theory about resetting to
zero is that testers will do the full QA process again (check it still
uses /opt, check a bug hasn't been introduced which introduces battery
management issues, check that it both installs, uninstalls and
upgrades cleanly).

> In other words, I test applications that I use or leave on device.
> It's not something I do randomly. I believe that the author of the app
> has committed to the app, so in return I commit to it as well. The
> rules and process are a good thing, but in the end it is people who
> do the things. No matter if it is the developer, the tester or the
> user, it's about the people.

Yep, however I now disagree with the logical conclusion of your
statement (which has been expressed elsewhere) which is "when we have
lots of users it'll all work itself out".

[and your original statement]
> Too complicated.

We seem(ed ;-)) to have a consensus between this thread, and the QA
marathon feedback, between both developers and testers that resetting
to zero was causing more problems than it solved. We've now moved on
to ways of maintaining some app karma from one release to another, but
it sounds like you disagree with that consensus? It may be that there
are two kind of testers:

    * Dedicated QA browsers. Will install any app from the QA queue
      to evaluate and score it. Will be done whenever they have time.

    * Dedicated application users. Will install upgrades of apps they've
      installed and evaluate/score them as pushed to.

Cheers,

Andrew

-- 
Andrew Flegg -- mailto:andrew at bleb.org  |  http://www.bleb.org/