[maemo-developers] QA process = bug fixing disincentive?

[From: Jeremiah Foster]

On Nov 3, 2009, at 20:36, Henrik Hedberg wrote:

> Tim Teulings wrote:
>
>>> Except how do you try to prevent abuse (whether intentional or
>>> accidental)? At least with the version number you've got some safety
>>> check (although it is in no way comprehensive). It also requires  
>>> more
>>> changes at more levels (I bet), so harder to implement.
>>
>> I think it is time to decide (again?) if we trust developers in their
>> atempt to get their software/package into extras or not.
>
>    Currently, we trust ten random testers rather than one well-known
> developer.

Please note that many of the people who have tested your app are  
developers, maemo staff, or longtime members of the community. They  
have a stake in your success since they use your application. No one  
needs to be rude or harsh, but the criticism they pass along is only  
meant to make Mauku better, it is not directed at you personally.

>    Why could not we trust well-known developers who have track record
> of producing high-enough quality software?

I don't think anyone does not want to trust the developers. I most  
adamantly have stated that we should, in fact we must, trust devs  
implicitly.

> They may have their own
> methods for testing, like couple of active and skilful dedicated  
> testers
> for the application domain. I see that more trustful than those random
> testers who vote subjectively based on their opinions of an user  
> interface.

One of the lessons of any QA process is that testing and development  
should be separate. This leads to greater ability in finding bugs and  
often shortens the development cycle because developers are developing  
based on user requests and doing more Test Driven Design.

>    We could have a group of accredited developers who have access to
> the Extras. They are committed to release only validated and verified
> packages. When a new developer wants to upload a package of his own,  
> an
> accredited developer could sponsor him, i.e. act like a gatekeeper.

But now it is you who doesn't trust people.

>    Sounds familiar? See Debian New Maintainers Process [1].

This is unfortunately not an example I would like to follow. Becoming  
a debian developer takes at least a year, and many developers go MIA.  
Debian's founder Ian Murdoch calls debian 'process run amok'. I don't  
think we need to replicate that here. Furthermore, I think Nokia would  
loathe to participate in a program that excludes anyone who paid good  
money to buy a device. I see no reason to do so and I think any  
barrier to entry for testers, power users, users and devs is a non- 
starter.
>
>    Another possibility is to have the team of accredited testers, who
> can make the final decision of their own. To become an accredited  
> tester
> required commiting to write sane bug reports and to make decisions on
> generally agreed checks, among others. Then there would not be a need
> for ten random strangers and ten day delays.

Again - no accreditation, no background checks, no process. Rather, we  
adjust the process so that trust is granted to everyone and we put up  
with occasional misuse. I think this is an excellent community and I  
trust the developers here to do the right thing.

Let's not forget this is designed to be a QA process - to make  
software better. That is the goal. If we are not reaching that goal,  
we have to change, I think everyone is aware from the maemo.org side  
that if this isn't working, it will get fixed to the satisfaction of  
all.

And despite various complaints, many are saying that the process will  
in fact produce better software. So we are in the right area anyway.

Jeremiah