[maemo-developers] Testing marathon & Q&A Feedback

[From: Ville Reijonen]

Hi,

> But the time should be better split. The basic feedback I got from busy
> Nokia employees with devices, professional experience and willingness to
> help is that the thumbs up/down should be applied to each QA criteria.

Yap, this I suggested. Good that this is verified. So, it should be 
possible to give thumbs up for single item on the QA list. This of 
course means that current 10 upvotes system does not work as is. 
Proposal below (feel free to hack):

-Every package has minimum 10 days of quarantee.
-Package overstaying over a month is booted back to devel.
  (If it is not accepted by then, it will never be)
-Package retains its history for later viewing.
  (Nice to see what has happened before)
-Every package has QA list.
  (Based the current wiki list - the testing criteria)
-Every item on the QA list has to be examined before acceptance.
  (The procedure should be the same for all)
-A person can check one or multiple QA items.
  (Do what you are good at or what you have time for)
-Same QA list item can be checked by multiple persons.
  (Never trust a single person, or do we? High karma -> power of two)
-Some QA items can be checked automatically.
  (optification for example)
-Bugfix will reset the quarantee timer.
  (The quarantine was clearly working, some more is therefore needed)
-Bugfix will remove the second upvote for every QA item.
  (Bugfix can introduce other bugs, better to be sure)
-A qualified application is released to extras.

As said, feel free to modify. First is always rough.. :)

..I have seen something quite similar, a familiar basic structure..
- Queues
- Quarantine
- Checklist
- History
- Check and cross check procedures
- Automation
Anyone else?
A hint: ever travelled with a pet?

>> Maybe this manna/karma thing has been though out, but somehow if feel
>> that the research for similar systems was not done before rolling it
>> out. There seems to be too many holes, and I just though a while. 
>> Every serious linux distribution has some kind of QA system. Most of 
>> the software makers have QA systems. Do not invent the wheel again..
>
>URLs proving your point, please?

I have limited free time, sorry. I'll just give yet another example. How 
extras security is supposed to be handled? How to report a security 
problem? Currently there is no way. It can not go thru voting, it'll be 
stalled in queue (note from earlier discussion). How Debian does it? Ah, 
separate security team which can bypass the queues. How about that..

Actually it is not just QA, it's the whole process.

> Not every Linux distribution has the consumer focus of Maemo 5 / N900
> and definitely not every Linux distribution needs to handle specific
> problems of mobile devices. Besides, not every Linux distribution pay
> actually much attention to the application packages being pushed by its
> contributors and it takes only 10 random installs from your distro to
> find out.

But some (distributions) have consumer focus? And some (distributions) 
handle specific problems (for mobile devices)? And some pay attention to 
what they ship? Of course there is no single one which is exact match. 
One needs to take the best parts, and add some own sauce. Building up 
from zero because none is similar is just plain silly. We could be using 
tools which already work and spice them a bit!

Having people voting for packages to go through queue does not make us 
any special yet - that will come later on.

> So no, there are not that many references for community QA models.

But there are some? If there is need to go deeper, maybe it is not QA 
models but some other community based structures what should be looked 
at. Maybe some friendly sociologist could help. I know only hairy 
computer scientists and business yuppies so no go for me :)

> I believe the Extras QA process is highly innovative and it has its
> chances of serving a useful purpose to the rest of the free software
> community.

Definitely, it will be.

-- 
VRe :: http://iki.fi/vre :: +358 40 5775 456