[maemo-developers] Testing marathon & Q&A Feedback

[From: Quim Gil]

Hi,

In general I think that new apps should tend to get all ratings reseted
if they go back to extras-devel because of a blocker, while app updates
would keep their current positive ratings through new extras-testing
iterations. This way we are conservative with new apps but more liberal
with updates. Makes sense?

ext Ville Reijonen wrote:
> Hi,
> 
>> But the time should be better split. The basic feedback I got from busy
>> Nokia employees with devices, professional experience and willingness to
>> help is that the thumbs up/down should be applied to each QA criteria.
> 
> Yap, this I suggested. Good that this is verified. So, it should be 
> possible to give thumbs up for single item on the QA list. This of 
> course means that current 10 upvotes system does not work as is. 
> Proposal below (feel free to hack):
> 
> -Every package has minimum 10 days of quarantee.
> -Package overstaying over a month is booted back to devel.
>   (If it is not accepted by then, it will never be)
> -Package retains its history for later viewing.
>   (Nice to see what has happened before)
> -Every package has QA list.
>   (Based the current wiki list - the testing criteria)
> -Every item on the QA list has to be examined before acceptance.
>   (The procedure should be the same for all)
> -A person can check one or multiple QA items.
>   (Do what you are good at or what you have time for)
> -Same QA list item can be checked by multiple persons.
>   (Never trust a single person, or do we? High karma -> power of two)
> -Some QA items can be checked automatically.
>   (optification for example)
> -Bugfix will reset the quarantee timer.
>   (The quarantine was clearly working, some more is therefore needed)
> -Bugfix will remove the second upvote for every QA item.
>   (Bugfix can introduce other bugs, better to be sure)
> -A qualified application is released to extras.

Looks good overall.

- If something can be checked automatically then it must be moved to the
jump between extras-devel to extras-testing, where the automated tests
are done.

- Maybe it would be good to present user karma next to their thumbs? It
would be useful to have an approximate idea of the "experience" put in
an evaluation, or at least it could be useful when someone is trying to
game the system creating a bunch of virtual users.

- Maybe 3 thumbs up in each criteria is enough? I mean, if someone with
high karma (read community reputation) says in the Power Management
entry "I have run powertop and this app is fine" next to his thumbs
up... most of us will give thumbs up only after checking that indeed our
battery is not drained after installing the app.

- If an app is dumped back to extras-devel because of a blocker in one
row, the karma there should be reseted but the other rows could just stay.


> As said, feel free to modify. First is always rough.. :)

ditto

-- 
Quim Gil
open source advocate
Maemo Devices @ Nokia