[maemo-developers] Testing marathon & Q&A Feedback

[From: Attila Csipa]

On Sunday 01 November 2009 15:20:07 Valerio Valerio wrote:
> as some know yesterday we did a Extras-testing marathon, in order to
> get more applications ready to Extras but also to test the Q&A process

Just to chime in, thanks to all the folks taking the trouble to test apps they 
might personally not even be interested in. Special thanks to Valerio for 
spearheading/moderating the effort. He also listed most points that caused 
delays and grief during testing, I'd like to add only a few minor points:

- The icon-cache (aka free space) leak bug make testing really difficult for me. 
I don't have too much free space so had to reboot every 15 minutes. Not fun. I 
see this as a serious problem for end users, those who agree please vote bug 
up. https://bugs.maemo.org/show_bug.cgi?id=5450

- Too many applications compared to time and available testers. The effort was 
valiant, but even so we hardly went through the first page, and even there, 
plenty of apps did not get 10 votes (regardless of up or down). We need more 
people doing this, so the effort could be split (e.g. tester group A checks one 
app while tester group B checks another).

- The approval process could have been more interactive - often the developers 
were right there, but because of the time of getting through the autobuilder, 
tester feedback response resulted only in a bug report, even though we could 
have had 'fixed' applications on the spot (online hackfest if you wish) - and 
appropriate votes to confirm the changes.

- The discussed fix-or-promote dilemma. I promoted the latest version of my 
mini-app to testing in anticipation of larger tester response (previous 
version already had 7 karma at that point). Now the app has 4 karma and no new 
feedback. It's hard to recommend to people to upload newest versions for 
testing when that means they go to the end of the test queue.

- Does this bring karma ? I sure think serious testing is worth a LOT and do 
not see that adequately reflected in karma. Could also be motivating in 
bringing more testers to the table. Ways/ideas of making this abuse-proof are 
welcome.

Regards,
Attila