[maemo-developers] Testing marathon & Q&A Feedback

[From: Valerio Valerio]

Hey,

as some know yesterday we did a Extras-testing marathon, in order to
get more applications ready to Extras but also to test the Q&A process
elaborated by the community. We had more than 60 testers (around 40
all the time), but not all actively participated, was good to see some
newcomers participating and learning more about Maemo :).
The testing was more slow than I expected, In about 5 hours, we tested
around 45 apps, but we write a lot of comments and bug reports that
will help to improve the tested applications for sure.
The major difficult that we faced was our maemo.org infrastructure,
most of the time was impossible to vote or comment a app, this was
very frustrating.

A lot of feedback was gathered around the Q&A process and the packages
interface, I'll name some of the improvements discussed below, if you
attended the marathon please report your experience as well, and the
help us improve the processes.

Package Interface:

- Developers should be able to remove their own apps from testing (at
lest the entry in the package interface) - We found some apps where
the developer explicit says to not vote the app, this helps to waste
more time.
- Voters should be able to change their votes - Mistakes happen,
sometimes we voted down a app, then the developer replies to our
comments, and we see that we did a mistake, but we can't change our
vote anymore.
- Only allow one vote per person - I think this is partially fixed,
but due to the lack of response from the servers, I saw a lot of
repeated votes and comments in some apps.
- Keep app karma - already discussed in other thread.
- Keep old comments - Put some kind of separator in the comments or
add a link to the old comments. It's easier to test based in the old
comments, we can test the major blockers first.
- Subscribe to comments option - Is really annoying to keep checking
the app comments page, seeking for a reply to our comments.

Q&A:

- THe Bug tracker shouldn't be a blocker, if their's a way to contact
the maintainer in the packages interface - A lot of developers don't
want to have a bug tracker for very small applications and they prefer
to receive the bug reports by e-mail, of course we should encourage
them to add their "bug contact" in the correct field, but almost
everyone agreed that shouldn't be a blocker.
- What to do with command line applications ? - I saw the reaction of
the testers when they installed some command line apps (Where's the
app icon ? Is this thing broken ? How to use it ?). IMO those are
'hackers' apps and an hacker will know how to install them, do we
really want these tool under /utilities or /network mixed with other
end users apps ? (IMO they should have their own category, if
/development doesn't fit here)

In general I strongly believe that we're in the right path, if we want
quality applications, we need to follow some criteria and we need to
respect the rules, same criteria for everyone, this is not a
popularity contest.
The only blocker that I see right now is the miss features in the
packages interface, as some already reported in other thread, is
really annoying start from zero everytime that a small fix is applied
to a app.
Improvements to the package interface is for me the highest priority
task that we've right now, this really need to be improved.


Best regards,

-- 
Valério Valério

http://www.valeriovalerio.org