[maemo-developers] Extras-testing improvements

[From: Niels Breet]

On Tue, March 9, 2010 10:02, Sascha Mäkelä wrote:
> I would like to see more interaction between the dev and the Testing
> Squad.
> In the current system after the dev promotes the package to testing, all
> he can do is to hope that everything is OK. If it's not, he has to promote
> new package and everything starts from zero again. I don't think this good
>  enough for anyone involved.
>

This is how testing works in every project. A test succeeds or a test
fails. If it fails, the developer needs to fix it. Lathe, rinse, repeat.

> What I would prefer would be a system where, testing is an ongoing
> process. That is, the dev can make fixes/improvements according to the
> bugs/suggestions reported by the Testing Squad and submit the updates
> often/early, without things starting from zero again. After the Testing
> Squad and the dev is happy with it, then the package could be locked and
> the short quarantine time (like 3 days, for example) would start.
>

All testing is done in public, so it is a transparent process. The testers
should test for the requirements as specified, if the requirements are met
the app get a vote up.

The quarantine time is there to have a certain period of time to find
critical bugs. A simple test might not find them but we hope that if
people are using the app for a while, this might become more clear. The
quarantine is the last step before it will be installed on thousands of
phones around the world.

We have to be serious about quality.


> With current system either the package fails completely or
> the improvements will have to wait for the next update. This is of no
> benefit to anyone. The end users won't have as good apps as they could
> have. The Testing Squad should be renamed to Border Control, because they
> can't really improve any app. They can only block them. And finally,
> instead of feeling a sense of accomplishment for contributing to the
> community, the devs often feel just frustrated.
>

QA and testing is not about devs. It is a necessary 'evil' developers need
to go through to assure that end-users have the least amount of problems.

We need to have a certain level of quality as a community. If you are not
willing to strive for that, then Extras might not the place for you to
publish your software.

That said, we can always improve the QA process. Suggestions are welcome.
It is not our plan to make your life as miserable as possible :)


> Cheers,
>
>
> Sascha
>

--
Niels Breet
maemo.org webmaster