[maemo-community] Test Management tool for maemo.org

[From: Jarmo.Tikka at nokia.com]

Hi again,

There were some questions about test management tools and processes I proposed for garage.
I try to address those here.

From: timeless <timeless at gmail.com>
On Tue, Jan 27, 2009 at 3:03 PM,  <Jarmo.Tikka at nokia.com> wrote:
>> We made a short study also about Mozilla's Testopia
>
>So, I'm not very familiar w/ Test things, I've had a couple of
>unfortunate run-ins with Quality Center.

And I am familiar mostly with our (Nokia) internal test management systems not open source ones. One of the reasons I would like to have maemo community comments.

>
>I believe mozilla.org has two Test related offerings: Testopia and Litmus.
>
>This isn't unusual for mozilla.org (well, perhaps it's low, I think we
>average 3 offerings per area).

We shortly studied Testopia but it did not seem to be very widly used. Another issue with Testopia was that it integrates to the Bugzilla and I think we would like to be able to update/upgrade maemo.org bugzilla without worrying do we have working test management system available for it. At least that was one of the principles we have when we setup maemo.org services couple of years ago.
E.g. have dependencies between services only through single login...

My email starts to go off topic so back to test mangement :). Would this Litimus be good condidate (e.g. good functionality, widly used and independent) test management service?


From: Andrew Flegg <andrew at bleb.org>
>>
>> I just went this community list emails quickly through to find out has
>> community already decided to setup some generic test management service
>> for all garage projects.
>
>Have you got a URL? I don't remember the discussion.

I could not find discussion either. I proposed this issue to Niels and some other our community admins about week ago.

>
>> I noticed that there has not yet been discussion on how maemo community
>> wants to handle test management issues. So I propose that we setup
>> TestLink as common test case/plan/result management tool for maemo.org.
>
>Would maemo-developers be better? Is there really a demand?

At least I have demand for my own projects. I do not know others, comments?

>
>Wouldn't projects which require testing already have a `make test' -
>is this looking at integrating these things with the VCS for
>continuous integration?

Each project needs to have their unit tests (and is possible those unit tests runnable automatically during build time but also during integration testing). So this test management service would be as you said mainly for integration testing where your product will be tested in true runtime environment and not in development environment.

Cheers,
//Jarmo