[maemo-developers] How to use extras-testing correctly?

[From: Marius Vollmer]

2009/9/24 Jeremiah Foster <jeremiah at jeremiahfoster.com>:
>
> All community developed projects have a Quality Assurance process. Debian's,
> which is what Maemo is based on, looks like this;
>
> 1. Submit your app to autobuilders
> 2. If it fails to build from source, start over at 1
> 3. If it builds, it stays in the new queue for ten days
> 4. After ten days it automatically gets promoted to 'testing'
> 5. After (roughly) eighteen months, testing is frozen (no more new apps)
> 6. After all Release Critical bugs are fixed, testing becomes stable

I think it is important to mention that Debian is much more careful when
giving upload rights to people.  It takes quite some dedication and time
(months) to even get to step 1 for the first time.

Also, packages don't get promoted from the NEW queue to testing, they
get promoted from unstable.  The very first upload of a package has to
pass the NEW queue before it goes to unstable.  Subsequent uploads of
the package go directly to unstable.  (This is just a detail.)

Also also, there are many more details to promotion than just waiting
ten days, of course.  The salient point is that promotion happens by
default unless it is stopped because of bugs, instead of being stopped
by default unless there is enough karma to allow it.  (This is not just
a detail.)

> Maemo is trying to innovate and crowd source the quality control and
> shrink that time as much as possible.

The big fundamental difference between Debian and Maemo Extras is that
Debian is producing one big release of a complete integrated
distribution, while Maemo Extras is a collection of mostly independent
applications that are released independently.  Maemo Extras is not
collectively frozen at any point.  (It will cool down as people lose
interest in it, but nobody is waiting for a stable release of Maemo
Extras as a whole.)

Maemo as a whole (including the platform produced by Nokia, applications
that are part of Nokia's releases such as Email and Sketch, and
independently developed add-ons such as Maemo Extras) should innovate
beyond Debian by defining a release process that allows multiple,
largely independent 'modules' in the distribution that are released
independently.

I would like to hear announcements like "Modest 4.0 has entered Maemo
stable".  I would also like to hear this for Debian, actually, I don't
think its a feature that all of Debian freezes at the same time.

Some time ago, I tried to sketch a setup that might allow this (if it
works):

    https://stage.maemo.org/svn/maemo/projects/haf/doc/mvo/system-model-2.txt

Now, what are the different flavours of Maemo Extras intended for?  I
would say:

 - extras-devel

   The development environment.  All new uploads are build in
   extras-devel.  Developers have extras-devel configured on their
   development boxes (in Scratchbox) and the buildbot installs build
   dependencies from it.

   Everybody subscribed to maemo-developers should have it configured on
   their devices (and should have a soldering iron ready to fix things
   when they break).

   Packages are allowed to enter extras-devel when they build and do not
   break the build environment.  It is OK if they can't be installed or
   make other packages uninstallable.

   Extras-devel is not for dangerous experiments, it is only for serious
   release candidates.  You should always be prepared that something
   that you upload to extras-devel reaches extras in a matter of days
   and will show up on end-user devices.

 - extras-testing

   Smoke testing.  Everybody who considers him/herself to be part of the
   Maemo community, everybody who attends the Maemo summit, everybody
   subscribed to maemo-users is expected to have extras-testing
   configured on their devices.

   As soon as you know that extras-testing exists, you should seriously
   consider using it.  You should tell your SO to use it.  Everyone who
   has a channel to contribute feedback should be invited.

   Packages can move to extras-testing when they are installable and no
   other package in extras-testing becomes uninstallable, among other
   criteria.

   It is important that also extras-devel can be used for beta testing
   as well, by developers.  As soon as something is in extras-devel,
   people should be able to give feedback about it in the same way that
   they would do it once the package is in extras-testing.

 - extras

   Our end users.  People who got conned into buying a two year
   subscription plan by an operator and who don't know the difference
   between S60 and Maemo, or for that matter, between Motorola and
   Nokia.


A drawback of this setup might be that you can only opt into being a
smoke tester for the whole of Extras, not for a specific application.  I
dont think this is actually a problem, since smoke testing is not beta
testing.

If you want to release a beta of something, we can try to hold it in
extras-testing so that end users are never tempted to install it.
However, sometimes you want to reach end user with your beta test as
well.  Just using a package display name of "Email - beta" might be
enough, or we could think about putting a more formal mechanism into the
Application manager UI.

My point being that a beta test of Foo is in its own package, "foo-2
2.0~beta2", not a new version of the old "foo" package.


Upps, that was way longer than I intended.  Hope you find something
useful in it.


PS: "crowd source"?