[maemo-community] Extras and Fremantle

[From: Graham Cobb]

On Friday 27 February 2009 08:47:08 Jeremiah Foster wrote:
> I feel the only way to combat this type of rogue repository creation
> is to assure that the maemo.org repos are;
>
> 	1. of significantly higher quality
> 	2. provide added value, like the way they are highly available
> 	3. are more secure

Hi Jeremiah

Great to have you on board!  Now that you have had a chance to get your feet 
under the desk I think it would be great if you could lead the process of 
defining what our repository structure should look like for the next year or 
so (i.e. Freemantle).  Of course, it is a community process but having a 
Debmaster on board means we have someone with expertise to help lead us and 
with some time to implement things!

I think most people would agree that when we set up the current structure 
(Extras and extras-devel) it was primarily to address the "repository hell" 
we had previously: every developer had their own repository so users didn't 
know where to find things, some packages (mainly ported from Debian) were in 
multiple repositories (with different versions and different build options), 
repositories would die and their contents would be lost, etc.

The extras/extras-devel structure has definitely significantly improved 
things: there is one place for users to go for production versions, there is 
one place for developers to put test versions, there is only one copy of each 
package, it is possible to automatically rebuild against new SDKs, etc.  

Of course there are some problems as well.  One of the main problems is 
quality.  It is a really hard problem not only because assuring and testing 
quality is hard but also because the right policy is unclear.  Before we put 
in place tools and processes to enforce quality rules we have to make sure we 
understand what the policy is that we are trying to implement.

At first sight it appears obvious: we should ensure that packages cannot get 
into the repositories unless they have sufficient quality (for some value 
of "sufficient").  But, on the other hand, if we set the bar too high we are 
in danger of recreating the repository hell: developers who cannot meet the 
bar (early versions, not enough time to improve quality, not enough time to 
prove the quality, not enough users to test and evaluate it) will end up 
having to create their own repositories again.  Personally, I consider that a 
worse problem than the (current) quality problem (others may disagree).

To illustrate the problem let me use some examples of my own packages:

GPE is a reasonably major set of applications for the tablet; fairly widely 
installed; several developers; reasonable quality; active bug tracker and 
some bugs even get fixed! I currently have a stable version in Extras, a beta 
version in extras-devel (which I intend to move to extras soon, to replace 
the current stable version) plus I have two other private repositories: one 
for the GPE development team which just builds from SVN every night, with no 
testing; the other for my own use, used primarily for installation testing.  
I do not put new versions even in extras-devel unless I have had a few people 
from the GPE team test that it installs and does not crash. This seems to 
work reasonably well except several end-users wish to use the Beta version 
(and I want them to, to get testing before promoting it to extras) and they 
find that if they enable extras-devel they are in danger of installing all 
sorts of other packages of very variable status and possibly breaking their 
tablet.

Xsisusb, on the other hand, is a very small project.  It has at most 4-5 users 
and it is really a proof-of-concept which requires more work before it is 
usable.  I do have packages in extras-devel but none in extras.  It is 
announced in ITT but does not have a download page: only hard-core hackers 
are likely to want this so that works fine.  On the other hand, if there was 
a quality bar for entry to extras-devel this package would certainly fail it 
and I would need to create my own repository.

OpenSync is a "real" project: several developers; related to major projects 
like KDE and packaged in Debian; I released previous versions for Gregale and 
Bora. On the other hand, OpenSync is currently being re-written and the 
current version is still very broken; definitely pre-alpha.  I have not even 
put it into extras-devel.  For this I have a personal daily build repository 
(for use mainly by me but also a couple of other people who are very keen to 
help with getting it working).  Once we get something which at least "works" 
at some basic level I will want to put an "alpha release" somewhere for the 
small team to work from.  I intend that to be extras-devel (just like for 
Xsisusb).  Again, it would definitely fail any quality metrics and is not 
intended for anyone to install unless they are actively in discussions with 
me or other members of the OpenSync team.

These are just three examples of projects in different stages and of different 
sizes (in terms of code and of users).  There are many other examples, 
different again.  So, I would be really interested to hear proposals from you 
(and anyone else) on how we should evolve our repository structure to support 
all these and minimise the need to attach packages to ITT posts or create 
personal repositories (although I believe they do have a legitimate role for 
personal use and for development team use).  Is it time for us to move to a 
Debian style stable/testing/unstable/experimental structure?  Can we do 
something clever with control file information to have the user select more 
or less safe versions of things to install?

Graham