[maemo-commits] [maemo-commits] r16521 - projects/haf/doc/mvo

Wed Oct 29 01:44:58 EET 2008

Author: marivoll
Date: 2008-10-29 01:44:57 +0200 (Wed, 29 Oct 2008)
New Revision: 16521

Modified:
   projects/haf/doc/mvo/system-model-2.txt
Log:
Distributions.


Modified: projects/haf/doc/mvo/system-model-2.txt
===================================================================

--- projects/haf/doc/mvo/system-model-2.txt	2008-10-28 16:48:07 UTC (rev 16520)
+++ projects/haf/doc/mvo/system-model-2.txt	2008-10-28 23:44:57 UTC (rev 16521)
@@ -13,8 +13,8 @@
   components; how the Maemo system is bootstrapped from source; how we
   import upstream sources; how we handle NMUs; etc.
 
-  This text has been written in parallel with Part I, so I had to make
-  a few assumptions about Part I that might turn out to be wrong.  The
+  This text has been written before with Part I, so I had to make a
+  few assumptions about Part I that might turn out to be wrong.  The
   assumptions are:
 
   - We are talking about the "Maemo" system, from its Harmattan
@@ -32,8 +32,7 @@
 
   - Maemo contains the "Maemo SDK": the tools needed to develop the
     Maemo OS and the add-ons: build tools, documentation tools,
-    packaging tools, image creation tools, pre-loaded contents,
-    customization package templates, etc.
+    packaging tools, image creation tools, etc.
 
   - Maemo is developed by multiple 'non-consenting' parties.
     Different parties work on different parts but they all work on the
@@ -48,7 +47,6 @@
     binary packages.  That's the interface between Part I and Part II
     of the Maemo system model.  I assume that Debian binary packages
     have already been introduced as part of the compilation process.
-
 ]
 
 * The Maemo archive
@@ -64,7 +62,7 @@
 Packages that are not in the Maemo archive are not part of Maemo and
 are shot on sight.
 
-All packages in the Maemo archive must work well with each other and
+All packages in the Maemo archive should work well with each other and
 in all possible configurations and targets, not just the officially
 supported configurations and targets.  They do this by declaring
 appropriate dependencies and conflicts with other packages.
@@ -75,11 +73,17 @@
 must be prevented from being installed, the reasons need to be
 identified and expressed with dependencies and conflicts.  
 
+There can only be a single source package in the whole Maemo archive
+with a given name and version; and only a single binary package with a
+given name, version, and architecture.  A given version of a package
+can be in more than one distribution at the same time, of course, but
+it must be the exact same ".dsc" or ".deb" file.
+
 ** Categories
 
 The packages in the Maemo archive are divided into one or more
-"Categories".  The categories are used to control uploads to and
-downloads from the archive.
+"Categories".  The categories are used to control uploads
+to and downloads from the archive.
 
 For example, only Nokia is allowed to upload to certain categories,
 and you need a Maemo Garage account to upload to others.  Packages in
@@ -175,29 +179,143 @@
 of development separate from each other.  There is a 'stable' line of
 development that has been released for general consumption and only
 receives bug fixes.  There is also a 'testing' line that contains beta
-releases for daring testers.
+releases for developers and daring testers.
 
 Packages can be copied from one distribution to another in order to
-release them.  Renaming or copying a whole distribution is an
-important special case of this.
+release them.  When copying packages, one or more source packages are
+copied together with all their binary packages.
 
-[ Let's try to come up with a concrete list or scheme how to publish
-  releases.  "harmattan-devel" and "harmattan-1", "harmattan-2",
-  "harmattan-3", etc?
+The distributions for Harmattan are "harmattan", "harmattan-testing",
+and "harmattan-proposed-updates".
+
+In detail:
+
+ - harmattan
+
+ This distribution contains the current release of harmattan.  Devices
+ of normal users use this distribution.
+
+ - harmattan-testing
+
+ This distribution contains the current state of the upcoming release
+ of Harmattan.  This is the build environment for most newly uploaded
+ source packages.  Developers use this distribution in the SDK.  Beta
+ testers use this distribution on their devices.
+
+ - harmattan-proposed-updates
+
+ This distribution is used to prepare small updates to packages in the
+ harmattan distribution that can not wait until the package in
+ harmattan-testing is of sufficient quality to be released.
+
+ In other words, harmattan-proposed-updates is mainly for preparing
+ security fixes.  It should be empty except for very short periods of
+ time.
+
+*** Compatibility
+
+Interface management will make the harmattan-testing distribution
+compatible with the harmattan distribution in a desired way.  Most
+packages compiled in harmattan-testing should be installable in
+harmattan.  Exceptions are allowed, but only by plan, not by accident.
+Compatibility requirements will be especially relaxed before the first
+release of harmattan.
+
+Compatibility is important because packages (and groups of packages)
+are released independently.  Consider a typical 3rd party application
+in the extras category: they are meant to be installed 'immediately'
+on devices that run the current harmattan distribution, not on devices
+that run harmattan-testing.  Still, they are compiled in a
+harmattan-testing environment.
+
+Problems with this will arise when a binary package automatically
+picks up a incorrect run-time dependency that is not satisfiable in
+harmattan.  The correct dependency would be satisfiable in harmattan,
+but the build tools used to compute it will not produce it.
+
+These problems need to be solved by fixing things so that the correct
+run-time dependencies are computed during build time.
+
+[ In other words, problems will arise when the interface management is
+  too conservative.  We will make unrelated packages suffer in order
+  to find these flaws, but I think it is worth it.  Putting such a big
+  responsibility on interface management is a good thing, IMO, and I
+  am confident that all flaws can actually be fixed.  We will be
+  forced to get a very firm grip on the compatibility between one
+  release and the next.
+  
+  As a concrete and very relevant example, consider dpkg-shlibdeps and
+  friends.  This is the machinery that computes the ${shlibs:Depends}
+  substitutions that are used to automatically pick up the right
+  dependencies on shared library packages.
+
+  This is done by having the tools take a close look at the symbols
+  that are actually used from the shared library.  Then the lowest
+  version of the library is found that exports that set of symbols.
+  In this way, the run-time dependencies are very accurate and not too
+  conservative.  We should take this system into use.
 ]
 
-There can only be a single source package in the whole Maemo archive
-with a given name and version; and only a single binary package with a
-given name, version, and architecture.  A given version of a package
-can be in more than one distribution at the same time, of course, but
-it must be the exact same ".dsc" or ".deb" file.
+*** Releasing
 
-** Architectures
+Packages are released by copying them in groups from harmattan-testing
+to harmattan.  For a simple application in extras, this can be a
+single package, which is copied under control of the original
+uploader.  For a big module like the OS, this will mean copying a huge
+set of packages under control of the release manager.
 
-If a package needs to be compiled for different target hardware
-architectures at the same time, a new architecture needs to be added
-to the Maemo archive.
+When copying a group of packages in this way, the harmattan
+distribution must not break, of course.  Most run-time and build-time
+dependencies between packages should still be satisfiable, and those
+that break should be identified and the breakage approved.
 
+[ Breaking harmattan might be considered OK when we release a OS
+  version that is incompatible with a 3rd party application.  Such an
+  application should ultimately not be able to delay the OS release.
+  This should be the exception and not the rule, of course.
+]
+
+Note that the SDK does not need to be released: harmattan-testing _is_
+the SDK distribution.  However, SDK packages should be copied to the
+harmattan distribution as well so that that distribution is always
+self-contained and recompilable.  This is part of the "do not break
+harmattan" rule: The harmattan distribution is also considered broken
+when its packages can not be recompiled.  This is important when using
+harmattan-proposed-updates for quick updates.
+
+*** Urgent updates
+
+The normal "Don't Panic" development process builds packages in
+harmattan-testing and slowly releases them to harmattan when they are
+ready.
+
+The "Panic Now" process for releasing security fixes and other urgent
+updates uses harmattan-proposed-updates in the following way.  The
+updated source package is uploaded to harmattan-proposed-updates and
+the responsible buildbot compiles it in a harmattan environment.  The
+update is tested by installing harmattan + harmattan-proposed-updates
+on a device.  If the update is considered releasable, the content of
+harmattan-proposed-updates is copied into harmattan and
+harmattan-proposed-updates is emptied.  If the update is not
+releasable, harmattan-proposed-updates is emptied as well, and the
+process starts over.
+
+It is important that harmattan-proposed-updates is almost always
+empty.  It would not be entirely inappropriate to send an email to
+everyone once per hour or so whenever there is a package in
+harmattan-proposed-updates.
+
+*** Continuity
+
+The Maemo archive can and should contain the distributions for
+the projects following Harmattan.
+
+When starting the "Isnogud" project (say), the isnogud,
+isnogud-testing, and isnogud-proposed-updates distributions are added
+to the Maemo archive.  They are created by copying the equivalent
+Harmattan distributions.  Development then starts in isnogud-testing,
+without much regard to compatibility to isnogud if so desired.
+
 ** Repositories
 
 The Maemo archive is published in one or more "Repositories".  A
@@ -220,27 +338,36 @@
 package management tools as well.  Some packages might be used to
 create specialized images, like the kernel image that is flashed into
 its own partition without a filesystem.  Some packages might be used
-to pre-load files into the prepared root filesystem.  These files are
-then not subject to package management at run-time, which might be
-appropriate for things like pre-loaded images and videos.
+to pre-load files into the prepared root filesystem, or into other
+filesystems.  These files are then not subject to package management
+at run-time, which might be appropriate for things like pre-loaded
+images and videos.
 
 The Maemo SDK runs in Scratchbox targets.  It is installed by
-extracting a 'rootstrap' tarball into the target.  The tarball is also
-prepared from packages in the Maemo archive (and nothing else).
+extracting a 'rootstrap' tarball into the target.  This rootstrap
+tarball is also prepared from packages in the Maemo archive (and
+nothing else).
 
 The tools for creating the installation images and rootstrap tarballs
 are contained in the Maemo archive and are maintained like any other
 component.
 
 [ I hope we are rid of Scratchbox devkits and toolchains in Harmattan.
+  I think that the Maemo SDK+ is still much too complicated with too
+  much magic, but that's a separate discussion.
 ]
 
 Both the Maemo OS in a device and the Maemo SDK in a Scratchbox target
-are maintained by installing packages from the Maemo archive (and
-removing already installed packages) with the package management
-tools.
+are maintained by installing packages from the Maemo archive and
+removing already installed packages with the package management tools.
 
 * Configurations
 
 - "Modules"
+
 - User controlled configuration
+
+- Forced chaining of module updates.  Not done with distributions.
+  Rather, done with a new feature of apt-get or the Application
+  manager: "Maemo-Upgrade-Only-From:" plus forcing of suitable
+  candidates for "=" dependencies.


