[maemo-commits] [maemo-commits] r16553 - projects/haf/doc/mvo

[From: subversion at stage.maemo.org]

Author: marivoll
Date: 2008-11-01 22:28:21 +0200 (Sat, 01 Nov 2008)
New Revision: 16553

Modified:
   projects/haf/doc/mvo/system-model-2.txt
Log:
Integration snapshots, releasing.


Modified: projects/haf/doc/mvo/system-model-2.txt
===================================================================
--- projects/haf/doc/mvo/system-model-2.txt	2008-10-31 16:47:20 UTC (rev 16552)
+++ projects/haf/doc/mvo/system-model-2.txt	2008-11-01 20:28:21 UTC (rev 16553)
@@ -28,7 +28,7 @@
     optional add-on "modules" that run on the Maemo OS or enhance it.
     The current "Extras" repository is part of Maemo, for example, and
     applications like the Browser or EMail are not necessarily part of
-    the "OS" module.
+    the OS module.
 
   - Maemo contains the "Maemo SDK": the tools needed to develop the
     Maemo OS and the add-ons: build tools, documentation tools,
@@ -38,7 +38,8 @@
     Different parties work on different parts but they all work on the
     single Maemo project.  Nokia is one party, the rest forms the
     Maemo community.  Nokia gets some special treatment as the 'owner'
-    of the Maemo project, but not much.
+    of the Maemo project and the maintainer of large and important
+    parts of it, but not much.
 
   - Ultimately, all our packages are published, with or without
     source, with or without explicit license agreements.  Things that
@@ -51,14 +52,17 @@
     binary packages.  That's the interface between Part I and Part II
     of the Maemo system model.  I assume that Debian binary packages
     have already been introduced as part of the compilation process.
+
+  A useful model is to think of the Maemo archive as a source code
+  revision repository: it might have a commit history, tags, branches,
+  merges, and so on.  It would be interesting to explore this analogy
+  fully: storing the package lists that define a distribution in a VCS
+  and use VCS operations to maintain them.
+
 ]
 
 * The Maemo archive
 
-[ This section can be chapter 2 of the Maemo policy document, maybe
-  after condensing it a bit. 
-]
-
 Maemo is maintained and distributed as a collection of packages.  All
 packages, both source packages and the binary packages that have been
 compiled from them, are contained in the "Maemo archive".
@@ -89,11 +93,11 @@
 "Categories".  The categories are used to control uploads to and
 downloads from the archive.
 
-For example, only Nokia is allowed to upload to certain categories,
-and you need a Maemo Garage account to upload to others.  Packages in
-most categories are freely distributable in source and binary form,
-but people need to explicitly agree to certain conditions before they
-are allowed to download from other categories.
+Only Nokia is allowed to upload to certain categories, and you need a
+Maemo Garage account to upload to others.  Packages in most categories
+are freely distributable in source and binary form, but people need to
+explicitly agree to certain conditions before they are allowed to
+download from other categories.
 
 The categories are:
 
@@ -149,17 +153,13 @@
 
 ** Distributions
 
-Most packages in the Maemo archive are part of one or more
-"Distributions".  The distributions are used to keep different lines
-of development separate from each other.  There is a 'stable' line of
-development that has been released for general consumption.  There is
-also a 'testing' line that contains beta releases for developers and
-daring testers.
+Packages in the Maemo archive are part of one or more "Distributions".
+The distributions are used to keep different lines of development
+separate from each other.  There is a 'stable' line of development
+that has been released for general consumption.  There is also a
+'testing' line that contains beta releases for developers and daring
+testers.
 
-Packages can be copied from one distribution to another in order to
-release them.  When copying packages, one or more source packages are
-copied together with all their binary packages.
-
 The distributions for Harmattan are "harmattan-stable",
 "harmattan-testing", and "harmattan-proposed-updates".
 
@@ -171,16 +171,11 @@
  of normal users use this distribution.
 
  Packages in harmattan-stable should be installable: all their
- run-time dependencies should be available.
+ run-time dependencies should be available, among other things.
 
  Packages in harmattan-stable should be rebuildable: all their build
- dependencies should be available.
+ dependencies should be available, among other things.
 
- Only one version of a package can be in harmattan-stable.  If a new
- version of a package enters harmattan-stable, the existing version is
- removed.  [Except for modules that use the "upgrade-only-from"
- feature... hmm.]
-
  - harmattan-testing
 
  This distribution contains the current state of the upcoming release
@@ -192,10 +187,6 @@
  if dependencies are missing, just as it is OK to have other critical
  bugs that prevent their release.
 
- More than one version of a package can be in harmattan-testing.  See
- the section "Expiration" below for how to keep the size of the Maemo
- archive under control.
-
  - harmattan-proposed-updates
 
  This distribution is used to prepare small updates to packages in the
@@ -253,40 +244,25 @@
   conservative.  We should take this system into use.
 ]
 
-*** Releasing
+*** Integration snapshots
 
-Packages are released by copying them in groups from harmattan-testing
-to harmattan-stable.  For a simple application in extras, this can be
-a single package, which is copied under control of the original
-uploader.  For a big module like the OS, this will mean copying a huge
-set of packages under control of the release manager.
+A "integration snapshot" is short-lived distribution that has been
+created by combining packages from a number of distributions.  This is
+typically done to test the result of integrating a new set of packages
+into some distribution without actually doing it.
 
-[ This is different from Debian, where the whole of testing is
-  released as a unit.  We don't do that since we want to release
-  'modules' independently.  See the "Configuration" section below.
-]
+At the point of creation a intergration snapshot is defined by a list
+of distributions, plus a list of packages to take from each
+distribution.  It is never modified after creation.
 
-When copying a group of packages in this way, the harmattan-stable
-distribution must not break, of course.  Most run-time and build-time
-dependencies between packages should still be satisfiable, and those
-that break should be identified and the breakage approved.  See the
-"Snapshots" section below for how to test whether harmattan-stable would
-break without actually running the risk of breaking it.
+Integration snapshots have names of the form "DISTRIBUTION_ID" where
+DISTRIBUTION is the name of the distribution in the list that has the
+oldest packages.
 
-[ Breaking harmattan-stable might be considered OK when we release a
-  OS version that is incompatible with a 3rd party application.  Such
-  an application should ultimately not be able to delay the OS
-  release.  This should be the exception and not the rule, of course.
-]
+The proto-typical use of a integration snapshot is to create a release
+candidate of a configuration module such as the Operating System.  It
+could be named "harmattan-stable_RC-OS-2008.51.16", for example.
 
-Note that the SDK does not need to be released: harmattan-testing _is_
-the SDK distribution.  However, SDK packages should be copied to the
-harmattan-stable distribution as well so that that distribution is
-always self-contained and recompilable.  This is part of the "do not
-break harmattan-stable" rule: The harmattan-stable distribution is
-also considered broken when its packages can not be recompiled.  This
-is important when using harmattan-proposed-updates for urgent updates.
-
 *** Urgent updates
 
 The normal "Don't Panic" development process builds packages in
@@ -294,28 +270,39 @@
 they are ready.
 
 The "Panic Now" process for releasing security fixes and other urgent
-updates uses harmattan-proposed-updates in the following way.  The
-updated source package is uploaded to harmattan-proposed-updates and
-the responsible buildbot compiles it in a harmattan-stable
-environment.  The update is tested by installing harmattan-stable +
-harmattan-proposed-updates on a device.  If the update is considered
-releasable, the content of harmattan-proposed-updates is copied into
-harmattan-stable and harmattan-proposed-updates is emptied.  If the
-update is not releasable, harmattan-proposed-updates is emptied as
-well, and the process starts over.
+updates uses harmattan-proposed-updates in the following way:
 
+ - The harmattan-stable distribution is frozen: no new packages can
+   enter it except from harmattan-proposed-updates.
+
+ - The updated source package is uploaded to
+   harmattan-proposed-updates and the responsible buildbot compiles it
+   in a harmattan-stable environment.
+
+ - The update is tested by making a integration snapshot of
+   harmattan-stable and harmattan-proposed-updates.
+
+ - If the update is considered releasable, the content of
+   harmattan-proposed-updates is copied into harmattan-stable.
+
+ - The harmattan-proposed-updates distribution is emptied and
+   harmattan-stable is unfrozen.
+
 It is important that harmattan-proposed-updates is almost always
-empty.  It would not be entirely inappropriate to send an email to
-everyone once per hour or so whenever there is a package in
-harmattan-proposed-updates.
+empty.  The harmattan-stable distribution is automatically frozen
+whenever harmattan-proposed-updates is non-empty.
 
-*** Snapshots
+[ That is, we use locking to avoid conflicts here, like pre-CVS
+  revision control systems.  This is appropriate for high-priority,
+  urgent updates, I'd say.
+]
 
-The harmattan-testing distribution changes frequently as developers
-upload new versions of their packages.  When testing certain aspects
-of the harmattan-testing distribution
+Care must be taken that the changes that enter harmattan-stable via
+harmattan-proposed-updates are also included in harmattan-testing.
 
-[ The Maemo archive as a revision control system... ]
+[ Fixes on your release branch must be included in your development
+  branch as well.
+]
 
 *** Continuity
 
@@ -390,7 +377,7 @@
 
 [...]
 
-* Installation targets
+* Installation
 
 The Maemo OS runs on a number of devices.  It is installed via one or
 more images that are prepared from packages in the Maemo archive (and
@@ -425,13 +412,74 @@
 are maintained by installing packages from the Maemo archive and
 removing already installed packages with the package management tools.
 
-* Configurations
+* Releasing
 
-- "Modules"
+Packages are released by copying them in groups from harmattan-testing
+to harmattan-stable.  For a simple application in extras, this can be
+a single package, which is copied under control of the original
+uploader.  For a big module like the OS, this will mean copying a huge
+set of packages under control of the release manager of that module.
+Multiple modules can be released together: when the OS is released,
+important applications like the Browser are usually released with it.
+However, the Browser can also be released independently inbetween OS
+releases.
 
-- User controlled configuration
+[ This is different from Debian, where the whole of testing is
+  released as a unit.  We don't do that since we want to release
+  'modules' independently.
+]
 
-- Forced chaining of module updates.  Not done with distributions.
-  Rather, done with a new feature of apt-get or the Application
-  manager: "Maemo-Upgrade-Only-From:" plus forcing of suitable
-  candidates for "=" dependencies.
+When copying a group of packages in this way, the harmattan-stable
+distribution must not break, of course.  Most run-time and build-time
+dependencies between packages should still be satisfiable, and those
+that break should be identified and the breakage approved.
+
+[ Breaking harmattan-stable might be considered OK when we release a
+  OS version that is incompatible with a 3rd party application.  Such
+  an application should ultimately not be able to delay the OS
+  release.  This should be the exception and not the rule, of course.
+]
+
+A integration snapshot can be used to test how the new
+harmattan-stable would look like.  It would be made from all packages
+in harmattan-stable and the packages that are to be released from
+harmattan-testing.  Once testing is done, one must check whether the
+snapshot is still uptodate: packages of the snapshot that matter for
+the release must not have changed in harmattan-stable in the mean
+time.  When the snapshot is no longer uptodate, a new release
+candidate has to be made.  This should not happen very often since
+releases of modules should be coordinated.  E.g., the release manager
+of the Browser should not be surprised by a new version of the OS
+suddenly appearing in harmattan-stable.
+
+[ In other words, we must detect conflicts and merge the new changes
+  until the conflicts are gone, like with CVS, Subversion etc.  
+]
+
+Note that the SDK does not need to be released: harmattan-testing _is_
+the SDK distribution.  However, SDK packages should be copied to the
+harmattan-stable distribution as well so that that distribution is
+always self-contained and recompilable.  This is part of the "do not
+break harmattan-stable" rule: The harmattan-stable distribution is
+also considered broken when its packages can not be recompiled.  This
+is important when using harmattan-proposed-updates for urgent updates.
+
+** Upgrade chaining
+
+Some modules are only supported to be upgraded on top of the
+immediately preceeding released version of that module.  People should
+not be allowed to skip released versions when upgrading.
+
+Forcing a chain of upgrades like this can be done by creating a new
+distribution for each release and changing the device configuration to
+point to the next distribution after an upgrade.  This gets unwieldy,
+however, in the presence of independently released modules and
+increases the complexity too much for something that should be the
+exception rather than the rule.
+
+Instead, a chain of upgrades is enforced by the package management
+tools on the device.  A package can request such a chain with the
+"Maemo-Upgrade-Only-From" field in its control file.  This field
+influences which version of a package is the installation candidate;
+packages whose Maemo-Upgrade-Only-From field is not satisfied are
+ignored by the package management tools.