[maemo-developers] busybox symlinks discussion

[From: David Horn]

On Mon, Dec 15, 2008 at 4:49 AM, Eero Tamminen <eero.tamminen at nokia.com> wrote:
> Hi,
>
> I put CC to maemo-devel.
>
> ext David Horn wrote:
>>
>> I just wanted to see if I could help out with the issue of busybox
>> .deb packaging, and the issue of symlinks and conflcts.
>>
>> I found the maemo-dev archives, and agree with the basic concepts of
>> keeping busybox (the binary) in one package (and thus being much more
>> likely to track upstream closely with little to no modifications), and
>> symlinks in another.
>>
>> The issue of symlinks to busybox that could be old (feature removed
>> from busybox), or missing (new feature added to busybox),
>
> Extra utilities built into Busybox without corresponding symlinks aren't
> really a problem as they take only little space (average BB applet is
> <2KB).
>
> (Features are more of a problem, but Busybox doesn't currently have
> any support for even reporting that some option for a tool wasn't
> enabled in BB configuration.  I have filed a bug about this to
> upstream Busybox BTS.)
>
>
>> or replaced
>> with a real utility came up, and I think I have a fairly robust shell
>> script that might be able to help.  I have attached a file named
>> bb_install that will clean up old symlinks, and create new symlinks
>> for busybox, but will NOT overwrite any real files, only old busybox
>> symlinks.
>
> I've been experimenting with this and actually have already
> a preliminary version of scripts that generate busybox packaging
> based on the links file generated by the busybox sources (i.e. using
> it needs a bit of iteration).  It:
> - has separate symlink mapping file (which states which symlinks belong
>  to which package and to correct directories to what Debian uses)
> - Supports multiple symlinks to same utility (Debian has couple of tools
>  with symlinks in multiple dirs and scripts can have absolute paths to
>  either of them)
> - Support alternatives (vi, awk...)
> - Adds dpkg-divert for /bin/sh, if /bin/sh is enabled in BB config.
>
> There's also a script to verify at package build time that packaging
> and BB agree on what symlinks should be generated.
>
>
>> This requires a busybox that supports the --install parameter (diablo
>> busybox does not currently)  I have a test busybox .deb (1.13.1) that
>> does work as a diablo replacement, and supports --install (attached as
>> well), and does NOT install any symlinks.  To install the new busybox
>> (attached), you will need to do the following:
>>
>> echo "#!/bin/busybox" > /usr/local/sbin/rm && busybox cp
>> /usr/local/sbin/rm /usr/local/sbin/tar && busybox chmod +x
>> /usr/local/sbin/rm /usr/local/sbin/tar && dpkg -i
>> /home/user/busybox_1.13.1-maemo4-diablo365_armel.deb && busybox rm
>> /usr/local/sbin/rm /usr/local/sbin/tar
>>
>> Since rm and tar will disappear as part of dpkg removing diablo
>> busybox prior to installing my new busybox (as already discussed, this
>> {temporary rm/tar} is a one-time need, as future busybox should track
>> upstream)
>
> If rm/tar symlinks are in separate package, temporary links
> are going to be always needed for upgrades.
>

I'm not certain I understand (or perhaps I am missing the point).  If
rm/tar symlinks ARE in the same busybox package as /bin/buybox, any
busybox package replacement  will break dpkg, and require temporary
links as a solution.

If however, there are NO symlinks in the busybox package, this seems
to work fine.  In other words, it is having the symlinks in the same
package as /bin/busybox that is at the core of the issue.  I agree
that it would be nice to have some handling of the rm/tar dpkg case
that is less likely to break moving forward.

How about 3 packages (compromise between your approach and mine) ?
(busybox, busybox-coreutils-symlinks, and busybox-symlinks)

busybox.deb package would only contain /bin/busybox  {just like
upstream debian busybox package}
busybox-coreutils-symlinks.deb would contain normal dpkg-style
symlinks for a small core set (including but not limited to rm/tar)
busybox-symlinks.deb would contain a dynamic symlink builder script,
and no dpkg-style symlinks

Anyone else have a suggestion ?

> After having tested these things a bit and briefly discussing with
> the dpkg maintainer, I had come to a conclusion that trying to support
> replacement of distribution essentials isn't a completely sane goal...
>

I'm right there with you!  Lack of sanity never stopped anyone before however ;)

> So, I changed the approach a bit so that all symlinks corresponding to
> essential Debian utilities (utilities in Debian packages marked as
> essential), would be still included into the busybox package itself.
> We just should try to make those in Busybox as compatible to GNU ones as
> possible by enabling relevant BB configuration options etc.[1]
>
> Only utilities corresponding to non-essential Debian packages would
> be in separate symlinks packages (there are a lot of those still).
>
>
> [1] If you have comments about BB utility incompatibilities that could
>    be solved just by enabling/disabling some BB option, please file
>    bugs to bugs.maemo.org and put me on CC.
>
>
>> Once install is complete, just run my bb_install script (busybox sh
>> bb_install) to setup (and cleanup if needed) the symlinks. It can be
>> run multiple times with no ill effects, and even supports side-by-side
>> busybox installs (/bin/busybox and /bin/busybox.development) with the
>> ability to switch all of the symlinks between them easily, and
>> dynamically only creating the correct symlinks that the particular
>> busybox binary supports, and without ever overwriting any real files.
>>
>> The second .deb package (named busybox-symlinks ??) could just contain
>> a script (like the bb_install file I attached), do a post-install of
>> running said script.  I have not created a busybox-symlinks package
>> yet, but wanted to see if you had any other ideas before I spent any
>> more time on this.  In this way, you do not need to create a new
>> package for EACH symlink, provided that the busybox-symlinks package
>> has its debian/control file setup with appropriate provides
>> statements, but it likely would not need the as many conflicts
>> statements since the symlinks are dynamic, and some other package (say
>> iputils-ping) could replace an individual symlink with a real file and
>> no extra effort, and without sacrificing busybox or the rest of the
>> symlinks.
>
> I think this could be a mess because the symlinks wouldn't be in
> the package database and the BB setup wouldn't correspond to any
> package configuration.  It's more like working around the package
> management system than trying to take advantage of it.
>

This much is true, but I tried to find precedent for how to handle a
multi-call binary in the debian world, and have not had a whole lot of
luck finding any good examples that help out with this scenario.
Hence, the roll-your-own approach I took.

> Tar handles extracting something on top of a symlink so that the
> busybox binary pointed by the symlink doesn't get overwritten, so
> adding real versions of stuff that Busybox handles would be fine.
> (Because symlink's not in package database, dpkg wouldn't give
> a conflict)
>
> However, getting back to a known, tested configuration could then be
> more of a hit-and-miss.  You would need to do it manually by removing
> any replacement packages you've installed and then re-running the
> Busybox symlink creation script.

Getting back to known, tested configuration always requires removing
replacement packages by it's nature.

>
> I guess SSU could automate this by trying to removing everything that
> Busybox can provide and then running the symlink script.  To prevent
> device getting unusable when the replacement packages are otherwise
> uninstalled, the symlinking script should be run always when a package
> is removed.  Apt may have a hook for this, but things could still
> break when package would be removed directly with dpkg.
>

I really can not speak to the needs of SSU, so anything that ssu needs
should be in the appropriate package (busybox-coreutils-symlinks ?)

I have not yet researched the third-party uninstall case (install
busybox symlink, install third party app that overwrites symlink, then
uninstall third party app).  Yes re-running a script can easily solve,
but this requires more thought.  Oh well, I guess the debian
maintainers guide needs some more perusal...

> You script is much simpler than mine though. :-)

If simpler == easier to maintain down the road for new Maemo releases,
then this is a good thing.   If not, then probably not ;)


--David Horn
-- 
Non quia difficilia sunt non audemus, sed quia non audemus, difficilia sunt.