[maemo-developers] maemo-developers Digest, Vol 59, Issue 25

[From: urho.konttori at gmail.com]

> > Also, the current model of centralized gigantic repository does not

> > scale up too well. Just look at the state of using extras-devel is on

> > the current devices (hint: slow pain).



> [ Urho, thanks for this opportunity to talk about how we want to make

> package management kick ass again. You guys might think we have

> arranged this (Urho sits two offices down the corridor from me), but

> we really haven't!

> ]



> I don't think the centralized nature is a problem here. Our current

> processes and implementations might not scale well enough, but getting

> rid of a centralized repository will not help much, if at all.

If we would actually start supporting repository delta files, then you  
would be at least partially right.
At the moment, one of the big issues in scaling repositories is the sheer  
size of the databases. Sure, we do ok now, but imagine the iphone app  
counts. Keeping such repo in sync on the device is insane.

Anyway, if I would be able to choose, I would choose central repo still for  
maemo 5. It scales nicely to low tens of thousands of apps.


> The resources needed to maintain a centralized repository can be quite
> easily parallized to make the repository itself scale: there can be many
> buildbots, many mirrors, a CDN, many testers, and many maintainers.

Sure, but still, users only need some dozens of apps from the repo, while  
devices are synching repo db totally unnecessary for the other hundreds (or  
thousands ... or more) packages. This doesn't scale. Think of all the trees  
we are burning to provide energy to the communication infra!

:)
> Now, the repository infrastructure and the processes around it can suck,
> of course, and putting another better designed and maintained repository
> into place might be needed. But that's only better because this other
> repository is better by itself; it's not better because we now have two
> of them. Shutting down the original sucky repository would be better
> still (but might not be easy, of course).

Yeah, we should at least add the repo delta support to the repos like...  
immediately.

> Distributing the packages over many repositories mostly increases
> coordination overhead. It doesn't help to scale. HAM still has to deal
> with the same number of packages, for example. (And yes, HAM sucks and
> badly needs some love, no argument from me against that.)

> For HAM performance, the important variable is the number of available
> versions, not the number of repositories. You can regain performance by
> reducing the number of available packages and versions. Forgetting
> about repositories altogether and installing everything from standalone
> .debs is one way to do that, but it's not a good way.

Sure, but many repositories allows the device to not to have to even ever  
care about those thousands of apps you couldn't care less about.

Still, central repo is mandatory for shared libraries, but not for the apps  
built on top of them. Damn. I'm starting to repeat myself too much.

> I believe there is a better way to make package management delightful:
> We only let HAM see a (consistent) subset of all available packages, and
> we make it possible to change that subset very efficiently at run-time
> (at "UI speed" without the need for any "Updating please wait" progress
> bars).

Ah, sounds like a splendid idea to me, and definitely doable with little  
problems. HAM performance is about 100x slower than it need be atm.


> That subset would include only the installed packages plus the ones that
> the user is currently interested in. Discovering packages that the user
> might be interested in is done without help from apt, via other means.

maemo.org/downloads would be my first pic as this discovery method.


> We are currently writing code for this. We don't have all the pieces in
> place yet, but we have some:

> - The "x-apt" package in Fremantle extras-devel. This is Harmattans
> version of apt, repackaged to be installable in parallel to
> Fremantle's version of apt.
<snip>

If these are really interesting, let's merge to fremantle.

<snip>

> - A maemo.org specific 'discovery client'. It interfaces with
> downloads.maemo.org over a custom protocol for browsing available
> applications. Right now it passes .install files to HAM for the
> actual installation, and my plan is to hook it up with mini-pacman
> instead and then optimize the hell out of the whole stack.

> (Haven't seen the code yet myself, Daniel Wilms and Niels Breet
> should know more.)

I didn't even know of this. Is it community or nokia effort? Can we get url  
to the code repo?


> > [...] I really do thing we have started to make our home our prison



> Then we should remove the bars and locks. Tearing down the whole house
> and going back up the trees would be overreacting quite a bit, no?

Well, this is a wakeup call. Either we react promptly (as in now, not  
in ... whenever harmattan comes out), or we will tear the house down to one  
degree or another.


Urho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.maemo.org/pipermail/maemo-developers/attachments/20100326/9c3bab3d/attachment.htm>