[maemo-developers] Mirrors

[From: Jeremiah Foster]

On Jan 3, 2010, at 17:37, Jeff Moe wrote:

> On Sunday 03 January 2010 11:45:17 Jeremiah Foster wrote:
>> On Jan 3, 2010, at 12:22, David Greaves wrote:
>>> Jeff Moe wrote:
>>>> As far as I can tell, there are no mirrors of the repositories.
>>> 
>>> Pretty sure Nokia/maemo.org went with a CDN which satisfies all the
>>> points you raised.
>>> 
>>> The maemo.org infrastructure problems are more to do with dynamic content
>>> and build services and are being addressed AFAIUI.
>> 
>> David is right on both counts. Most free software projects do have mirrors
>> but we are lucky in having Nokia as a partner in Maemo because they have
>> taken on the significant cost of pushing maemo.org content to mirrors. In
>> this case the mirrors are a top notch Content Delivery Network.
> 
> * That doesn't make us "lucky".

Pretty ungenerous of you. I think that maemo is extremely luck to have a corporate sponsor who puts up with the criticism, justified and unjustified, that we all throw at it. Of course there is something in it for them, but the infrastructure isn't free. Trust me, Red Hat does help Fedora out of the goodness of their heart either. And HP has been a very good friend to debian. 

> The repositories are frequently down. If there 
> were mirrors we could just point at the mirrors. They were down yesterday, for 
> instance.
> 
> * Why not complement this "top notch" service with mirrors and get the best of 
> both?

Maybe we ought to have an official mirror service, not necessarily a bad idea.

> 
> * I'm not talking dynamic content here. I'm talking a simple http repo. I 
> understand that mirroring dynamic sites like talk, the wiki, bugzilla, etc. 
> are harder to mirror.

Um, several gigs of debs and gigs of bandwidth is non-trivial. This will cost. And that is just repos.

> 
> * Every other single distro of note, whether community, corporate, or a mix 
> has a mirror. Can you point me to one that doesn't? What makes Nokia's way so 
> much better? Again, it would be much more convincing if it actually *worked* 
> reliably.

Bugs URLs please. Hard facts please, stuff we can present to Nokia and say: "Akamai is not performing."
> 
> What's the downside to having mirrored content?

Keeping it in sync. Another resource to monitor, yak shaving, etc.
> 
>> Garage is being replaced, you can see lots of information about that
>> progress on Maemork: http://www.qaiku.com/channels/show/maemork/
> 
> "Lots of infomation?" Like an occasional tweet? I have been following that.

You undermine your argument when you become pejorative and dismiss out of hand the significant work being done. 
http://wiki.maemo.org/Maemo.org_Sprints/December_09#Tasks 

Plus there is lots of stuff happening behind the scenes. I agree that we should be more communicative about this, but it is happening.
> 
>> Garage has been a victim of its success and of the significant growth that
>> maemo has seen. Many people are working hard to make sure that garage will
>> continue to work smoothly for the whole community. I personally am working
>> towards that goal and I know that teams inside and outside Nokia are
>> working towards that too.
> 
> Yes, garage is slow and that is a bit understandable. Even that could be 
> mirrored I'm guessing since sourceforge has lots of mirrors when you go to 
> download and garage is running sf-esque code.  But I wasn't talking about 
> garage in particular, I was talking about the repository. Surely the 
> repository functioning isn't dependent on garage code. If so, garage should 
> sync that over to another box which can be sync'd publicly.

Garage runs it all. The repos, the web site, gforge, etc. There are some other resources but nothing significant. This is all getting built out by an order of magnitude. But it takes time because obviously we want as little downtime as possible. 
> 
> Again, what's the downside to following the tried and true practices of every 
> gnulinux distro of the last decade+?

The repositories are a public resource. Nothing but time, cost and energy stop you from mirroring them yourself. Having an infrastructure to mirror different from Akamai might be good either as a redundancy measure or as an emergency resource. I think you'll find it non-trivial and your time will be better spent helping the new infrastructure and pointing out where Akamai fails to deliver so we can fix it.

> -Jeff
> 
> P.S. Tangentally: change your /etc/resolv.conf nameservers and you get a 
> different download server.

Sounds like intentional Akamai magic.

Jeremiah