[maemo-developers] Autobuilder: building svn tags from garage

[From: Ed Bartosh]

2009/8/27 Henrik Hedberg <henrik.hedberg at innologies.fi>:
> Graham Cobb wrote:
>  >> Personally, I would much rather the autobuilder dependency problem
> was fixed
>  >> (with some method for submiting multiple packages with build
> dependencies and
>  >> having them build in the right order, with the dependencies being
> satisfied)
>  >> instead of this particular feature.
>
>    This should definitely be fixed first and soon. Thanks! :)
If everybody thinks that support of multiple package builds is more
important I'll do that.

>
> Ed Bartosh wrote:
>  > Submitting is the main problem. As far as I know dput can't upload
>  > several packages at the same time. Any ideas how to do this?
>
>    I find that you all are thinking this too complicated. Depency
> graphs, web based interfaces, discussion about extending the dput and
> such...
>
>    The current situation is that the build is triggered when a .dsc
> file has been uploaded. Fine.
>
I'm going to change the code as Marius suggested. .changes will
trigger the build.

>    Let's require that a developer must upload .dsc files (or packages,
> if you like) in the order he or she wants the packages to be build.
> Fairly reasonable, I think. There are timestamps, and scp modifies those
> when it uploads a file into the server. Simple as that. Even make is
> using those magical timestamps.
>
For one package it's already implemented in dput. .changes file is
uploaded after all sources are uploaded.
For multiple packages this is the problem, which we're trying to solve here.

>    Now, the only real modification is that the autobuilder should use
> the latest version of a package that has been built - not some ancient
> package found from the repository.
>
>    So, if a package A depends on a package B, I should upload the .dsc
> file of the package B first. I can upload the .dsc file of the package A
> immediately after that. The autobuilder builds the package B first,
> because it came first (the timestamp is older), and then the package A.
> When building the package A, it is using the package B from _the latest
> build_, not from the repository.
>
As I already said building in a proper order is already solved task.
There is no need to upload packages in build order if it can be done
automatically. It will only create extra job for developers.

>    I think the main problem with the current implementation is that
> even if you have uploaded a package and if it has been successfully
> built, it is not used when building other packages immediately after
> that. You have to wait unspecified time until the package hits the
> repository. Without that delay, one could even write a sequential upload
> script based on email build notifications. (Yes, I know that I could
> repeatedly download the Packages file from the repository, search the
> version number of my library packages, and trigger the uploading of my
> application based on that knowledge. Too complicated!)
>
True. But if we manage to define a group of packages as one build I
can solve this problem by creating local repo, adding packages to it
as soon as they're built and using it for the build.
Of course better solutions would be to make adding to extras-devel
work faster, but i tend to think it's near to impossible :)

>    If you care the situation when building fails, you could write an
> additional rule that all other packages from the same uploader are
> removed from the build queue if building of a package fails. Thus, if a
> package A depends on a package B, but the building of the package B
> fails, the autobuilder is not building the package A if it had already
> been uploaded into the queue.
>
If group of the packages is in the separate directory this is even
better. The same uploader can upload several groups and failure of one
group wouldn't cause removing of others.
This is what I've almost achieved by using dput.cf *_upload_command keywords.

-- 
BR,
Ed