[Chandler-dev] Post tbox builds of existing Chandler codebase
grant at osafoundation.org
Wed Aug 13 09:06:26 PDT 2008
One more random comment on this thread:
Yesterday on IRC, it was pointed out by a user whose handle I don't
exactly recall that we could consider using a build service for some
Linux platforms. For example, OpenSuse has one (that can apparently
produce builds for multiple platforms):
Also, Ubuntu has something similar, at
Heikki pointed out that our builds are somewhat quirky, and it might
take a fair amount of work to mash the various subprojects Chandler
builds into a form where they could be built via a service. But still,
probably worth investigating.
On 4 Aug, 2008, at 08:42, Grant Baillie wrote:
> Hi, Jared
> I'm basically fine with the assumptions and idea, and would be up
> for hosting a build machine ... I have a few random comments to
> throw in:
> - Another way of thinking about the current setup is that it
> produces 3 different types of "output":
> 1. Test/build results (as viewable on the tinderbox status page)
> 2. Distribution packages (.deb, .dmg, .tar.gz, windows
> installer .exe): things that end up on the downloads pages when we
> do single builds.
> 3. Tarballs of builds of the projects we rely on outside of
> Chandler: These are created during "full builds", and developers
> typically copy them onto the build server via the "Copy tarballs"
> page (linked from the tinderbox status page). There, they are
> available via http when you do a "make install" inside the chandler
> project itself. (This happens at some point during the scripts we
> run for #2).
> Your proposal mostly deals with #1 (the getting rid thereof :D) and
> #2. So far as I know, there currently isn't a one-stop "I'd like to
> build a .deb on the current platform now" script -- the current one
> relies on #3 having been done already. So, some investigation/build
> system work is needed to make this work.
> - In an ideal world (i.e. one in which setuptools was around when
> the build system was built ;) all the projects in #3 would be
> buildable as python eggs, and then we'd rely on pypi.org +
> easy_install/setuptools to create #3. This would relieve us of the
> practical burden of storing (and allowing upload of) binaries for
> many platforms on builds.osafoundation.org. I'm not sure if this is
> really a useful streamlining of the build process at this point,
> though. It certainly would be handy in cases where we're relying on
> unpatched external projects and don't need to be building the binary
> - If I remember right, the build server does cache the source of
> some external projects, rather than svn export/checkout those
> projects directly. I believe this was done so as not to have our
> tinderboxes depend too much on servers other than our own, but in
> the "slow build" approach this could probably be done away with.
> - As a side note, on the subject of external dependencies and
> unpatched external projects, a while back Heikki (and others) had a
> look at what we would need to do to get Chandler into the Hardy
> Heron Ubuntu repositories. Heikki's writeup can be found at:
> - As another side note, Mac OS X supports cross-compiling, i.e. on
> my current Leopard Intel machine I can build executables for Mac OS
> X 10.3, 10.4, 10.5 (as well as intel/ppc). However, tweaking our
> Makefiles (and the Makefiles of the projects we build in #3) is
> probably a lot of work. It might be worth spending a day or two
> looking into this, though.
> On 16 Jul, 2008, at 18:34, Jared Rhine wrote:
>> We know the time is soon approaching where we'll need to close the
>> room where all the OSAF Chandler Desktop tinderbox/build-boxes are
>> housed. Sniff.
>> This will have a huge impact on our ability to build Chandler
>> Desktop. We need a new scheme.
>> I'm hoping to give us enough time to get through 1.0 release and
>> maybe a bug-fix release beyond that before the tbox get shut off.
>> There's not currently a hard date by which we need to shut off the
>> tinderbox, but we should be kind to our hosts and work towards this
>> shutdown. Perhaps end of August is a realistic shutdown time.
>> How could we work towards a realistic post-tbox build scheme for
>> Chandler Desktop?
>> A brief IRC discussion today leads to this basic proposal:
>> * Build official Chandler Desktop releases manually on a Mac laptop
>> running Parallels and hosted at a developer's house
>> - Find/allocate a MacBook Pro laptop, with Parallels VM software
>> - Configure the laptop to be able to build Mac Intel releases
>> - Install two VMs; one for Windows and one for Ubuntu
>> - Have new releases (and milestones/RCs) kicked off manually by the
>> machine owner on all three platforms
>> This idea is based around these assumptions:
>> - It's ok to build releases much more slowly. Slow-and-steady
>> builds off a single machine is better than no builds at all.
>> - It's ok to kill a couple of the build types. Debug builds, PPC
>> builds in particular.
>> - Any volunteers who provide additional platforms, dedicated
>> machines, or labor would be happily incorporated somehow into a
>> Desktop builds process. We'd always support pointing to "contrib"
>> builds if any get created and contribed builds could easily be made
>> - Continuous integration testing goes away. No more official OSAF
>> tbox, unless someone volunteers on one or more platforms.
>> - We shouldn't invest a ton of labor/capital into preparing a tbox/
>> build cluster replacement for the old codebase. It may thrive, but
>> most developers are skeptical. It's most important to have *a* way
>> to create new releases of the old codebase as opposed to having a
>> *great* way to create new releases.
>> - We're ok with some problems if the volunteer's DSL gets knocked
>> offline, or their cat pees on the laptop under the desk, or
>> whatever. Heck, the box doesn't even need to be on except for when
>> it's building software.
>> - We're preferring options which reuse our existing hardware and
>> software assets.
>> - We agree there's no realistic way to move the current tbox
>> cluster elsewhere. The large masses of full-tower desktop
>> machines, iMacs, etc, are enough of a burden to volunteer hosts
>> we're not going to spend much time trying to preserve that cluster.
>> There are some additional refinements we can envision:
>> - Host the laptop in a DMZ at someone's house, for network security.
>> - Arrange for multiple people to have access to box and VMs remotely.
>> - Provide VNC-based login.
>> - Automate the build/upload process significantly. Have the VMs
>> poll for instructions on kicking off a new build. Either HTTP
>> polling or POP mailbox pulling (ala the way tbox communicates right
>> - Give the machine a static IP or arrange a dynamic DNS name via
>> dyndns.org or similar.
>> If we go with something like this approach, there's some basic work
>> items we could kick off:
>> - Identify/allocate a MacBook Pro laptop for the build machine
>> - Identify a willing volunteer to host the machine and preferably
>> be willing to kick off builds
>> - Configure Mac layer to host Mac Intel builds
>> - Create two VMs (Windows + Ubuntu) and get them ready to build
>> Chandler Desktop instances
>> - Start documenting how builds on this machine would be produced
>> across all platforms. (As well as tricky issues like uploading new
>> plugin versions, etc).
>> It'd be great to get some creative alternatives to the above, or to
>> hear multiple offers of hardware/hosting/etc. The above is
>> hopefully a realistic, small-scope approach that gives us at least
>> one post-tbox-cluster option to being able to produce new releases
>> on the old codebase. Note, we still have a "how to build Desktop"
>> problem with any new codebases too.
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> Open Source Applications Foundation "chandler-dev" mailing list
More information about the chandler-dev