[Chandler-dev] Speeding up full builds
heikki at osafoundation.org
Mon Oct 30 22:37:23 PST 2006
I realized that I have gathered quite a few ideas on how to speed up the
full tinderboxes (or full builds actually), many of them when discussing
the problems with various people.
One reason of slow builds are simply bugs where we do the same thing
over and over when one time would do. For example, downloading some of
the sources when we already have them is taking unnecessary time.
We also do some operations in not so smart ways. For example, doing two
passes over some source directory to find and *.pyc and *.pyo and then
pipe the lists to rm - we could simply use a single find command.
Especially on Linux, but perhaps to some extent on Mac, we could utilize
system libraries instead of building them ourselves. For example, I've
already tested that I can build and run Chandler fine on Ubuntu using
the system ICU and OpenSSL libraries. (Chandler seemed to build and run
on the Mac with system OpenSSL but I got a scary-looking warning.)
make has the often overlooked -j switch to split the work into multiple
jobs. Especially on a multi-cpu system this can be a big time saver. The
problem is that it can be tricky getting the build actually work. I
already know that full build won't work, but clean, sources and expand
targets seem to work, and possibly some of the external libraries we use
could be built with -j even if we can't use it for all of them. The
problem with -j is also that if there is a build failure, trying to
decipher the build log is a nightmare (I found that using -j 2 gave me
almost all of the -j benefits on my dual core system while still leaving
the log reasonably readable).
I also know of distributed compiling, but I don't have any personal
experience with it.
Anyone know of other tips and tricks we could use, or has experience
with -j or distributed compiling?
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 254 bytes
Desc: OpenPGP digital signature
Url : http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20061030/7796dfe1/signature.pgp
More information about the chandler-dev