[pylucene-dev] Static builds and OS X Universal support

[Kevin Ollivier]

Hi Andi,

On Jun 19, 2006, at 11:30 AM, Andi Vajda wrote:

>
> On Mon, 19 Jun 2006, Kevin Ollivier wrote:
>
>> Getting back to some PyLucene stuff, and I see on Mac 2.0 is still  
>> using shared library builds. Were there problems with the static  
>> builds, or was there another reason we still can't use them?
>
> The tricks we used to get a static build of _PyLucene.so with  
> libgcj.a to work
> work only with gcj 3.4.x.
>
> I tried to make it work with gcj 4.x but it seems to involve more  
> than adding a few more of the missing symbols that were needed to  
> get it to work under 3.4.x. I've had no time to seriously look into  
> it.

Ah, I see. Thanks for pointing this out. I'll try to take a look at  
it at some point, but since it's hard to estimate how little or how  
much time a fix would take, I'll probably at least get some builds of  
my app out using the current system first.

>> The shared bits that come with PyLucene take up quite a bit of  
>> space, and significantly complicate the issue of making a  
>> Universal binary. If everything were statically linked, we could  
>> pretty easily create a Universal binary by running the following  
>> command after the intel and ppc builds complete:
>>
>> lipo -create /path/to/intel/_PyLucene.so /path/to/ppc/_PyLucene.so  
>> -output /path/to/universal/_PyLucene.so
>
> Creating a Universal Binary would require gcj 4.x since there is no  
> gcj 3.4.x on Intel Mac.

If you create both PPC and Intel builds on one machine, yes. But when  
you use lipo to create a Universal binary, really, you don't need to  
worry about using the same build process (or tools) on both  
platforms. They are pretty literally two separate binaries thrown  
together. So even if your Universal binary has pieces of gcj 3.4.x in  
it, they will be completely ignored by Intel builds. On Intel, it  
would use the gcj 4.0.x bits, or whatever you linked in for the Intel  
build.

> I should add that gcj for Intel Mac OS X is itself sort of a  
> miracle at the moment thanks to Sandro Tolaini's patches to gcj  
> 4.0.2 available at http://gcc.gnu.org/ml/java/2006-05/msg00151.html

One I'm very happy to see, as I'm using an Intel Mac and everything  
else in my app toolchain is Intel, so this was the last 'stumbling  
block' to developing on my Intel box. Thanks to both of you for  
that! :-)

>> This should be all that's needed for a Universal binary. It's not  
>> so easy, however, with the shared libraries to worry about. It's  
>> possible we could lipo the shared libraries together, but that  
>> will probably bloat the space requirement for the PyLucene  
>> extension alone to around 50MB
>
> While a universal binary is elegant it becomes a lot less so when  
> you consider the doubling of the size requirements.

IMHO, it's mostly a convenience to users. Unless the size difference  
is huge, it's nicer not to require users to know which type of Mac  
they have.

>> Granted, for me it will only be useful for Tiger builds right now  
>> (due to wxPython Universal issues), but it would still be very  
>> useful, and IMHO the transition will need to happen eventually.  
>> I'd like to be able to think of one day being able to only  
>> maintain one Mac binary for my software. ;-)
>
> And I would *love* to be able to use the same version of gcj  
> everywhere.
> But that's proven a very elusive goal...
>
> On Windows, I've been stuck with gcj 3.4.2 until I figure out how  
> to build a better one (I'm working on build a 4.0.2 gcj for  
> windows). gcj on windows needs a *lot* of work and is getting very  
> little attention. Just building the damn thing requires a PhD in  
> gcc configure/makefile hackery.
>
> On Ubuntu, gcj 4.1.1 seems to not work but gcj 4.1.0 works great on  
> Mac OS X (but I can only build it with darwinports). On Ubuntu, gcj  
> 3.4.6 works great.
>
> On ppc Mac, I've been using gcj 3.4.3 or gcj 3.4.5 for a long time  
> and it's worked well.
>
> On intel Mac, thanks to Sandro Tolaini's patches, I'm now able to  
> use gcj 4.0.2. Without his patches, where would be *no* gcj on  
> intel os x for the foreseeable future.
>
>> I'd go ahead and attempt this myself, but it sounds like the  
>> process to get gcj working on Intel is pretty involved, and I was  
>> hoping it would be pretty easy to do a static (and Universal)  
>> build on Mac if you've got everything already setup on your end.  
>> Otherwise, it would be very helpful if you could provide the  
>> complete GCJ build you used and spare me some pain. ;-)
>
> The intel gcj binaries I produced are available as a download from
> http://downloads.osafoundation.org/compilers/osx/i386 and compiling  
> it yourself is pretty easy with the patches I mentioned earlier  
> applied to a stock 4.0.2 gcc source tree.
>
> But first, the static link problem needs to be solved for 4.x again.
> The trick we used for 3.4.x is not good enough.

I see; sorry for oversimplifying the issue. I hadn't considered that  
the solution would only work for gcj 3.4.x, but I should've known  
better. ;-/ At least with the Intel build I can remove the speed hit  
on Intel Macs, which is a big factor if people begin to use my apps  
heavily.

It's a shame gcj doesn't get more attention. Although PyLucene is a  
lot of work to maintain, it is such a great tool; there is really  
nothing else out there like it, and making it into a CGI gets rid of  
the (hefty) Java app server requirements. Anyways, thanks for all  
your efforts!

Regards,

Kevin

> Andi..
>
> _______________________________________________
> pylucene-dev mailing list
> pylucene-dev at osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>