[pylucene-dev] fatal error in GC : too many heap sections.

[Pete]

On Tuesday November 20 2007 10:39:00 am Bill Peverill wrote:
> Java Lucene route. The GCJ version has served us well in a single thread
> app, and we believe a JCC version will work great, but sorting out the
> JCC/windows build process may be more risky for us than simply implementing
> Java Lucene, with all its horrors. [Simply? Did I just say that?]

Look, building JCC from source is just not that hard.  Get the build 
depdencies (binaries s.b. available for your platform), do a svn checkout,  
`python jcc/setup.py`, edit the Makefile (basically just set paths, there are 
examples), `make`, `make test` `make install`. Take a day or two, if it 
works, it'll be worth your time. 

As I already mentioned and should have been indicated from the ML/google, the 
solution to the "too many heap sections" is using LARGE_CONFIG, which is a 
build-time option to GCJ.  If you're not comforable building PyLucene-JCC, 
don't even bother going this route, it's way harder.

If you want to go the Java route, try Solr.

> But First we will build a periodic restart mechanism for our GCJ
> implementation. Except for the leak, this system is VERY fast and

This is not as simple as it sounds, especially if you don't want to lose 
in-process requests or requests coming over the wire.

> functionally complete, so we want to see it through [as do our paid
> subscribers!]

For that, I'll just read "as does your boss". ;-)

>  The secret will be to find a metric which can reliably
> predict an impending failure.. -time- is the most basic, but we would

We never found anything that was a particularly good indicator and settled for 
timed rotation.

> Pete, it looks like you used this solution for a while as well.. did you
> just restart on a schedule, or were you able to monitor for a predictive

We restart on a schedule, but that's driven by our update mechanism, which 
mirrors the Solr index distribution approach.  As I said, we were pretty much 
going to do restarts anyway.  The difference from Solr is that while they 
swap indexes in-process, we swap processes.  We recently threw things behind 
a load-balancer, which simplifies things somewhat, but it's still ugly.

> event? Also, you restarted the process/service, but did you have to restart
> the machine periodically as well, or was your solution stable without
> machine restarts? [if restarting a service every hour can be called
> stability.]

If you need to restart your box for a problem like this, you need a new OS.

I think this is starting to get a bit OT for this list.  Best of luck.

-- 
Peter Fein   ||   773-575-0694   ||   pfein at pobox.com
http://www.pobox.com/~pfein/   ||   PGP: 0xCCF6AE6B
irc: pfein at freenode.net   ||   jabber: peter.fein at gmail.com