[pylucene-dev] IndexWriter.addIndexes() holding onto the GIL?

[Andi Vajda]

> The problem I'm having is that IndexWriter.addIndexes() blocks the whole
> program, not just the thread that's running it.  Below is a small script

Yes, that is true, the call to java does not release the GIL. I already added 
an overload to IndexWriter.optimize(yield) that releases the GIL, I can do the 
same for IndexWriter.addIndexes(). At some point these tricks are going to 
bite us though as some java calls call back into python via extensions and I 
did not wrap these calls with GIL code yet.

I think IndexWriter.addIndexes() is safe though unless you use a python 
extension of Lucene Directory which I doubt since I very recently added 
support for that.

So, I just checked in code that overloads the two IndexWriter.addIndexes()
methods by allowing an extra boolean argument, yield, to relinquish the GIL if 
True.

For example:

   directory = FSDirectory.getDirectory('index', False)
   writer = IndexWriter(RAMDirectory(), StandardAnalyzer(), True)
   writer.addIndexes([directory], True)

Andi..