[pylucene-dev] status of pylucene & mod_python

[David Moore]

Ofer Nave wrote:
>> -----Original Message-----
>> From: pylucene-dev-bounces at osafoundation.org 
>> [mailto:pylucene-dev-bounces at osafoundation.org] On Behalf Of 
>> David Moore
>> Sent: Monday, March 26, 2007 4:34 AM
>>
>> Ofer Nave wrote:
>>     
>>>> -----Original Message-----
>>>> From: pylucene-dev-bounces at osafoundation.org
>>>> [mailto:pylucene-dev-bounces at osafoundation.org] On Behalf Of Andi 
>>>> Vajda
>>>> Sent: Sunday, March 25, 2007 5:38 PM
>>>>
>>>> On Sun, 25 Mar 2007, Ofer Nave wrote:
>>>>     
>>>>         
>>>>> I'm guessing the PyLucene code starts up and tries to
>>>>>       
>>>>>           
>>>> create threads,
>>>>
>>>> Nowhere in the Lucene core code is there a "new Thread(" statement.
>>>> PyLucene doesn't start any threads either.
>>>>
>>>> All threads are started by your code or the environment 
>>>>         
>> your run it 
>>     
>>>> under. A web framework is likely to start a pool of threads before 
>>>> PyLucene is even imported. You might need to customize that thread 
>>>> pooling code to ensure the proper threading class, 
>>>> PyLucene.PythonThread is used.
>>>>         
>>> However, in this case, I am not starting any threads, nor is my web 
>>> framework starting any threads.  I'm running under apache with the 
>>> prefork module (no threads), under mod_python (may or may not be 
>>> threaded - don't know enough about mod_python),
>>>       
>> There's your problem.  mod_python takes _all_ the threads it 
>> runs with from Apache, which creates a pool of 
>> threads/processes for itself to use.  So, you're running 
>> under a non-PyLucene thread as soon as you're running under 
>> mod_python.
>>     
>
> Hm.  Would you happen to know if it's possible to disable this behaviour?
>
> -ofer
>
>   
I know it's not possible to - this is how things are done under Apache. 
This is why, until gcj can late-initialize a thread into its garbage
collection, PyLucene won't work under mod_python.  All solutions I've
seen are something like solr (http://lucene.apache.org/solr/) - i.e. run
Lucene/PyLucene in a different process and have some sort of
inter-process communication to run the index.

HTH,
David Moore