[pylucene-dev] PyLucene CherryPy threads conflict

[Pravin Shinde]

On 7/25/06, Andi Vajda <vajda at osafoundation.org> wrote:
>
>
> It's a perfect question for a FAQ. If you submit something about this for
> a
> FAQ, I'll start one and add your contribution to it.


 Hi,
Here I tried to put my experiance in integreting cherrypy with PyLucene as
Howto or Tutorial.
If anyone can suggest me corrections or modifications
or if they can add some more meaningful contents, feel free to do so.

-----------------------------------------------------------------------------------
= CherrPy PyLucene integration =

== Problem ==
There are some problems in calling some PyLucene API from CherryPy code.
I got following Java exception when I tried calling ''PyLucene.IndexWriter''
from CherrPy.

{{{
writer = PyLucene.IndexWriter( "c:/index", PyLucene.StandardAnalyzer(),
True)
JavaError: java.lang.NullPointerException
}}}

The same API works fine if called from python console.

{{{
>>> PyLucene.IndexWriter( "c:/index", PyLucene.StandardAnalyzer(), True)
<IndexWriter: org.apache.lucene.index.IndexWriter at 17f3fa0>
>>>
}}}

The reason for conflict lies in the threading mechanism used by CherryPy.

CherryPy page handlers never run in the main thread, a new thread is created
for handling  new request. These new threads are local threads created from
''threading.Thread''

Basically, for a python only main thread can call into PyLucene without any
problems.
For other threads to work with PyLucene, the boehm-gc component of the java
runtime ( i.e. the garbage collector) should be informed about their
creation. If these threads are created as an instance of ''
PyLucene.PythonThread'', it will make sure that the thread is created via
libgcj and libgcj garbage collector is aware of it.


== Solution ==
The idea is to use ''PyLucene.PythonThread'' instead of normal ''
threading.Thread'' in CherryPy source code.

I did following modifications in CherryPy source code with CherryPy
2.2.1and PyLucene
2.0.0 on Windows 2000 to work together.

The cherrypy code should be available in following directory inside python
installation directory.
''python_installation_dir\Lib\site-packages\cherrypy''

Files which have ''threading.Thread'' are
''_cpserver.py''
''_cpwsgiserver.py''

You need to add statement ''import PyLucene''
and replace ''threading.Thread'' with ''PyLucene.PythonThread'' in above two
files.

Also disable CherryPy's autoreload (by adding autoreload.on=False to
configuration file) because autoreload uses low level threads.

These changes worked fine in my case.
----

For more details link
http://lists.osafoundation.org/pipermail/pylucene-dev/2006-June/001107.html
-----------------------------------------------------------------------------------

-- 
Regards,
Pravin Shinde
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/pylucene-dev/attachments/20060727/1000759a/attachment.html