[pylucene-dev] "thread_get_state failed" + abort in PyLucene

[David Pratt]

Hi Terry. You might want to google nxlucene for hints. I know they are 
using a twisted app successfully for lucene search with pyLucene. There 
repository is open to view the code. The search server is xmlrpc and 
returns rss - hth. I'd like to know how this all turns out. Can you post 
back to the list if you have success. Many thanks.

Regards,
David

Terry Jones wrote:
>>>>>> "David" == David Pratt <fairwinds at eastlink.ca> writes:
> David> Hi Terry. I am curious which reactor are you using for this. Any
> David> threads need to be java threads with pylucene. Not sure if this
> David> might work but perhaps creating a thread pool with the right kind of
> David> threads could help. Somewhere on this list a while back, someone put
> David> together some thread pool code that you might find helpful for
> David> this. Twisted also has its own thread pool so this would need some
> David> modification. Have you tried testing your index outside of twisted
> David> to see whether the issue persists.
> 
> Hi David
> 
> I'm using poll, though not for any particular reason. Do you recommend I
> try with something else? (I'll try epoll and select and see if they help).
> 
> I'm not using any threads in my code. Are you suggesting I create an
> appropriate thread pool and then get Twisted to use it?
> 
> Testing outside of Twisted seems fine. I can't make it fail, even when I
> call the same functions eventually called by the web2 interface, with the
> same XML payload it receives, etc. I should have mentioned that too.
> 
> One other thing to mention is that the crash doesn't seem to happen when I
> run twistd with -n (no daemon) option. JP Calderone on the Twisted list
> suggested that PyLucene might be somehow thrown off by the initial fork of
> twistd. That doesn't seem to be the case though, as I rewrote things to
> defer the PyLucene import (using __import__) until after twistd has
> started. So I'm not sure what to make of the crash only happening when
> twistd runs as a daemon.
> 
> Thanks for your help.
> 
> Terry
>