[pylucene-dev] Pylucene and threading

mark williamson mark at junklight.com
Fri Nov 19 14:11:53 PST 2004


Hi,

Yeah - the problem is that it stops. sometimes.

What you say makes sense I think

The temptation to put pylucene into its own single
threaded service separate from everything else is great...

cheers

mark



On 19 Nov 2004, at 21:25, aurora wrote:

> You didn't say what problem you've actually encountered ;)
>
> Let me guess. I'm haven't used twisted. But I checked from their
> documentation a little:
>
>    
> http://twisted.sourceforge.net/TwistedDocs-1.2.0/howto/ 
> threading.html#auto2
>
> Accord to it, reactor.callInThread( ) queue methods to be run by a  
> thread pool.
>
> attachCurrentThread() need to be wrapped for the life time of the
> thread. In twisted, the thread pool manage the life cycle of the
> thread so your wrapping wouldn't be effective.
>
> One alternative is to create a thread yourself just for running
> PyLucene. Use some tool such as QueuedExecutor
>
>    
> http://g.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/ 
> QueuedExecutor.html
>
> I almost ported one to Python. But the Windows problem is solved so
> I'm putting this on hold.
>
>
> On Fri, 19 Nov 2004 08:37:35 +0000, mark williamson  
> <mark at junklight.com> wrote:
>> Hi,
>>
>> I'm having an odd problem. My app which runs in twisted and uses
>> PyLucene
>> works on my Mac (OSX 10.3.6) but not on my linux box (Gentoo,2.4
>> kernel,dual
>> processor).
>>
>> I've used the following mechanism to handle the thread (pinched from
>> this list I think):
>>
>> class startFn(object):
>>        def __init__(self, fn):
>>           self.run = fn
>>
>>         ....
>>       pyluceneFn = lambda:
>> PyLucene.attachCurrentThread(startFn(self.mergeIntoArchive))
>>       reactor.callInThread( pyluceneFn  )
>>         .....
>>
>> On Gentoo mergeIntoArchive never gets called. It runs fine on the mac.
>>
>> I installed from the binaries - on gentoo I tried the binaries that
>> come in the download AND
>> I've since deleted those and emerged gcj and tried with those  
>> binaries.
>>
>> Has anyone got any ideas?
>>
>> cheers
>>
>> mark williamson
>>
>> _______________________________________________
>> pylucene-dev mailing list
>> pylucene-dev at osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>>
> _______________________________________________
> pylucene-dev mailing list
> pylucene-dev at osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>



More information about the pylucene-dev mailing list