[pylucene-dev] finalizing the deadly embrace

[Andi Vajda]

On Mon, 21 Jan 2008, Brian Merrell wrote:

> The latest changes seem to have solved my problem.  The refcounts are fairly
> stable (some fluctuation) and I have indexed 10 times as many documents as
> before without problem.  Thanks for resolving this so quickly!

Phew !

Andi..

>
> -brian
>
> On 1/19/08, Andi Vajda <vajda at osafoundation.org> wrote:
>>
>>
>> On Thu, 17 Jan 2008, Andi Vajda wrote:
>>
>>> Thinking about this some more, I believe that Anurag's finalizer proxy
>> idea
>>> is on the right track. It provides the "trick" needed to break the
>> deadly
>>> embrace when the ref count of the python object is down to 1, that is,
>> down
>>> to when the only reference is the one from the Java parent wrapper.
>>>
>>> When the finalizer proxy's refcount goes to zero, it is safe to assume
>> that
>>> only Java _may_ still be needing the object. This is enough then to
>> replace
>>> the strong global reference to the Java parent wrapper with a weak
>> global
>>> reference thereby breaking the deadly embrace and letting Java garbage
>>> collect it when its time has come. When that time comes, the finalize()
>>> method on it is normally called by the Java garbage collector and the
>> python
>>> ref count to the Python extension instance is brought to zero and the
>> object
>>> is finally freed.
>>>
>>> This assumes, of course, that when such an extension object is
>> instantiated,
>>> the finalizer proxy is actually returned.
>>>
>>> I should be able to implement this in C/C++ so that the performance hit
>> is
>>> minimal and in a way that is transparent to PyLucene users.
>>>
>>
>> I checked the implementation of this idea into svn trunk rev 381.
>> It is no longer necessary to call finalize() by hand :)
>>
>> I removed the finalize() calls from test_PythonDirectory.py and
>> test_Sort.py
>> can now be run for ever, without any leakage.
>>
>> It is necessary to rebuild both JCC and PyLucene to try this out.
>> I'd be curious to see if this solves your problem, Brian ?
>>
>> Andi..
>> _______________________________________________
>> pylucene-dev mailing list
>> pylucene-dev at osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
>>
>