[pylucene-dev] gcj garbage collector initialisation and threads

[Andi Vajda]

On Tue, 18 Jul 2006, David Fraser wrote:

> Standard gcc 4.1.1 runs some of the tests fine but:
> - segfaults / locks up on test_PyLuceneThread
> - segfaults test_RangeFilter
> - sometimes segfaults test_RangeQuery
> - segfaults test_Sort
> - segfaults test_PythonDirectory (at variable points within the test)
> - says Aborted on test_Highlighter
> - FAILS all 18 tests in test_PyLucene with "Lock obtain timed out"

Feels familiar.

> These all seeming like threading problems...
>
> The patch you were given on that list seems like an earlier version of
> the one attached to the bug (also from Bryce McKinlay) so I used the
> version from the bug.
>
> gcc 4.1.1 with the patch from bug 13212 runs most of the tests fine but:
> - FAILS all 18 tests in test_PyLucene with "Lock obtain timed out"

This is because of the earlier crashes. You probably need to remove the lock 
files in /tmp manually. Also, remove the left over index directory that is 
created by test_PyLucene.py

> I presume therefore that the Lock obtain timed out is unrelated to the
> threading tests (any ideas Andi), and that this patch solves the
> threading problems on gcc 4.1.1. Yay!

Yeah, this is what I saw on Gentoo with 4.1.1 + the earlier patch by Bryce
McKinlay.

> The next steps are:
> 1) Run tests for using standard Python threads rather than creating them
> with PyLucene.PythonThread, and see what needs to be added to support
> them (presuming calls to AttachThread methods...)

Yes, this would be the major achievement. If you can get rid of 
PyLucene.PythonThread, this would be a big day in PyLucene land !

When creating a thread that is not a PyLucene.PythonThread, before calling any 
function in libgcj that involves allocating memory, you need to first call 
libgcj's JvAttachCurrentThread() API, the API that was allegedly made to work 
correctly with that patch to gcj 4.1.1. There currently is no such API 
accessible from Python (I did not wrap it in java.cpp) but it should be pretty 
simple to add it. Let me know if you need help with that.

Thanks for pursuing this !

Andi..