[pylucene-dev] memory leak status

Andi Vajda vajda at osafoundation.org
Fri Jan 11 10:28:23 PST 2008


On Fri, 11 Jan 2008, Pete wrote:

> Perhaps you're not understanding me...  I've got code that looks more or less
> like:
>
> class DoesThingsWithSearcher(object):
>    def __init__(self, factory):
>        self.searcher=factory() # who knows/cares what we get here,
>                                # as long as it has the right interface
>
>    def alldone(self):
>        self.searcher.close()
>        try:
>            self.searcher.finalize()
>        except AttributeError:
>            pass # must have been a Lucene object, not a python extension

I see. There is a misunderstanding here. That searcher object is not a 
Python extension unless you implemented a PythonSearcher Java subclass 
somewhere with native methods that then JCC generated wrappers for.

By Python extension I mean any instance of any class in the java directory 
tree of PyLucene, such as PythonAnalyzer which is used to implement an 
analyzer in Python. How to write such an extension is described further in 
the "Writing Java class extensions in Python" topic in [1].

Currently, unless you added it, there is no such extension for a Searcher.
Any searcher instance you create is a regular Java Lucene instance wrapped 
by a Python object, not a subclass implementing all or part of Searcher with
actual Python methods.

> Yes, in this case, a Python searcher should have called finalize() in it's
> close() (which should probably be considered 'best practice' anyway), but if
> the object doesn't *have* a close(), there's no good answer.  Just calling
> finalize() and catching AttributeError seems to be appropriate.  Not a big
> deal.

Again, there is no need to call finalize() on anything unless it's an 
instance of a custom Python extension. All of these custom Python extension 
instances are explicity created via their class name by you, if at all. 
PyLucene never creates any by itself.

When you ask PyLucene for a Searcher, a Term, a Query, or any other default 
Java Lucene class, these are not considered extensions and they are freed 
automatically when no code refers to them anymore.

It is only Python extensions, that is by default, subclasses of any class 
defined in the 'java' directory tree of PyLucene [2] that need this special 
finalize() treatment. If you are not using any of them (or not defining your 
own), then you can ignore this finalize() issue completely.

Andi..

[1] http://svn.osafoundation.org/pylucene/trunk/jcc/jcc/README

[2]
yuzu:vajda> find java -name '*.java'
java/org/osafoundation/lucene/analysis/PythonAnalyzer.java
java/org/osafoundation/lucene/analysis/PythonCharTokenizer.java
java/org/osafoundation/lucene/analysis/PythonTokenFilter.java
java/org/osafoundation/lucene/analysis/PythonTokenStream.java
java/org/osafoundation/lucene/index/PythonTermDocs.java
java/org/osafoundation/lucene/queryParser/PythonMultiFieldQueryParser.java
java/org/osafoundation/lucene/queryParser/PythonQueryParser.java
java/org/osafoundation/lucene/search/highlight/PythonFormatter.java
java/org/osafoundation/lucene/search/highlight/PythonFragmenter.java
java/org/osafoundation/lucene/search/PythonFilter.java
java/org/osafoundation/lucene/search/PythonHitCollector.java
java/org/osafoundation/lucene/search/PythonScoreDocComparator.java
java/org/osafoundation/lucene/search/PythonSimilarity.java
java/org/osafoundation/lucene/search/PythonSimilarityDelegator.java
java/org/osafoundation/lucene/search/PythonSortComparator.java
java/org/osafoundation/lucene/search/PythonSortComparatorSource.java
java/org/osafoundation/lucene/store/PythonDirectory.java
java/org/osafoundation/lucene/store/PythonIndexInput.java
java/org/osafoundation/lucene/store/PythonIndexOutput.java
java/org/osafoundation/lucene/store/PythonLock.java
java/org/osafoundation/util/PythonComparable.java
java/org/osafoundation/util/PythonIterator.java
java/org/osafoundation/util/PythonReader.java
java/org/osafoundation/util/PythonSet.java



More information about the pylucene-dev mailing list