[pylucene-dev] momery leak with PythonThread

[Andi Vajda]

On Thu, 27 Apr 2006, Yura Smolsky wrote:

> I have following program. It should starts threads and then frees
> them. When I use for this task threading.Thread then there are not
> leaks here. When I use PyLucene.PythonThread then I got very noticeable
> memory leak.
> PyLucene is last from trunk. Python is 2.4.2. Tested on winXP and debian systems.
>
> #!/usr/bin/python2.4
>
> import PyLucene
> import time
>
> def funny(num):
>    l = []
>    l.append(num)
>    #time.sleep(0.001)
>
> ths = []
> for i in range(1000000):
>    th = PyLucene.PythonThread(target=funny, args=[i])
>    th.start()
>

  Hello Yura,

Indeed there is quite a leak. I was able to reproduce the bug you found. With 
the attached patch, a potential fix, after about half a minute the process 
size stabilizes at around 38Mb on my Mac.
Could you please verify that it solves the issue for you too ?

Thanks !

Andi..
-------------- next part --------------
Index: PyLucene.py
===================================================================
--- PyLucene.py	(revision 262)
+++ PyLucene.py	(working copy)
@@ -45,14 +45,6 @@
     happy, any python thread using libgcj must be of this class.
     """
 
-    class runnable(object):
-
-        def __init__(self, callable):
-            self.callable = callable
-
-        def run(self):
-            self.callable()
-
     def __init__(self, *args, **kwds):
 
         super(PythonThread, self).__init__(*args, **kwds)
@@ -63,12 +55,22 @@
         current = threading.currentThread()
         assert (current.getName() == 'MainThread' or isinstance(current, PythonThread)), "PythonThread can only be started from main thread of from another PythonThread"
         
+        class runnable(object):
+            def __init__(_self, callable):
+                _self.callable = callable
+            def run(_self):
+                try:
+                    _self.callable()
+                finally:
+                    del _self.callable
+                    self.javaThread = None
+
         threading._active_limbo_lock.acquire()
         threading._limbo[self] = self
         threading._active_limbo_lock.release()
         
-        runnable = PythonThread.runnable(self._Thread__bootstrap)
-        thread = self.javaThread = Thread(runnable, self.getName())
+        thread = self.javaThread = Thread(runnable(self._Thread__bootstrap),
+                                          self.getName())
         thread.start()
 
         self._Thread__started = True