[pyicu-dev] Memory Leak?

[David Bolen]

I've just begun experimenting with PyICU and I'm seeing what appear to
be some pretty significant memory leaks, so much so that I'm guessing
I may just be doing something very wrong.  I was wondering if anyone
might have some suggestions as to what it is.

At the moment all of my tests have been under Windows (when I get a
chance I plan on trying under Unix).

For example, a simple test using Locale() objects shows the following
(using task manager to check memory usage):

          - - - - - - - - - - - - - - - - - - - - - - - - -
Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
(Memory at 2740KB)

>>> from PyICU import Locale
>>> 
(Memory at 4436KB)

>>> for i in range(10000):
...   l = Locale('en_US')
...
>>> 
(Memory at 6272KB)

>>> for i in range(10000):
...   l = Locale('en_US')
...
>>> 
(Memory at 7920KB)
          - - - - - - - - - - - - - - - - - - - - - - - - -

The growth seems to continue indefinitely (e.g., it doesn't seem to be
a fragmented working set that eventually stabilizes).  I realize the
2.4 release isn't fully up to date, but since I also see the behavior
with 2.3, I don't think it's an internal Python issue.

I've chosen Locale above, but it seems to happen with just about
everything (and for example, 10000 NumberFormat classes created with
createInstance will cause growth of about 13MB).

The above is from the pre-built PyICU-0.3 binaries along with a
pre-built ICU-3.2 library.  But I've verified similar behavior in the
following combinations when building PyICU from source (the current
SVN head):

* PyICU-svn built against ICU-3.4, both compiled by VC7 (VC.NET 2003)
  and linked with Python 2.4.
* PyICU-svn built against ICU-3.4, using a pre-compiled ICU library for VC7.
* PyICU-svn built against ICU-3.4, both compiled by VC6, and linked against
  Python 2.3.

Note that in the last case, I needed to temporarily disable the use of
the internal C API to create datetime types in common.cpp, as well as
replace the use of the Py_RETURN_NONE macro in bases.i and locales.i
since that to account for differences from Python 2.3 to 2.4.

Also, when building PyICU from source in these cases I have tried both
SWIG 1.3.24 and 1.3.28.

Interestingly, swig 1.3.28 generates warnings during compilation
(about the use of the -nodefault option) and runtime such as:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyICU import Locale
>>> l = Locale('en_US')
>>> l = Locale('en_US')
swig/python detected a memory leak of type 'icu::Locale *', no destructor found.

This continues to happen even if I replace the "-nodefault" option
with "-nodefaultctor" and "-nodefaultdtor" as suggested by the swig
warning during compilation.  I haven't really tried to evaluate why
PyICU is choosing to disable constructor creation.

Given the scale of this apparent leak, I have to imagine it would have
been seen in other cases where it might be getting used, so I'm hoping
I'm just doing something wrong, and would appreciate any thoughts.

Thanks.

-- David Bolen