[pylucene-dev] building pylucene with gcc uses 1gb RAM

[Andi Vajda]

On Jun 1, 2008, at 4:55 AM, "Peter MacRobert" <originalpete at gmail.com>  
wrote:

> Hi there
>
> I'm trying to build pylucene for a Debian platform. The server is a  
> VPS slice with only 500Mb RAM, and gcc is taking up 300Mb RAM and  
> over 600Mb of swap. The build grinds to a halt and won't finish,  
> even though it doesn't exit.
>
> This is as far as it gets, and then hangs forever:
>
> CFLAGS="" /usr/bin/python -m jcc --jar lucene-java-2.4.0-649619/ 
> build/lucene-core-2.4.0-649619.jar --jar lucene-java-2.4.0-649619/ 
> build/contrib/snowball/lucene-snowball-2.4.0-649619.jar --jar lucene- 
> java-2.4.0-649619/build/contrib/highlighter/lucene- 
> highlighter-2.4.0-649619.jar --jar lucene-java-2.4.0-649619/build/ 
> contrib/analyzers/lucene-analyzers-2.4.0-649619.jar --jar lucene- 
> java-2.4.0-649619/build/contrib/regex/lucene-regex-2.4.0-649619.jar  
> --jar lucene-java-2.4.0-649619/build/contrib/queries/lucene- 
> queries-2.4.0-649619.jar --jar build/jar/extensions.jar --package  
> java.lang java.lang.System java.lang.Runtime --package java.util  
> java.text.SimpleDateFormat --package java.io java.io.StringReader  
> java.io.InputStreamReader java.io.FileInputStream --exclude  
> org.apache.lucene.queryParser.Token --exclude  
> org.apache.lucene.queryParser.TokenMgrError --exclude  
> org.apache.lucene.queryParser.QueryParserTokenManager --exclude  
> org.apache.lucene.queryParser.ParseException --python lucene -- 
> mapping org.apache.lucene.document.Document 'get:(Ljava/lang/String;) 
> Ljava/lang/String;' --mapping java.util.Properties 'getProperty: 
> (Ljava/lang/String;)Ljava/lang/String;' --sequence  
> org.apache.lucene.search.Hits 'length:()I' 'doc:(I)Lorg/apache/ 
> lucene/document/Document;' --version 2.4.0-649619 --files 1 --build
> running build_ext
> building 'lucene._lucene' extension
> gcc -pthread -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -fPIC - 
> DPYTHON=1 -I/usr/lib/jvm/java-1.5.0-sun/include -I/usr/lib/jvm/ 
> java-1.5.0-sun/include/linux -Ibuild/_lucene -I/usr/lib/python2.5/ 
> site-packages/JCC-1.9-py2.5-linux-x86_64.egg/jcc/sources -I/usr/ 
> include/python2.5 -c build/_lucene/__wrap__.cpp -o build/temp.linux- 
> x86_64-2.5/build/_lucene/__wrap__.o -fno-strict-aliasing
> cc1plus: warning: command line option "-Wstrict-prototypes" is valid  
> for Ada/C/ObjC but not for C++
> In file included from /usr/include/python2.5/Python.h:8,
>                  from /usr/lib/python2.5/site-packages/JCC-1.9-py2.5- 
> linux-x86_64.egg/jcc/sources/JCCEnv.h:54,
>                  from build/_lucene/__wrap__.cpp:2:
> /usr/include/python2.5/pyconfig.h:932:1: warning: "_POSIX_C_SOURCE"  
> redefined
> In file included from /usr/include/stdio.h:28,
>                  from /usr/lib/jvm/java-1.5.0-sun/include/jni.h:21,
>                  from build/_lucene/__wrap__.cpp:1:
> /usr/include/features.h:150:1: warning: this is the location of the  
> previous definition
>
> Please advise on how I can build this on a machine with low-ish RAM  
> resources.
>

You can change the value for the
--files parameter (see Makefile) which is 1 by default.
Increasing it causes __wrap__.cpp
to be broken up into about that many pieces. For PyLucene, a one-chunk  
__wrap__.cpp is over 100000 lines of C++.

See jcc's README for more info.

Andi..


> Thanks,
>
> Peter
>
>
> -- 
> Peter MacRobert
>
> > Facilitating value-added convergence by harnessing collaborative  
> metrics.
> _______________________________________________
> pylucene-dev mailing list
> pylucene-dev at osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/pylucene-dev/attachments/20080601/6dc0bc70/attachment.htm