[pylucene-dev] indexing performance

[Filip de Waard]

Hello,

Until today, I've never had a single worry about performance in my  
short but exciting Python experience. However, now I'm trying to  
index over six million books from a MySQL database using PyLucene and  
I'd like to speed it up.

I have posted my indexer script at http://pastie.textmate.org/75938.

Tomorrow I'll start playing with a profiler, but in the meantime:  
does anyone have any recommendations as to how to be most efficient  
in regard to the Python code, database interaction and of course the  
PyLucene indexing process? Or maybe I'm doing something horribly  
wrong in my script?

Any pointer would be most appreciated.

Regards,

Filip de Waard


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/pylucene-dev/attachments/20070704/838ee50e/attachment.htm