[pylucene-dev] DbDirectory

[Yura Smolsky]

Hello, Garrett.

I've build everything except Python 2.4 which _bsddb is built against
version BerkleyDB I use. But I am still receiving exceptions from
Java, when I add much data in one transaction.

What are you Lucene for?

GS> I've concluded that I have to compile PyLucene myself given the
GS> instability with DbDirectory. I need to get more info on the errors that
GS> are occurring. I'm now having problems with the DB_DIRTY flag.

GS> I'm also running into seemingly random crashes in Python, which I think
GS> are being caused somewhere during Java's exception processing. I wonder
GS> if the fact gcj is compiled in a cygwin environment while everything
GS> else is VC++ is turning up some problems.

GS> In any event, Lucene in a Python environment is compelling enough to
GS> merit the pain (I hope). I'll post whatever I can here.

GS>  -- Garrett

GS> Yura Smolsky wrote:
>> Hello.
>> 
>> Guys,
>> does it work on your Python-PyLucene-BerkleyDB platform?
>> Check commented place below.
>> 
>> I think Berkley DB Support is not very stable. Tests for it don't work
>> on my box.
>> 
>> Maybe I am doing something wrong?..
>> 
>> # create datadir
>> import tempfile, os
>> datadir = "indexdbok"
>> if not os.path.exists(datadir):
>>     os.mkdir(datadir)
>> 
>> # initialize dbenv
>> import bsddb
>> dbenv = bsddb.db.DBEnv()
>> dbenv.open(datadir,
>>     bsddb.db.DB_CREATE |
>>     bsddb.db.DB_INIT_TXN |
>>     bsddb.db.DB_INIT_MPOOL |
>>     bsddb.db.DB_THREAD)
>> 
>> # create files and blocks databases
>> txn = dbenv.txn_begin()
>> filesDb = bsddb.db.DB(dbenv)
>> filesDb.open('f.db', 'blocks', bsddb.db.DB_BTREE,
>>     bsddb.db.DB_CREATE | bsddb.db.DB_THREAD, txn=txn)
>> blocksDb = bsddb.db.DB(dbenv)
>> blocksDb.open('b.db', 'blocks', bsddb.db.DB_BTREE,
>>     bsddb.db.DB_CREATE | bsddb.db.DB_THREAD, txn=txn)
>> txn.commit()
>> 
>> 
>> # create an index writer
>> txn = dbenv.txn_begin()
>> from PyLucene import DbDirectory
>> dir = DbDirectory(txn, filesDb, blocksDb, 0)
>> 
>> from PyLucene import  StandardAnalyzer, IndexWriter, Document, Field
>> writer = IndexWriter(dir, StandardAnalyzer(), True)
>> writer.setUseCompoundFile(False)
>> 
>> # when I decrease the number of cycle then it works
>> # when I increase then writer.optimize() produces
>> # #   def optimize(*args): return
>> _PyLucene.IndexWriter_optimize(*args) # # ValueError:
>> java.lang.NullPointerException for i in range(20):
>>     doc = Document()
>>     doc.add(Field.Keyword("id", "1"))
>>     doc.add(Field.Text("title", "q"*100000))
>>     writer.addDocument(doc)
>> 
>> writer.optimize()
>> writer.close()
>> 
>> txn.commit()
>> 
>> 
>> filesDb.close()
>> blocksDb.close()
>> dbenv.close()
>> 
>> 
>> 
>> Yura Smolsky,
>> 
>> 
>> 
>> _______________________________________________
>> pylucene-dev mailing list
>> pylucene-dev at osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/pylucene-dev





Yura Smolsky,

-> AlterVision.biz <- Achieve Your goal!