 |
[Dev] ZODB for object persistence [Was: ZODB is not a Storage Technology]Mike C. Fletcher Mon, 11 Nov 2002 16:56:13 -0500
As a note, that's the strategy I took with ConflictSolver, and we (myself and my client) found it a very poor performer. ZODB4 requires that you basically re-create each in-memory object (call Connection.sync, which invalidates the whole cache in the current code) every time you do a transaction. Zope's transactions tend to be entire web-form submissions (relatively large interactions compared to "user actions" in a live GUI), so "user action"=transaction works fine there. As an idea of scaling, ConflictSolver's primary control (a scalable timeline) would (when it got slow) be accessing around 200 database items for every refresh, and accessing each 5 or 6 times (not a big deal beyond the first access). A transaction/action required that each object be deconstructed (ghostified), then re-loaded from the database. Given that users could generate at least 4 or 5 actions/second (sitting there, pounding on a particular button a few times), I wound up with a number of situations where the user was sitting around waiting for the display to redraw. Chandler (if it doesn't have an optimised cache) is likely going to follow a similar pattern, lots (hundreds, maybe thousands) of objects accessed for every display of a screen (with brute-force programming, rather than application-level caching). Forcing all of those objects to be reloaded for every user action is going to be a performance problem. The cache (which, as Jeremy says, is a work-plan for v4 final) would seem to be crucial to making this design approach work. Enjoy all, Mike John Anderson wrote: > Morgan wrote the the ZODB test program. I don't think he did a commit > at every creation. The way I was imagining using ZODB in Chandler, a > commit would happen once per "user command" and correspond to an > undoable action -- so if they should be relatively rare. > > Morgen: can you post the details of your test program. > > John Anderson > > Jeremy Hylton wrote: > >>>>>>> "JA" == John Anderson <john@osafoundation.org> writes: >>>>>>> >>>>>> >> >> JA> My test program for object creation using ZODB4 goes from >> JA> somewhere around 20,000 objects created per second to 300 per >> JA> second once you throw in transactions. I'd like to use >> JA> transactions for undo and resource exaustion recovery. >> >> How did you throw in transactions? If, for example, you added a >> commit after every object creation, I think I'd be happy if you could >> achieve 300 creations/sec <0.3 wink>. >> >> A transaction is a relatively heavyweight operation and object >> creation is an incredibly lightweight operation. I'm sure transaction >> commits can be optimized, but it's still going to be a big hit if you >> use very fine-grained transactions. Do you think that's necessary for >> you application? >> >> I'd also note that important parts of ZODB4, notably the object cache, >> are still using throw-away code written in Python. There's a reason >> we haven't released ZODB4 yet. For the Dec. alpha release, we'll have >> an object cache written in C that should improve performance. If >> you're looking to benchmark code, ZODB 3.1 will give you more >> realistic numbers. >> >> Jeremy >> >> PS My own crude benchmark (creation an object passing two args to >> __init__) was: >> >> classic class 118000 creations/sec >> new-style class 66700 creations/sec >> Persistent class 181 create+commit/sec (using FileStorage) >> >> > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > Open Source Applications Foundation "Dev" mailing list > http://lists.osafoundation.org/mailman/listinfo/dev > -- _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/
|