[Chandler-dev] Checkin 5/15/2008

Mimi Yin mimi at osafoundation.org
Thu May 15 11:30:06 PDT 2008

- finishing up acceptance tests
- had a bunch of items to update in Chandler - status notes etc
- going to do a few new metrics graphs to see if there was a change  
in downloads since the new landing page.
- started to plot some revised numbers based on some suggestions from  
JeffreyH. Jeffrey, I changed the data to use the difference in hub  
syncs vs downloads. The problem is the difference in hub syncs is  
negative for some months.
- working on 1.0 launch items

- spent most of yesterday stomping out problems with that debian bug,  
but pretty sure we're good now
- got butt saved by JeffreyH on smtp server problems
- today working on web work queue until just now when randy found a  
serious client side timezone bug in 0.15 - still hoping to fix that,  
do some testing and start release process for 0.15 today

- found and fixed a problem with undo of commit logging...   
implemented draft cellcache decorator w/tests
- replaced the Time service's ad hoc mechanism with the cellcache  
mechanism, more tests...
- draft of the "pub/sub hub" type, with tests...  and it works, yay!  
i.e., it's a fully generic publish/subscribe component for the  
trellis, even though the plan is to use it for relationship  
management of birefs
- need to finish more tests and nail down some minor details of the  
cellcache tool
- hope to checkin today, then do cellcache docs next week
- then: ripping out old API, fixing up SQLAlchemy hooks, generally  
getting ready for an 0.7a2 release of Trellis

*jeffrey - do you have an idea of what performance is like of  
managing birefs, pje?
*pje - no but plenty of low-level stuff in trellis that be trivially  
moved to C (ie. pub-sub notify rule is made of very trivial looping)
-- in general, i've kep an eye in the design of trellis to keep  
things that happen a lot very simple, so that they can be ported to C
*jeffrey - yup, i'm not expecially worried, just curious - i imagine  
you'll fill us in once you get the optimization stage
*pje - Well, we won't get to optimization until we actually get back  
to app-level work.   Trellis has a huge amount of tests, and they run  
in 1.5 seconds on my PC right now, so no need to optimize yet.
* jeffrey - sweet

* pje - Essentially, when you make or break a biref link, a rule will  
put the info into a pub-sub hub, and then the usual update rule of  
the collection will just receive news of the link via the hub. So the  
overall performance of biref maintenance should be equivalent to  
inserting or removing the link from both sides, plus the pubsub overhead

- make 0.7a2 release w/o actual biref support yet
- just cleanup/consolidation of various changes and API additions
* connect/disconnect API
* cellcache API
* change connect/disconnect to operate in a separate pulse
* allow rollback across multi-pulse transactions
* various bugfixes
* pub/sub hub
* Time service using cellcache
- and about to pull out the old API and some of its roots.
- That's a lot of refactoring, so I think another release is in order  
to consolidate that.

*jeffrey - Really, for 0.7chandler, I think our performance issues  
live mostly in the pub/sub layer.  I think the Trellis will give us a  
big win just by allowing us to decouple things and test them in  
isolation, but have you thought about building in some level of  
instrumentation for pub/sub?
* pje - what kind?
* jeffrey - dunno, obviously we can just run the profiler, but
* pje - btw, the pub/sub is indexed by the values you're subscribing  
to, so only small set of match rules are checked against a given msg  
- if that's still not efficient enug, the match loop (innermost) is  
trivial to reimplement in C
* pje - personally, doubt that bottleneck will be there. trellis in  
general has more overhead than the pub/sub mechanism does :)

* grant - we should be able to write some kind of functional perf tests
* grant - IMO, in 0.7chander, our perf issues have a lot to do with  
working set
* jeffrey - agreed grant
* pje - working set? you mean number of items in play?
* grant - items or python object sin general, eg. indexes, skiplists,  
lower-level bdb structures

* pje - btw, unlike repository birefs ( believe) the pub/sub  
mechanism is loosely coupled, if the opposite end of the ink is not  
in memory, it won't be loaded (ie. if you have one-to-many links  
where one side has a huge collection, but you haven't accessed huge  
collection, pub/sub will just drop the msg on the floor)
* jeffrey - don't have specific suggestion, just musing to myself  
that it'd be nice to have log file that tells you how the publish  
callbacks have actually taken
* pje - that would be more overhead than the pub-sub, you'll see when  
you see the code, it's ridiculously simple
* jeffrey - right, don't mean pub/sub machinery is problem, mean that  
dispatching often adds a layer of obfuscation that can mask a problem  
in various callbacks
* pje - ah, but there are no "callbacks" in trellis - so already  
thoroughly obfuscated ;)
* jeffrey - heh, right - anyway, we should let mtg progress

- more audio slideshow work
- unifying title pages, tweaking text
- replied to bunch of list mail - got more people on board to do user  
stories, etc

- My day was eaten by the Debian beasties, well, by the clamav  
beasties, which happened to be triggered by the Debian beasties
- Anyway, mail's back up now, and I caught up on my Java, coding  
today, I hope

- Fix mnemonics/accelerators in Andreas's 0.7.6 German .po file.
- Test 0.7.6-rc1 downloads on all platforms.
- Some minor internal cleanup; will check in changes to trunk today.
- Assuming remaining acceptance tests pass, will release 0.7.6 today.

- spent a bunch of time yesterday debugging some ical4j timezone  
issues, pulled out my hair, drank heavily, submitted a bug  
documenting what I found and will work with the ical4j dev (Ben) to  
get it resolved at some point (the issue has been around forever so  
its nothing new)
- trying to shake out 0.15 bugs before we call it cooked
- once 0.15 is out, I'll go back to 1.0 bug queue
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20080515/3e2484c0/attachment.html

More information about the chandler-dev mailing list