[Chandler-dev] deferring index re-indexing

[Andi Vajda]

Yesterday Friday on IRC, Bryan and Grant requested support for being able to 
set multiple attributes "at a time" on an item. By this, they wanted a way to 
turn off side-effects such as observers, monitors and index changes until all 
values are set. This could have significant impact on performance by avoiding 
redundant side-effects while values that belong together are set.

Implementing a general solution to this is tricky and we decided to start with 
having a way to temporarily turn off index re-indexing. Python 2.5's 'with' 
construct seems a good fit for this feature and I'm pleased to announce the 
new reindexingDeferred() API allowing to temporarily turn off re-indexing 
globally on a view:

    with view.reindexingDeferred():
        item.a1 = v1
        item.a2 = v2
        another.a3 = v3
        item.a1 = v4

Inside the with-block, re-indexing is turned off and accessing indexes that 
need re-indexing is denied, failing with a LookupError. Once execution exits 
the block, all indexes that need re-indexing are re-indexed automatically.

The performance advantage of this construct is that redundant re-indexing 
operations are discarded, only the last re-indexing change for a given item in 
a given index is kept.

These "with view.reindexingDeferred():" blocks can be nested, only when the 
outer one completes, the re-indexing takes place.

In order to support this feature, I had to rework index monitor support quite 
a bit. The Item._reIndex() method (that was overridden in collections.py) is 
gone and replaced by an optional onCollectionReindex() hook method.
See parcels/osaf/pim/collections.py for an example implementation.

Questions, comments are welcome.

Should I implement "with view.observersDeferred():" next ?
It could be nested with "with view.reindexingDeferred():" and the order of 
nesting would give some control over which side-effects are turned on and off 
and when.

Andi..

ps: to use the 'with' statement in python 2.5, you need to have a
         from __future__ import with_statement
     statement as the first python statement in your file.