[Chandler-dev] Reload performance

[Grant Baillie]

So, I had a look at the performance of reload on Friday. Hopefully,  
some understanding of what's going on here will help improve other  
EIM-related performance metrics (e.g. syncing shares, importing  
calendars). Here are my findings to far ... feel free to weigh in  
with questions/comments.

Since some observer methods (e.g. the one that updates  
ContentItem.displayDate) were prominent in profiles, I tried a little  
trick (or a couple of little tricks):

1. I changed it so that the reload ran before the main Chandler  
window appears.
2. I delayed creating the master ContentItem indexes until the reload  
was done.

This patch gives a 30-35% improvement reload times:

[My own repository, 1.8 GHz iMac G5]:
   Trunk (r14631): 5m38s
   Patch: 3m35s

[PerfDataSet02.dump, 2.16 GHz dual core Intel Mac]:
   Trunk (r14631: 8m30s
   Patch: 5m47s

What this seems to say is that we are spending a chunk of reload time  
indexing. So, you might wonder if deferring reindexing on the  
repository view would help here. It turns out that:

- Deferring indexing with the main Chandler window up doesn't work:  
there's an error updating the global event start & end time indexes  
(parcels/osaf/pim/calendar/Calendar.py).
- Deferring indexing without the main window gives me an error from  
the repository:

2007-06-11 10:04:12,917 application.Application ERROR: Failed to  
reload file
Traceback (most recent call last):
   File "/Users/grant/src/chandler/application/Application.py", line  
606, in reload
     activity=activity)
   File "/Users/grant/src/chandler/parcels/osaf/dumpreload.py", line  
208, in reload
     rv.commit()
   File "/Users/grant/src/chandler/repository/item/Monitors.py", line  
259, in reindex
     index.moveKeys(deferredKeys)
   File "/Users/grant/src/chandler/repository/item/Indexes.py", line  
509, in moveKeys
     raise KeyError, key

I'm not sure what's up with this last error ... looks like a  
repository bug to me. Andi, let me know if you want a patch & .chex  
file to reproduce.

--Grant