[Chandler-dev] Pre-loading items as a performance optimization

[Morgen Sagen]

On Oct 4, 2006, at 2:00 PM, Grant Baillie wrote:

>
> On 4 Oct, 2006, at 10:03, Morgen Sagen wrote:
>
>> I'm looking into improving the performance of going backwards/ 
>> forwards a week in the calendar view (bug 5651), so first I got  
>> some profiling numbers under the following circumstances (on my  
>> dual-1.8MHz G5):
>>
>> Imported the test 3000 event calendar into Chandler and quit.   
>> Restarted Chandler and jumped forward a week, which takes 2.0  
>> seconds.  0.75 seconds is spent generating occurrences; of that  
>> 0.75, 0.54 seconds is taken up by inserting keys into indexes.
>>
>> Restarted Chandler and repeated jumping between the same weeks as  
>> before.  This takes 1.0 seconds.  In this case, the occurrence  
>> items are already generated, and instead we're loading them from  
>> the repository.  0.3 seconds is spent loading the items.
>>
>> Repeating the jump again takes 0.6 seconds, which is further proof  
>> that item loading is taking between 0.3 and 0.4 seconds in this  
>> example.
>>
>> My thought was that if we could generate occurrences and/or load  
>> relevant items from the next and previous weeks in the background  
>> (Andi suggested OnIdle), that could save some time when the user  
>> clicks the week navigation arrows.  Does anyone see any pitfalls  
>> in this approach?  Is it possible to load items into the main  
>> repository view from another thread?  Also, I would need some  
>> guidance in determining exactly which items need to be pre-loaded.
>
> Along these lines, BTW, it seems that there are gains to be had by  
> removing the loading of items from the mEnd() and mStart()  
> functions in Calendar.py. I'm testing changes that do this, and  
> these do improve things a lot (0.59s to 0.22s for  
> PerfLargeDataJumpWeek in the measurement I did).

I did try replacing those mStart and mEnd calls like this:

    testVal = getattr(EventStamp(view[key]), startAttrName)

with calls like this:

    testVal = view.findValue(key, getattr(EventStamp,  
startAttrName).name)

to avoid loading, but overall a similar number of items was being  
loaded anyway (32 items loaded versus 34, if I recall).  Also, there  
are some cases where findValue( ) doesn't work -- when an attribute  
is Calculated, the item needs to be loaded and getattr used.

~morgen