[Chandler-dev] Performance of first time view switch vs subsequent switching

[Andi Vajda]

On Tue, 13 Feb 2007, Heikki Toivonen wrote:

> D John Anderson wrote:
>> Actually it's a little more complicated than this.
>>
>> There are 3 situations:
>>
>> 1) we need to copy a new tree of blocks and need to attach new contents
>> 2) we can reuse a cached copy but need to attach new contents
>> 3) we can reuse a copy and don't need to attach new contents
>
> Does that also cover the case where we have everything loaded into
> memory vs. things coming from disk? I.e. cases where you have had
> Chandler running for a while and have switched already between
> collections, and the case where you shut that Chandler instance and
> restart it with the same repository?
>
> It is starting to look like we can't reasonably measure all cases. So do
> we continue the status quo or measure something else? And if something
> else, what is that?

I'm not sure but you guys may be barking up the wrong tree. In the context of 
perf tests, there is a 3000 event calendar involved at some point. If 
importing that calendar results in a __repository__ directory with lots of 
so-called __db.freezer.nnn.4K files (I've got up to 2,000 at times), then any 
subsequent repository I/O is considerably slower. The solution, according to 
Sleepycat, is adding more cache to the repository environment (currently set 
to 64 Mb) in order to avoid this overflow scenario.

I've posted a question on the sleepycat forum about this. Namely, how can I 
get rid of these files once they're there as it doesn't look like they're all 
going away after the import completes...

Andi..

ps: two workarounds are currently available:
     - increase cache (edit line 219 of DBRepository.py, I can add yet another
       command line flag :) )
     - run without MVCC support with --nomvcc