[chandler-users] Chandler experiences: Memory and performance issues

[Andi Vajda]

On Fri, 6 Jun 2008, Peter Novak wrote:

> By no way do I want to start a flamewar here. But I cannot miss this
> opportunity for a small irony :-).
>
> On Fri, Jun 06, 2008 at 08:39:02AM -0700, Andi Vajda wrote:
>
>>>> So I have a question: Is there anything I can do to make Chandler run
>>>> faster on my computer except for buying a new one?
>>
>> Buying more memory is cheaper than buying a new computer. I recently
>> bought 4 gb of ram for a MacBook Pro for $70.
>
> So I will go and ask my boss to buy some new memory for my laptop. And
> when he asks why I need it, I will tell him that it is because of my new
> fancy organizer/calendar app :-D. Well, well... You can imagine the result ;-).

By no means did I suggest you go out and buy an extra 4 gigs :)

But the amount of time you spend waiting about things to happen in Chandler 
is more expensive to your boss than the price of an extra gig or two of ram, 
isn't it ? (tongue in cheek).

> Now seriously. Me being a software engineer as well, I feel to
> understand a bit of quality of software. As I already said, till now I
> like Chandler approach to PIM very much, so I am willing to forgive it
> small hickups and discomforts here and there. But the situation I
> described above clearly shows the absurdity here. A PIM application
> requires better HW than a set of software development tools! And the
> advice I get is to buy more memory... :-( How come that a simple
> database tool for storing 2000 entries is slower and less responsive
> than a 3D rendering tool, or a C++ compiler in a full run?

Ah, if only Chandler were a simple little database tool.

> Well, my feeling is, that this will one day become (if it did not yet -
> I don't know, I am too new here) a roadblock for Chandler's adoption.
> And it would be a pity as there seems to be a lot of potential in its
> very simple idea. I am still puzzled by it and I like it very much -
> well, I have a lot of time for it while Chandler does anything :-). I
> even noticed that when I move mouse over the main window menu, my CPU
> spikes to 100%... :-|

Something is clearly off here. We've had people complain about performance 
issues before, especially on Windows and there have been bugs and issues 
that were resolved here too. But not always. Some mysteries persist.

For example, sometimes a virus checker is in the way. If that's the case in 
your situation, you might want to consider configuring it not to check the 
__repository__ directory in your chandler profile dir (see [1]).

Last time this topic came up, it wasn't the case and the problem went 
unresolved. But two or three minutes to startup Chandler and to quit it is 
something way off the bounds I'm used to. For full disclosure, I'm not using 
Windows but Mac OS X. Still, Chandler is no speed demon, many things could 
be improved. But the experience you're reporting is off the charts I know 
of.

Another issue that has come up before is that starting Chandler after a cold 
boot takes a loooonger time than after a wake from sleep. On Mac, sleeping a 
laptop is the most common usage style. It seems to be less common on Windows 
or Linux, so there people are paying a higher price in startup time because 
nothing is cached. Most of the time spent in starting Chandler, believe it 
or not, is in dynamically linking the myriad of shared libraries involved, 
not in actually starting the program. Once the OS has done that one, all 
this is cached until you reboot or run out of RAM.

Andre, who also responded to your message earlier, said that Chandler was 
doing a full repository check() on quit. This is a debugging feature only 
and should not be used by default. If there is a bug on Windows (I don't see 
this on Mac) where somehow Chandler is misconfigured to run check() on quit 
always, it may take quite some time for you to do so. Does your Chandler say 
"Verifying data..." while quitting. If so, we may have an egregious bug here 
- simple to fix, but embarrassing :)

> I guess a lot of those performance issues are caused by how Python works
> and perhaps by the way how its garbage collector works. I am not a
> Python expert, but wouldn't it be possible to manually invoke the
> garbage collector more often? I cannot imagine that Chandler code
> genuinely allocates 200+ MB to handle such a small database... BTW, I
> also noticed, that when I iconify the main window, the memory consumed
> by Python drops to about 40-50 MB...

Python is not Lisp or Java. Its memory objects are refcounted and normally 
freed without garbage collection except for objets in reference cycles. 
Currently, gc.collect() is invoked when a repository view exceeeds the 
default limit in number of items currently resident and is purged of the 
least recently used items. This number is fairly small in the background 
views (calendar syncing, for example) and hence gc.collect() should get 
called on a regular basis.

Most of the low-level code in Chandler, including python itself, is 
implemented in C. Python performance is not on the hot seat here. The 
current implementation of Chandler is very busy with event notifications and 
that is felt throughout the app, especially in the UI area.

Andi..

[1] http://chandlerproject.org/bin/view/Projects/ProfileDirectory