[Cosmo-dev] Dojo widget/event performance benchmarking

[Matthew Eernisse]

I had already done a bit of this benchmarking after the initial widget 
conversion with the dialog box, but I thought Ted's e-mail about 
performance in the UI was a good reason to do a bit more testing and 
write up what I found.

WIDGETS

This was my first go at serious Dojo widget work, and it was the first 
time I've had to poke around in the innards of them -- trying to figure 
out the best way to plug them into our existing code.

I was frankly a bit appalled at the sheer volume of methods/props loaded 
as boilerplate into each widget -- and some of them, seeming convenience 
methods like 'setWidth' simply return 'dojo.unimplemented' so you have 
to implement them yourself anyway. It's a real kitchen-sink approach.

This was a good opportunity to do a real apples-to-apples comparison in 
performance -- and based on the measurements I did before and after the 
conversion, there is a distinct performance cost for using Dojo widgets 
for your UI components. (I used a simple 'var dtA = new 
Date().getTime(); var dtB = new Date().getTime(); var diff - dtB - dtA;' 
approach bracketing the method in cal_main.js that invokes the dialog.)

Here's the comparison in the amount of time neeed to display the modal 
dialog box -- before, and after the conversion.

1. The median number from invoking the box once (just by clicking), five 
times.

Plain JS/DOM/CSS
Firefox: 4ms
IE6: 0-16ms
IE7: 0-16ms
Safari: 22ms

Dojo widget
Firefox: 15ms
IE6: 16-33ms
IE7: 15-33ms
Safari: 33ms

2. The average time for invoking the dialog 10 times in a programmatic 
for-loop.  (I did this because the Windows system clock apparently has 
10ms/tick resolution, so the initial measurements were pretty worthless 
on IE -- I consistently got figures of 0ms, 15/16ms, or 32/33ms.)

Plain JS/DOM/CSS
Firefox: 3.1ms
IE6: 14.0ms
IE7: 10.9ms
Safari: 21.1ms

Dojo widget
Firefox: 9.9ms
IE6: 26.6ms
IE7: 15.6ms
Safari: 42.9ms

EVENTS

I also did a side-by-side comparison with dojo.event.connect, a single, 
local listener, and, for the sake of more comparison, the event handling 
system in my own Fleegix.js library (which does not include all the 
double-zoomie AOP stuff in dojo.event.connect). I did my testing with 
the trunk version of both Dojo and Fleegix.js.

The good news is that my original tests in Firefox were far off the mark 
-- a lot of the time in the test was actually time spent groveling 
through the DOM looking for the element to attach the event to. The bad 
news is that Dojo's event.connect is exceedingly slow in Internet 
Explorer (both 6 and 7), and performance degrades seriously the more you 
use it.

Attached is the code I used to do the event-attachment test. Please let 
me know if there's a better or more accurate way to test, or to focus 
better on the actual event-listener attachment piece itself.

Here are some of the tests I ran:

1. Attach a single event to each of 100 divs in a loop.

dojo.event.connect
Firefox: 116ms
IE6: 687ms
IE7: 546ms
Safari: 766ms

fleegix.event.listen
Firefox: 16ms
IE6: 16ms
IE7: 0ms
Safari: 24ms

Single, local listener
Firefox: 7ms
IE6: 0ms
IE7: 0ms
Safari: 7ms

2. Attach a single event to each of 100 divs in a loop, clear document 
(using disconnect/unlisten on each div), then do a new run with 100 new 
divs.

dojo.event.connect
Firefox:
initial run 119ms, clear 98ms, second run 125ms
IE6:
initial run 578ms, clear 1438ms, second run 3750ms
IE7:
initial run 562ms, clear 1406ms, second run 3672ms
Safari:
initial run 354ms, clear 341ms, second run 426ms

fleegix.event.listen
Firefox:
initial run 16ms, clear 7ms, second run 16ms
IE6:
initial run 16ms, clear 15ms, second run 16ms
IE7:
initial run 0ms, clear 16ms, second run 15ms
Safari:
initial run 25ms, clear 22ms, second run 27ms

Single, local listener
Firefox:
initial run 7ms, clear 4ms, second run 7ms
IE6:
initial run 16ms, clear 0ms, second run 15ms
IE7:
initial run 0ms, clear 16ms, second run 15ms
Safari:
initial run 7ms, clear 9ms, second run 7ms

3. Attach a single event to each of 200 divs in a loop.
dojo.event.connect
Firefox: 213ms
IE6: 4313ms
IE7: 4047ms
Safari: 831ms

fleegix.event.listen
Firefox: 36ms
IE6: 16ms
IE7: 16ms
Safari: 52ms

Single, local listener
Firefox: 14ms
IE6: 15ms
IE7: 0ms
Safari: 18ms

The takeaway here is that anything over the initial 100 invokations of 
dojo.event.connect rapidly becomes unusably slow in Internet Explorer 
(trying to attach 300 events took 16.5 seconds, 400 took 44.2 seconds, 
500 caused IE to become non-responsive). Clearing attached events before 
adding more seemed not to make any difference.

CONCLUSION

The Dojo widget system adds a lot of convenience and approachability to 
our code, and the Dojo event system gives us a lot of power.  But "with 
great power comes great responsibility," so we're going to have to be a 
bit careful about how and where we use this stuff (i.e., avoid it in 
performance-sensitive or repetitive pieces of the UI), given the impact 
it appears to have.

We should also test in all supported browsers a little more proactively 
as we go, to ensure that we're not creating something that will be 
totally unwieldy for people not using Firefox.

I'm also really interested in finding a way to get performance 
benchmarking into our QA for the UI so that we can decide on and 
maintain some minimum performance marks as the app gets bigger and more 
complex. It's better to be proactive than have to play catchup later.


Matthew

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/cosmo-dev/attachments/20061122/f7b1c986/index.html