[Dev] Special session -- IRC Office Hour - Part II, Wednesday, 14 September 2005, 11AM PDT

[Heikki Toivonen]

Katie Capps Parlante wrote:
> We now have bugs for the 0.6 top use cases:
> http://wiki.osafoundation.org/bin/view/Projects/PerformanceProject

There doesn't seem to be instructions nor times for the startup test. I
have a proposal for a way to test this, although it is not as accurate
as I'd like, so we may want to tweak it.

I added these instructions on the wiki as well:

 Startup

    * Create script file end.py: import sys; sys.exit(1)
    * Create a new repository by starting Chandler with --create
    * Start Chandler 3 times with: time chandler -f end.py
    * Pick the run whose time was in the middle (the "real" time row)

The thing is, it seems to take 1-2 seconds (for me and bear) between the
time the UI comes up and Chandler goes away. The UI is also usable - I
can click buttons etc. A more accurate way might be to add --exit
command line option which would effectively call sys.exit(1) as the last
thing in Chandler startup

Also, the other tests don't have anything about running the tests
multiple times. Performance tests are notorious in that way - you have
to run them several times and get some kind of middle/average value, as
there is otherwise too much noise in the system. On the other hand, if
it is something run by hand you don't want to do it too many times. I
think 3 or 5 times (drop highest, lowest, average rest) would be
generally ok.

-- 
  Heikki Toivonen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://lists.osafoundation.org/pipermail/dev/attachments/20050913/a8065c35/signature.pgp