[Cosmo] Re: cosmo work
bcm at osafoundation.org
Thu Jan 12 14:02:02 PST 2006
On 1/12/06, Heikki Toivonen <heikki at osafoundation.org> wrote:
> Apologies for this taking so long (all the Chandler releases processes
> and pending bug fixes...), but now I finally have some cycles for Cosmo.
we can put a lot of load on cosmo with hammer, but it's deficient in
several ways. it only does GET and PUT of fixed sized files, so while
it simulates typical foxmarks usage pretty well, it's not very
realistic in terms of chandler synchronization. furthermore, it
doesn't test edge cases like PUTing gigantic files or calendars with
thousands of events.
i'd like to see either hammer extended or a new tool found/built that
can simulate the chandler sync cycle, specifically sharing a calendar
(MKCOL, MKCALENDAR, PUT, PROPPATCH), syncing a calendar (PROPFIND,
GET, PUT, PROPPATCH), unpublishing a calendar (PROPFIND, DELETE).
i've looked at jmeter in the past, and i sent the url of a similar
tool to this list earlier today (have already forgotten its name).
extending one of those to do webdav/caldav may be an option.
whatever choice is made, we should have some kind of harness that lets
us vary the amount of load that we put on the server and that records
the relevent output in a way that's easy for us to publish (on the
wiki, in a document, etc). that probably means html and/or pngs.
aside from putting load on cosmo and testing various scalability
scenarios, jared has requested a suite of functional tests that he can
run to verify a cosmo-demo upgrade. it strikes me that this might be
similar/the same/a subset of whatever tests qa uses/will use to accept
a cosmo build. but then again, maybe not, i'm not up to speed on
testing methodology ;)
in any event, i'd love to have a set of end to end tests that execute
outside the server in order to verify dav, cmp and ui functionality.
nothing that i'd run before committing a change to svn (that's what my
in-server unit tests will be for), but something we can run whenever
releasing builds to qa and for jared to run to verify that his
cosmo-demo upgrade is working as expected.
i rank the functional tests as higher priority, since we do have a
basic load testing setup that has gotten us quite far with 0.2.
oh btw, we should be developing all tests against the trunk, since
0.2.6 is quiescent, seemingly stable and not in need of another
maintenance release (*knocking on wood*). we can backport anything we
thanks heikki, glad to have you aboard! and if anybody out there in
the world feels the urge to lend a hand, let us know :)
More information about the Cosmo