[Cosmo-dev] Re: [Scooby-dev] Baselining Performance on Cosmo

[Mikeal Rogers]

I have a bunch of suggestions for feedback and additional tests in  
line below.

-Mikeal

On Jun 14, 2006, at 1:42 PM, John Townsend wrote:

> One of the things that we would like to do in the short term is to  
> put in place a small set of performance tests that measure basic  
> operations that clients would do against Cosmo.
>
> Once we have these tests in place, we would perform baseline  
> measurements in a controlled setting. With each release, we would  
> run these tests again to make sure that our performance in the  
> basic set of operations hasn't slipped.
>
> To get things started, Brian Moseley and I brainstormed last week  
> about the "Top 5" operations that we should consider testing.  
> Here's that list:
>
> - FoxMarks usage
>   - GET a regular file (WebDAV)
>   - PUT a regular file (WebDAV)
>   - signup via CMP
>
> - Chandler usage
>   - MKCALENDAR (CalDAV)
>   - MKTICKET on that calendar (CalDAV)

Do we want to MKTICKET on an empty calendar?

>   - PUT events into calendar (CalDAV)

How many? what type (recurring?), over what time frame & how often in  
that timeframe?
Multiple ics files for each event?, one large ics file for all events?

>   - Syncing
>      - PROPFIND on calendar (both large and small calendar) (CalDAV)

How large? How small? What kind of events, see above for examples.

>      - GET on resource

What is in the resource?

>      - PUT on same resource

How large is the resource already.

REPORT?
Depth?

>
> I don't have basic Cosmo usages for Scooby. Bobby, can you help me  
> out with that? Does anyone have anything else we should be  
> measuring regularly.
>
> The plan is that once we have a final reviewed list, our QA team  
> can go to work creating these tests and measuring these operations  
> with each RC candidate that we give them. We should also be able to  
> run these tests in our development environments as well.

What hardware? What jvm environment?

How many requests per second? How long should PUT's take after x  
amount of events added?

How many users are we sending simultaneous requests to?


>
> I am posting this now and planning to do a Last Call on it on  
> Friday morning. I hope to have a final list we can hand over to QA  
> on Monday.
>
> Thanks,
> --> towns
>
> _______________________________________________
> scooby-dev mailing list
> scooby-dev at lists.osafoundation.org
> http://lists.osafoundation.org/cgi-bin/mailman/listinfo/scooby-dev