[Dev] 0.7 perf goals

[Mitchell Kapor]

Agree with Katie's comments, plus I'd add I think it's important to  
identify "typical" case requirements, e.g., a typical calendar might  
be 500 items (or whatever) and then build performance goals around  
the typical cases.

On Dec 6, 2005, at 10:47 AM, Katie Capps Parlante wrote:

> Heikki Toivonen wrote:
>
>> There are a couple of approaches we could take. One is to  
>> concentrate on
>> the cases that are furthest from our 0.6 goals. The other is to  
>> take a
>> broad look and work on all the cases that are not yet under the ideal
>> limits. In the first case our numeric goals would remain  
>> unchanged. In
>> the second case we would set new, tighter goals for the cases that  
>> did
>> make it under acceptable limits but are not yet under ideal limits.
>> Personally I am in favor of the latter. I have some initial  
>> suggestions
>> here:
>> http://wiki.osafoundation.org/bin/view/Journal/ 
>> ZeroSevenPerfGoals20051204
>>
>
> Another approach we could take is to increase the size of the  
> collections (and repository) that we expect to handle. Instead of a  
> 3000 item calendar, we could say a 10,000 item repository, with  
> several collections/calendars. Nailing down the tenets might help  
> us pick a good target, but the general goal here would be to  
> ratchet up the size of the repository. We could combine this with  
> focus on a specific set of cases and/or ratcheting down the  
> response times.
>
>
>> There's also a question of policy. Do we want to mandate a strict  
>> policy
>> of not allowing performance regressions (regressions backed out, no
>> questions asked), or do we consider each regression on a case by case
>> basis? The danger with the second approach is that if we leave the
>> regression in, and other code starts piling on top of those  
>> changes, it
>> can be hard to back out even if we wanted to. Personally I am  
>> slightly
>> in favor of a strict policy, but the other approach might also  
>> work well
>> if we have just a few regressions.
>> Mozilla has long maintained a very strict policy where regressions  
>> are
>> simply backed out. If you really wanted your fix/feature in, there  
>> were
>> two options: make a change such that performance does not regress, or
>> convince everyone that your change is so important it trumps  
>> performance.
>>
>
> This approach seems pretty reasonable for Mozilla (or Safari, which  
> I hear has a similar system), where the intense focus is on a  
> particular well-known use case. In our case, we're continuing to  
> tweak the functionality and even change the formulation of the use  
> cases that we measure. I think it would be overly constraining at  
> this point to have a formal "back all regressions out" policy.
>
> Cheers,
> Katie
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "Dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/dev
>