[Dev] 0.7 perf goals
heikki at osafoundation.org
Tue Dec 6 13:28:33 PST 2005
Alec Flett wrote:
> For instance, say I broke something yesterday at 5pm, and the delta
> shows up at 7pm and displays until 9pm. Nobody is going to notice that.
> We'll all come in in the next morning and see that some of the
> performance numbers are red... but aren't there always SOME numbers red?
> Only someone such as yourself, who knows the tests well enough to know
> what's normally red and whats not, will notice that something is amiss.
I actually tend to take a look at the perf graphs as the last thing each
evening, so I tend to notice them (looking at the day graph). Obviously,
I'd rather be able to take a look next morning but that isn't possible
until we get the graph extending further back.
If the perf jump happened on all platforms with the same checkin, then
it seems real even if there is some random noise. Changes on a single
platform are harder, but if the jump levels off around the new value,
then it's usually real.
The Mac has the most fluctuation in values, so it is the least reliable
of the perf boxes we have. There are also some tests that have more
fluctuation than others. Generally, the less time a test takes the more
it is susceptible to fluctuation. Some of the worst tests fluctuate up
to 30% on the Mac, but when you plot a line through it is pretty level.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 253 bytes
Desc: OpenPGP digital signature
Url : http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20051206/98f461b8/signature.pgp
More information about the Dev