[Chandler-dev] No more perf regression bugs?

Andi Vajda vajda at osafoundation.org
Mon Apr 16 15:51:28 PDT 2007


On Mon, 16 Apr 2007, Heikki Toivonen wrote:

> I've been under the impression that we cared about performance
> regressions, although not (yet?) enough to back out a change that caused
> one. My thought was that we'd get the bugs filed when they happened, and
> analyzed the changes later to determine what caused the regression. I
> also thought that it would be easier to determine the cause of the
> regression (and fixing it) by working with the change information rather
> than just doing regular performance work (start with profile etc.).
>
> However, some engineers at least don't want to work with this past
> change information, and won't be looking at performance bugs at all, as
> far as I understand.
>
> I'd like to know if everyone feels this way. If yes, then I can stop
> filing performance regression bugs. I'd probably be able to simplify the
> performance reporting tools quite a bit if we didn't care about
> regressions (like maybe only reporting the absolute test results on
> tbox, and nuking all other results).
>
> If some feel perf regression bugs are usable, then I have a question
> about what to do with perf regression bugs that are caused by an
> engineer who does not want to look at performance regression bugs.
>
> I am somewhat annoyed by the situation, mostly because I'd like to avoid
> doing work that is considered useless. I do understand people have
> different styles of approaching problems.

It depends on what is a regression. If the perceived regression happened 
because the test was bogus then the regression is bogus too. If the perceived 
regression happened because a new feature was added then the regression is 
also bogus. If the perceived regression is because a bug was fixed then the 
regression is still bogus.

That is not to say at __all__ that if the new code is too slow it shouldn't be 
made faster. All I'm saying is is that looking at it as a regression may not 
be very helpful in these cases.

Then, of course, there are cases where the regression is real. Something else 
changed somewhere and a particular test was made slower as a side-effect. 
These need to be looked at as a regression.

What can you do about your feeling of wasting your time ?
Making damn sure that the regression you're calling really is one.

Andi..


More information about the chandler-dev mailing list