[Chandler-dev] Checkin process when the tree isn't green?

[Bryan Stearns]

I don't want to distract people from working to make the tinderbox green 
again, but I don't think we're following the process we had in place (or 
maybe the process isn't what I thought it was :-) )...

I thought the general idea (see 
http://wiki.osafoundation.org/bin/view/Projects/CheckinRules) was:
- You're not supposed to check in unless the tree is green across the 
board, unless you're fixing a failing test.
- When you check in, you're supposed to be around to make sure your 
checkin doesn't ungreen the tree.
- If a failure occurs after your checkin, you're supposed to let others 
know that you're working on it (through one or more of: dev-list 
message, tinderbox note, or IRC comment)
- If you're not around, others may back out your change.

That doesn't always work, however: we've occasionally gotten into 
situations where, in spite of the above, multiple changes cascade into 
an extended period of failing tests -- like, *now*. In times like this, 
I thought the idea was:
- P1 bugs get filed for each failure
- The failing tests get disabled on the platforms on which they fail 
(with bug numbers in the comments)
- People are assigned to fix those bugs and reenable the tests.

This has worked well before, but we don't seem to be doing it now, and 
the problems are going on long enough that people (me, anyway) are 
spending lots of time trying to get tests to pass locally before 
checking in (because if a failure is not caused by my local changes, I 
end up running all the tests at least three times before figuring this 
out: this can easily eat an hour or more. Some days, I only have a few 
hours of productive focused coding time, so this flatlines my 
productivity). People (me again) also come to ignore the build-failed 
messages, and learn that checking in when the tree is partly yellow is 
an accepted part of the development process.

Or is the process supposed to be something different? (and if so, please 
update that wiki page!)

...Bryan