[Dev] Build Sheriff and improved tools

Katie Capps Parlante capps at osafoundation.org
Wed Feb 1 18:37:02 PST 2006

We had an in-person conversation about build sheriff and improved tools 
at our weekly platform group meeting (1). After some discussion, we came 
up with this plan for the short term:

+ Bear will write a script that sends email when a Tinderbox goes red. 
It will only send email to people who have checked in after the last 
green build was started, or some similarly reasonable algorithm. PJE and 
Bear will collaborate on the algorithm.

+ We will not run this script against the performance Tinderboxes until 
they are more consistently green.

+ All committers are responsible for running unit tests and making sure 
their commit did not cause Tinderboxes to run red or orange. We do 
understand that developers don't run tests on every platform before 
every commit, and sometimes commits in quick succession will cause 
conflict problems, so sometimes a commit will cause a problem.

+ Committers are responsible for noticing when their commit has caused a 
problem. Committers can watch Tinderbox, or respond to the email.

+ We will be proactive about disabling tests that cause false positives 
(bogus test failures). In particular, we will disable tests that do 
outside network operations. (The long term solution is to create mock 
servers or clients for this purpose).

+ Heikki, Bear and others will be proactive about communicating about 
well known common false positives, so that they are easily recognized.

+ Heikki is going to proceed with drawing up a "sheriff" schedule. We 
anticipate that the above measures will be enough that we can all "self 
police" in most cases. The sheriff's role is to monitor the build and 
make sure build problems don't fall through the cracks.

The core of the proposal is that comitters will stand up and take 
responsibility for not breaking the build or the tests. We'll also 
invest in the build, tests, and Tinderbox so that we avoid wasting 
people's time with false alarms.

Thoughts on the proposal?

In the long term, we can continue to work on the projects that have been 
discussed, making Tinderbox ever more useful.

Personal aside: I'll point out that it is *very* bad form to do a commit 
without running tests and then disappear for hours. It is a lack of 
respect for other people's time, which is counter to our values here.


(1) http://wiki.osafoundation.org/bin/view/Journal/PlatformMtg20060131

Heikki Toivonen wrote:
>> A few people offered to help with tools. Here are some tools we could
>> have some improvements with (all that I could remember/think of):
> I haven't heard any takers for the projects I mentioned. If tool
> improvements are not happening, then the manual process is the option we
> are left with.
> ------------------------------------------------------------------------
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> Open Source Applications Foundation "Dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/dev

More information about the Dev mailing list