[Chandler-dev] Chandler Desktop Releases
pbossut at osafoundation.org
Mon Sep 24 17:34:08 PDT 2007
Dan Steinicke wrote:
> Philippe Bossut wrote:
>> Proposal and points of contention:
>> - Release once a month: some proposed 6 weeks (4 weeks dev time, 2
>> weeks debug stabilization) but the idea is that we work under an
>> evergreen trunk policy so we don't destabilize trunk, also a monthly
>> release makes it easier to punt something to the next release (users
>> won't have to wait too long) than a 6 weeks cycle.
> It seems like the above statement assumes that a green tinderbox
> status == the trunk is stable. I don't think our tests are
> comprehensive enough for that to be 100% true. Certainly as we
> approached 0.7 release there were a number of bugs indicating
> instability which were not causing tinderbox tests to consistently fail.
> Maybe the tinderbox test are good enough for us to act like this is
> true (and hopefully the tests will continue to get better). If we
> decide to go forward with this plan I think we should understand we
> are accepting a lower standard for what a stable trunk is.
Good point Dan. Such a plan has a chance to work only if we do a really
serious effort with automated testing in the coming months. As you know,
John is working on improving the script recording system right now and
we will be discussing this tomorrow during a sprint meeting. Script
recording won't be covering all cases and it has its own challenges
(can't really verify for instance that the rendering is correct or even
that the result is the one intended) but, at least, it should catch
crashes and instability.
In the short run, we can get by by being extra careful and put more time
in human testing. In the mid term (beyond three months) though, this
would build too much frustration and can't be sustainable. This
certainly points to the fact that we should put testability up front in
our development effort, something which as it happens, was one of the
fundamentals pointed to by PJE in his initial post.
More information about the chandler-dev