[cosmo-dev] steps to push new cosmo release out
adam.christian at gmail.com
Thu Jan 17 13:52:47 PST 2008
I am going to try to stay as available as possible for helping you
guys get this going. I am committed to the continuing improvement of
Windmill and as I won't be able to dedicate the same kind of time to
running and writing Cosmo tests but I will certainly do what I can.
One note that may or not be of value is that we do have people
successfully continuously and reliably running Windmill tests with
buildbot. I am not familiar with buildbot myself but I'm sure
assistance can be found if you wanted to try going down this road.
On Jan 17, 2008, at 1:19 PM, Travis Vachon wrote:
>> Bear, what's the current status of windmill tests performed by tbox?
>> As part of validation, Adam would watch the windmill tests run to
>> see if anything weird happened, and also doing a manual run-through
>> of currently-not-working windmill targets like IE6. I doubt Adam's
>> up for that anymore, so we either need to replace that whole round
>> of work or reduce the number of windmill runs that we watch.
> I'd be happy to own the windmill testing process, but "watching for
> weird stuff" sounds totally infeasible. The windmill tests take at
> least 30 minutes to run on firefox last time I checked and are about
> as engaging as paint drying. Multiply that by 4 platforms and x
> number of failed runs that require re-runs and this quickly spirals
> out of control. In any case, "weird stuff" is almost impossible to
> nail down, even behavior that looks normal could easily be
> unexpected behavior.
> Windmill outputs messages to the console on test failures that I
> suspect we could use to make windmill testing fully automatic. One
> problem with this is that we don't have asserts properly verifying
> many of the tests, but that's something we need to improve anyway.
> I'd suggest:
> * Work on making windmill tests runs fully auto.
> * Fix assert failures as we notice them missing.
> * Devote some amount of time to an "assert audit" of the windmill
> tests. This would be a good opportunity to acquaint anyone
> interested with the test system.
> cosmo-dev mailing list
> cosmo-dev at lists.osafoundation.org
More information about the cosmo-dev