[cosmo-dev] steps to push new cosmo release out

[Adam Christian]

I am going to try to stay as available as possible for helping you  
guys get this going. I am committed to the continuing improvement of  
Windmill and as I won't be able to dedicate the same kind of time to  
running and writing Cosmo tests but I will certainly do what I can.

One note that may or not be of value is that we do have people  
successfully continuously and reliably running Windmill tests with  
buildbot. I am not familiar with buildbot myself but I'm sure  
assistance can be found if you wanted to try going down this road.

Adam

On Jan 17, 2008, at 1:19 PM, Travis Vachon wrote:

>>
>> Bear, what's the current status of windmill tests performed by tbox?
>>
>> As part of validation, Adam would watch the windmill tests run to  
>> see if anything weird happened, and also doing a manual run-through  
>> of currently-not-working windmill targets like IE6.  I doubt Adam's  
>> up for that anymore, so we either need to replace that whole round  
>> of work or reduce the number of windmill runs that we watch.
>>
>
> I'd be happy to own the windmill testing process, but "watching for  
> weird stuff" sounds totally infeasible. The windmill tests take at  
> least 30 minutes to run on firefox last time I checked and are about  
> as engaging as paint drying. Multiply that by 4 platforms and x  
> number of failed runs that require re-runs and this quickly spirals  
> out of control. In any case, "weird stuff" is almost impossible to  
> nail down, even behavior that looks normal could easily be  
> unexpected behavior.
>
> Windmill outputs messages to the console on test failures that I  
> suspect we could use to make windmill testing fully automatic. One  
> problem with this is that we don't have asserts properly verifying  
> many of the tests, but that's something we need to improve anyway.  
> I'd suggest:
>
> * Work on making windmill tests runs fully auto.
> * Fix assert failures as we notice them missing.
> * Devote some amount of time to an "assert audit" of the windmill  
> tests. This would be a good opportunity to acquaint anyone  
> interested with the test system.
>
> -Travis
> _______________________________________________
> cosmo-dev mailing list
> cosmo-dev at lists.osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev