[cosmo-dev] Cosmo .10 testing status

[Jared Rhine]

> I am still in the process of automated testing...

Sorry, could you repeat the specific automation list?

(Full list is, I believe: Unit, Windmill, Protocol, Migration)

Is the status:

Unit: Done by tinderbox
Windmill: Done before RC
Protocol: Being done now
Migration: 0.10 includes a schema change, so being done now.

There seems to be an emerging consensus that with our new capability  
to do "one-button build" test instances, we can (and should) move from:

   (old): "cook RC, put on qacosmo, then kickoff formal testing"

to

   (new): "push 'next release button', confirm automated testing and  
sanity test of new features against that tree, then cook RC".

We did the Windmill validation before the RC this time, a nifty  
innovation.  I propose that we continue to do that; it just makes  
sense (of course, it will be done automatically by Tinderbox by at  
least a subset of supported browsers).

It seems to also make sense to go further all do all "basic automated  
validation" on a branch before cutting an RC.  So can we also say  
"let's do protocol tests against the push-button instance before RC"?

I can see possibly removing "migration testing" from the "pre-RC" list  
of things to do.  Depends on exactly how much we want to do before vs  
after an RC is cut.  I can also see including it, assuming we don't  
repeat after the RC is cut.

-- Jared