[Cosmo-dev] Test Environment Discussion (pulled out of thread Mikeal was using)

[Mikeal Rogers]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> My big point is that if we want to cut the cycle time for release  
> down *even the normal QA test procedures have to be continuous*
>
> that's it in a nutshell - everything has to be running all the time  
> in order for bug discovery and bug fix cycles to shorten.

For manual testing, regresssions, bug verification, new test  
authoring, RC validation, I wouldn't want the server I'm using to  
change every checkin. I could definitely see this happening every  
night at Midnight PST, but not every checkin.

The exact same process for building, migrating, etc, we use for the  
test server instances should be run on tinderbox every checkin to  
catch all these issues. You said, Jared's script does a lot of "extra  
production stuff" and I think it's just as important that we know  
when this breaks as any other part of the build breaks and I think  
it's important that at least one of our day to day test instances  
looks just like production. If this doesn't fit in to the tinderbox  
setup then that's fine, seeing it run every day (at Midnight) just to  
setup the QA manual testing environment/server would be enough to  
start with.

I think the script that builds all this has to be considered a part  
of the cosmo product, just like all the maven build files are, and  
checked in to the cosmo repo. If something breaks this script it  
should be treated just like if someone broke any of the other maven  
builds.

- -Mikeal
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFGzyK3Iy1Ff2Qzo+wRAhzXAJ9O1IJi3rS+AHrfXxQ9RckUoal/mACbBXz1
E8HUQ2wH/ouioZ0rNxzcsZE=
=7g+5
-----END PGP SIGNATURE-----