[Cosmo-dev] osaf.us VM and qacosmo

[Aparna Kadakia]

Just to summarize it, we would be using qacosmo for doing functional  
testing of the daily checkpoint builds. qacosmo gets wiped off with  
every checkpoint build and gets rebuilt. This is where cosmo  
automated tests will be run against as well.

qa.osaf.us wil be our qa staging instance mirroring production where  
we run the integration, load and performance tests that simulate real  
traffic. This will be cloned from the production box so we can  
isolate those issues before we go live. Typically qa.osaf.us will be  
tested against at the end of the release when we are closer to going  
live. I expect most of the next few qa sessions to be carried out on  
qa.osaf.us.

+1 to mikeal's proposal

On Jan 24, 2007, at 4:18 PM, Mikeal Rogers wrote:

> Forwarded from a private thread with Jared's consent.
>
> On Jan 24, 2007, at 9:28 AM, Jared Rhine wrote:
>
>> Mikeal Rogers wrote:
>>> Instead of provisioning a new machine for qacosmo I was thinking  
>>> it might be best to just add another virtual machine on osaf.us,  
>>> qa.osaf.us?
>>
>> That's definitely possible.  Are you thinking the purpose would be  
>> for QA on the service, or QA of cosmo in general?  If QA for Cosmo  
>> in general, how would the box differ in function from qacosmo?   
>> Are you looking specifically for something with greater horsepower?
>>
>> I'd like to help identify the issues existing with qacosmo.   
>> There's potential for FUD in this area, as well as real problems  
>> with qacosmo.
>>
>> If nothing else, a qa.osaf.us-like thing would help characterize  
>> problems that people do see with qacosmo; you can try to replicate  
>> such behaviors on another box and see.
>>
>> The multi-CPU aspect of such a VM can also be used for scaling  
>> tests, though I'm guessing that QA is not going to have much ready- 
>> to-go before Preview to test actual scaling (as opposed to raw  
>> performance, which seems to be progressing nicely.)
>>
>> If we proceed with a new VM for QA, let's get it on a list.  If  
>> it's for the service, it should be asked for on service-dev.  If  
>> it's for Cosmo, it should be asked for on cosmo-dev.
>>
>> Though there's some production traffic on the physical machine  
>> currently, after production moves to a new box, the physical admin/ 
>> VM box will be pretty quiet.  Maybe some late-night metrics- 
>> crunching.
>>
>> -- Jared
>>
>
> What I would like to do is have two seperate environments for two  
> different stages on testing.
>
> First phase testing would be checkpoint and daily testing of cosmo,  
> it would also be the server that we write new tests against etc.  
> I'd like to avoid issues with this environment that would block us  
> from testing and at the same time made sure that we aren't just  
> finding low priority bugs that aren't prioritized all that highly  
> because they are too different from the osaf.us environment (ie.  
> derby issues).
>
> Second phase testing would be testing a against an environment that  
> is as close to osaf.us as possible. Reverse proxy, built from  
> source using Jared's scripts etc. This way we flush out all the  
> remaining priority issues.
>
> I would like to get rid of any other legacy services running on  
> qacosmo and write provisioning script that installs a checkpoint  
> build and configures a new mysql database for our "first phase"  
> testing environment. No reverse proxy in this environment. For  
> "second phase" qa.osaf.us environment I'll turn to Jared to help us  
> get it up and the build/provisioning process automated.
>
> I would suggest that we alternate each qa session between these  
> environments to flush out as many issues as possible during the  
> release cycle, but day to day testing should primarily be conducted  
> on qacosmo. This also provides an automated place that we can  
> regress production specific issues.
>
> -Mikeal
>
> _______________________________________________
> cosmo-dev mailing list
> cosmo-dev at lists.osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/cosmo-dev/attachments/20070124/4bdb227f/attachment.html