[Cosmo-dev] osaf.us VM and qacosmo

[Aparna Kadakia]

Ted,
During the end of the release we will update qa.osaf.us instance  
often enough for there to be meaningful testing done on it. We'll  
still need to figure out what that means and how much effort that is  
on QA and IT's time. We could initially shoot for as often as twice a  
week and see if that is enough and then vary it if it needs to.

Aparna

On Jan 24, 2007, at 5:03 PM, Ted Leung wrote:

> This seems reasonable to me.  I have one question, though.   How  
> often will qa.osaf.us be updated near the end of the release?  If  
> we have next Wednesday's session on qa.osaf.us, will it have the  
> fixes checked in on Tuesday?
>
> Ted
>
> On Jan 24, 2007, at 4:31 PM, Aparna Kadakia wrote:
>
>> Just to summarize it, we would be using qacosmo for doing  
>> functional testing of the daily checkpoint builds. qacosmo gets  
>> wiped off with every checkpoint build and gets rebuilt. This is  
>> where cosmo automated tests will be run against as well.
>>
>> qa.osaf.us wil be our qa staging instance mirroring production  
>> where we run the integration, load and performance tests that  
>> simulate real traffic. This will be cloned from the production box  
>> so we can isolate those issues before we go live. Typically  
>> qa.osaf.us will be tested against at the end of the release when  
>> we are closer to going live. I expect most of the next few qa  
>> sessions to be carried out on qa.osaf.us.
>>
>> +1 to mikeal's proposal
>>
>> On Jan 24, 2007, at 4:18 PM, Mikeal Rogers wrote:
>>
>>> Forwarded from a private thread with Jared's consent.
>>>
>>> On Jan 24, 2007, at 9:28 AM, Jared Rhine wrote:
>>>
>>>> Mikeal Rogers wrote:
>>>>> Instead of provisioning a new machine for qacosmo I was  
>>>>> thinking it might be best to just add another virtual machine  
>>>>> on osaf.us, qa.osaf.us?
>>>>
>>>> That's definitely possible.  Are you thinking the purpose would  
>>>> be for QA on the service, or QA of cosmo in general?  If QA for  
>>>> Cosmo in general, how would the box differ in function from  
>>>> qacosmo?  Are you looking specifically for something with  
>>>> greater horsepower?
>>>>
>>>> I'd like to help identify the issues existing with qacosmo.   
>>>> There's potential for FUD in this area, as well as real problems  
>>>> with qacosmo.
>>>>
>>>> If nothing else, a qa.osaf.us-like thing would help characterize  
>>>> problems that people do see with qacosmo; you can try to  
>>>> replicate such behaviors on another box and see.
>>>>
>>>> The multi-CPU aspect of such a VM can also be used for scaling  
>>>> tests, though I'm guessing that QA is not going to have much  
>>>> ready-to-go before Preview to test actual scaling (as opposed to  
>>>> raw performance, which seems to be progressing nicely.)
>>>>
>>>> If we proceed with a new VM for QA, let's get it on a list.  If  
>>>> it's for the service, it should be asked for on service-dev.  If  
>>>> it's for Cosmo, it should be asked for on cosmo-dev.
>>>>
>>>> Though there's some production traffic on the physical machine  
>>>> currently, after production moves to a new box, the physical  
>>>> admin/VM box will be pretty quiet.  Maybe some late-night  
>>>> metrics-crunching.
>>>>
>>>> -- Jared
>>>>
>>>
>>> What I would like to do is have two seperate environments for two  
>>> different stages on testing.
>>>
>>> First phase testing would be checkpoint and daily testing of  
>>> cosmo, it would also be the server that we write new tests  
>>> against etc. I'd like to avoid issues with this environment that  
>>> would block us from testing and at the same time made sure that  
>>> we aren't just finding low priority bugs that aren't prioritized  
>>> all that highly because they are too different from the osaf.us  
>>> environment (ie. derby issues).
>>>
>>> Second phase testing would be testing a against an environment  
>>> that is as close to osaf.us as possible. Reverse proxy, built  
>>> from source using Jared's scripts etc. This way we flush out all  
>>> the remaining priority issues.
>>>
>>> I would like to get rid of any other legacy services running on  
>>> qacosmo and write provisioning script that installs a checkpoint  
>>> build and configures a new mysql database for our "first phase"  
>>> testing environment. No reverse proxy in this environment. For  
>>> "second phase" qa.osaf.us environment I'll turn to Jared to help  
>>> us get it up and the build/provisioning process automated.
>>>
>>> I would suggest that we alternate each qa session between these  
>>> environments to flush out as many issues as possible during the  
>>> release cycle, but day to day testing should primarily be  
>>> conducted on qacosmo. This also provides an automated place that  
>>> we can regress production specific issues.
>>>
>>> -Mikeal
>>>
>>> _______________________________________________
>>> cosmo-dev mailing list
>>> cosmo-dev at lists.osafoundation.org
>>> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
>>
>> _______________________________________________
>> cosmo-dev mailing list
>> cosmo-dev at lists.osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
>
> _______________________________________________
> cosmo-dev mailing list
> cosmo-dev at lists.osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/cosmo-dev/attachments/20070124/c73a369a/attachment-0001.html