[cosmo-dev] Pre-RC automation tests

[Ted Leung]

On Nov 28, 2007, at 2:10 PM, Aparna Kadakia wrote:

>
>>
>> Based on the last few releases - 0.8/0.8.1 and 0.9/0.9.1  it seems  
>> like we haven't yet reached the point where our automation is able  
>> to catch showstopping brown paper bag bugs.    I think that we'd  
>> be well served to have an impromptu collaborative QA session at  
>> some point in the process, either in the final pre-RC stage that  
>> we are discussing now, or for the first RC.
>>
> I believe we have been doing collaborative migration test session  
> or every release that has significant data migration involved. And  
> we should continue doing that. Having said that,  I don't believe  
> those sessions have really helped us catch any of "brown paper bag"  
> bugs (for those wanting to know what is a brown paper bag bug see  
> below ) or atleast not that I recall.
> We don't land up finding most of these crafty bugs up until people  
> start using the app for their daily workflows.
> As for doing the QA sessions, I vote for doing them once an RC is  
> done. Migration testing is sort of open ended and we can't stall  
> creating an RC till all of migration testing is completely.

Doing them once we cut an RC is fine by me.

Ted