[Cosmo-dev] Collaborative QA for Preview

[Aparna Kadakia]

Preview release, as we all know, is a big milestone release for OSAF.  
Obviously, our quality standards are going to be very high for this  
release. As a 4 member QA team, divided between Cosmo and Chandler,  
we are grossly understaffed for doing detailed, exhaustive testing of  
all the new features. Without going on a mass hiring spree, we would  
like to brainstorm on ideas about how we could collectively as a team  
commit ourselves to raising the quality standards for this release.

Some of the things we think would help are:
1. More automation
Add more automated tests to the test suite. This would especially  
help in regression testing of existing features by reducing manual  
cycles. This also includes adding unit tests for all the new features  
being checked in. So when we hit Feature Complete deadline, it would  
mean all the unit tests for the specific feature are also done. These  
would be run continously on tinderboxes and thereby help catch any  
regressions.

2. More pariticipation in the IRC QA sessions
As you all know we now have a dedicated IRC office hour for QA  
sessions on wednesdays at 11:00 AM PST. Traditionally only the QA  
team, PPD team and the dev managers have been participating in these  
sessions. What we would like is to see is more participation from  
other team members, especially in cross-product and cross-feature  
testing. Hopefully this is not asking for too much. Just like we make  
time for staff meetings, design sessions etc, we should make time for  
this in our schedule and commit ourselves to exercising the app in  
different ways and testing new features to help uncover bugs.

3. More internal dogfooding
If we could just get people internally to download Chandler  
periodically and use it for maintaining their personal calendars as  
well as updating the office calendar that would be a big help.

The other suggestions we have heard are:
1. Maybe increasing the frequency of the IRC QA sessions from once a  
week to twice a week with partipation from all prouct teams.
2. All the teams dedicate 1 day a week to testing after feature freeze
3. Hiring a contractor till preview ( I am not sure how much value  
this would add)
We would like to hear more suggestions if you have any that have  
worked in your previous companies.

A few things to remember that the bugs in the software will turn up  
eventually and we will need to fix all of them before the release. So  
it would be most efficient on everyone's time in finding them earlier  
than later. Also when you cross test features, you bring fresh  
perspectives on usability of the feature, uncovering any design bugs  
earlier on, if any. This would be valuable user feedback for the PPD  
team as well.

At the end of the day we need to release quality software and if that  
means we need to get more creative and figure out ways in which we  
can leverage the strength of the team for getting there I sure hope  
we can do that.

Comments, suggestions are most welcome..
Aparna