[Cosmo-dev] QA Test Scripts

[Mikeal Rogers]

All the non-UI test cases are here;

http://svn.osafoundation.org/server/cosmo/trunk/src/test/functional/ 
protocol/

You need to install py.test, after doing that you just run the  
py.test command line utility and they auto-magically all run.

The UI test cases are being automated with windmill and should be  
checked in this week or next week here;

http://svn.osafoundation.org/server/cosmo/trunk/src/test/functional/ 
windmill/

There shouldn't be anything in the UI that we can't automate,  
although it's doubtful that we would ever have time to create a test  
for every possible verification. If a div slides over a couple pixes  
we can find a way to catch that, but we probably don't have time to  
write automated tests to verify that every single pixel is in alignment.

During the next release we'll be writing up testplans for everything  
that we currently test, and we'll be adding more as new features come  
in.

The ideal process is as follows.

- A feature is specified and design and dev writes some initial docs
- QA writes a test plan
- The feature drops into the code base and QA does verification and  
logs bugs
- QA automates 90% of the feature

-Mikeal

PS. We intend on writing a tool to keep track of test plans and  
results and have a more interactive checkpoint testing process where  
we mark manual tests off of the list. Ideally the automation system  
will interface with this as well to check off the automated tests  
that appear in the test plans. This also would give us an easy way to  
know how much automation we have at any given time. No, I don't have  
a date -- sometime after Preview :)