[Cosmo-dev] Migrating broadsword to py.test

[Ted Leung]

+1

On Oct 18, 2006, at 2:53 PM, Mikeal Rogers wrote:

> I've taken my head up from yelling at the selenium-rc to poke  
> around a bit and do some planning for the continuous test system.
>
> I stumbled upon py.test (http://codespeak.net/py/current/doc/ 
> test.html) a few weeks ago and have been going through it in my  
> spare time. Most of the problems we will have to solve in CTS (test  
> collection, handling resources, modularizing results, etc) are  
> solved or at least addressed in py.test.
>
> For the broadsword tests to integrate in to any continuous system  
> and run reliably with minimal upkeep, they need to be moved to _a_  
> standard framework. The good thing about this is that some of the  
> problems we currently have with the tests would be addressed in a  
> different framework, we just may need to write some of the existing  
> functionality back in.
>
> I've spent the last week or so becoming very familiar with  
> unittest.py (see svn.osafoundation.org/tools/pyselenium/trunk/src/ 
> seleniumunittest.py) and I can't say I'm very happy with it. I have  
> a laundry list of complaints but the largest being that it isn't  
> really modular enough and the different pieces all assume that you  
> aren't overriding anything or changing the default behavior, so  
> modifying one class means that you have to account for the changes  
> in every other component. I've been through every line in the  
> module (it isn't that large) and the only thing I was able to use  
> without almost completely overwriting or starting from scratch was  
> the TextTestRunner. I considered breaking up unittest.py and making  
> it more modular and easier to work with but I can't see any track  
> to getting the fixes committed back to standard lib ( I see no  
> 'unittest' or 'test' mailing list on python.org and  
> pyunit.sourceforge.net hasn't been updated in a couple years ).
>
> Going through py.test the last few weeks has been really nice, I'm  
> not going to call out the entire list of features but for anyone  
> interested you can look through the documentation, it's a huge  
> improvement over unittest.py. The community is also pretty active  
> ( 3 - 6 commits per day since I joined the list ) so I'm confident  
> that any problems we have can actually get fixed.
>
> Moving broadsword to py.test would entail removing TestObject.py,  
> rewriting HTTPTest, DAVTest and JSONTest to using py.test, and  
> removing all the boilerplate code in the tests. I also plan on  
> moving away from the test client being embedded in the subclass  
> along with the framework. All this together means the current cosmo  
> tests would be about half the code they are now and we would get  
> timing, some stress testing framework, multiple resource  
> management, better debugging, and a much simpler plan for the  
> Continuous Test System as it will have to solve far less of our  
> problems. My initial estimate is about a week of dev time to move  
> broadsword to py.test.
>
> Note: After going through py.test I see no reason to write OAF  
> (Open Automation Framework). Every problem we were trying to solve  
> with OAF is solved in py.test, a huge list of features are added,  
> the boilerplate required for tests is reduced, and we have an  
> active community working on the framework already.
>
> -Mikeal
> _______________________________________________
> cosmo-dev mailing list
> cosmo-dev at lists.osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev