[Cosmo] Re: [Dev] Proposal for new automation architecture

[Mikeal Rogers]

The current plan is to use silmut which is the tool that Heikki wrote  
in Python for automated functional tests of Cosmo.

None of our unittest systems and functional test systems are actually  
integrated in any way except scripts that bear has written to run  
them and parse the results.

Ideally, what I would like to do is write the framework in such a way  
that test cases written for the framework could be executed by a a  
python unittest system. Phillip has given me some good feedback on  
how this can be done, it was his idea. Although the current plan is  
to still have the framework run on it's own and not depend on any  
python unittest framework. But that is more for Chandler than Cosmo.

I don't see why the unittests and the client automated tests would  
have to be in the same language to run together. The framework is  
designed in such a way that the output is highly customizable and  
could be integrated in to any kind of results tracking repository  
that a unittest system was keeping it's results in.

The running of the tests written for this framework can also be  
executed in a variety of ways, and eventually even be distributed to  
multiple machines managed by a single host framework (definitely a  
version 2 kind of feature), and maybe even manage two resources  
simultaneously (cosmo server and chandler client for example, version  
2 feature as well). The point is that the framework is modular and is  
designed to work in a variety of future scenarios.

Also, the TestCase.action() and TestCase.validate() methods take in  
any python method and expect arelatively simple return. Therefore ANY  
test that can be wrapped in a python method can be executed using  
this framework. It could be a shell script, or some Jython code,  
anything that can be wrapped in a python method.

So we can decide later what level of integration and what approach we  
want to take, the framework as it is currently designed can support  
nearly any approach and shouldn't be a hinderance.

-Mikeal

On Feb 10, 2006, at 12:06 PM, Lisa Dusseault wrote:

> We're not yet committed to standardizing on *any* single test  
> framework for Cosmo, OAF or other.  In fact we'll have more than  
> one -- we are beginning to run Litmus (granted, not exactly a test  
> framework) automatedly, we have junit tests which aren't going  
> away, and we are considering what to focus on for automated client  
> requests.
>
> Have you considered the Python doctest-based client automated tests  
> that Heikki proposed and the Slide java/XML data-file oriented  
> approach that Grant posted on?   Comments?
>
> If the actual *unit* tests remain in Java, can the client automated  
> tests (the protocol request suites) be in Python without harm?
>
> Lisa
>
> On Feb 10, 2006, at 11:56 AM, Kervin L. Pierre wrote:
>
>> Hello Group,
>>
>> I am going to share my initial reaction to this
>> proposal though I understand some may resent
>> someone outside the development team coming
>> across as critical...
>>
>> Philippe Bossut wrote:
>>> (http://wiki.osafoundation.org/bin/view/Projects/ 
>>> OpenAutomationFramework) first. Keep in mind also that Mikeal is  
>>> trying to solve a problem that includes Chandler and Cosmo.
>>
>> Unifying the testing procedure and tools sound like
>> a good idea but this will introduce Python as a
>> dependency for testing Cosmo.
>>
>> a. Python is a large dependency simply for tests
>>    when Java has no shortage of testing frameworks.
>>
>> b. Someone working on cosmo may have no python
>>    experience at all and we may want to encourage
>>    developers to add their own tests as they go along.
>>
>> c. Internally Cosmo and Chandler have very little
>>    in common and so very few tests are going to
>>    be able to be shared.
>>
>> Best Regards,
>> Kervin
>>
>> _______________________________________________
>> Cosmo mailing list
>> Cosmo at osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/cosmo
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "Dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/dev