[Chandler-dev] CATS3

[Dan Steinicke]

I like the idea of having one way to get a list of tests, that all the 
different methods of running the tests can use. 

I also like not having the tests not depend on each other.  

What I don't understand is why you want to use a attribute in the test 
to cause it to be excluded when a simple text file list of excluded 
tests would seem to me to be:
easier to code
easier to discover and use
not have the problems with the test not importing outside of Chandler python

-Dan

Mikeal Rogers wrote:
> We've been running recorded scripts as tests for some time now and 
> recently started solidifying the way the framework works and got them 
> running on tinderbox. What we have is now being documented, although 
> it's still a work in 
> progress, http://chandlerproject.org/Projects/ChandlerAutomatedTestSystemThree 
> .
>
> There are two big differences between the old cats framework and the 
> new one. The first is that the tests don't, and in fact _must not_ 
> depend on each other. The second is a much more contentious issue, 
> which is that the framework uses metadata in the test modules to tell 
> the framework how it should run.
>
> When temporarily excluding tests from a single or all platforms you 
> add a special attribute to the module. Declaring common dependencies 
> (like account settings) is done in much the same way.
>
> The first problems we ran in to were due to the fact that the test 
> scripts wouldn't import outside of chandler's python which meant that 
> the tests had to be collected differently in rt.py . I have a patch 
> in bug 9450 <https://bugzilla.osafoundation.org/show_bug.cgi?id=9450> 
> to help fix this, now the framework and the test modules can be 
> imported in rt.py's python and don't depend on any chandler code until 
> you execute the actual framework or the test methods.
>
> We now have 3 ways that the tests are collected and executed; from a 
> menu inside the chandler Tools menu, automatically from the chandler 
> command line, or from rt.py. My patch should allow us to get all of 
> these paths using the same framework and clean up the problems we've 
> had with all the different execution methods in the past.
>
> -Mikeal
> ------------------------------------------------------------------------
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev
>