[Chandler-dev] CATS3

[Mikeal Rogers]

> I thought the current test framework already included this  
> information within FunctionalTestSuite.  This single file method  
> can easily be extended to include recorded tests and would also  
> have the benefit of being a single place to run both Functional and  
> recorded tests.  After all, are not recorded tests just another  
> functional test?

It's always been my impression that we will be replacing the "old"  
functional tests with recorded tests. Running them from the same  
framework wasn't really considered and isn't how they run now. There  
were many bugs/feature requests for the old framework which we solved  
in the new one and as far as I know we've always intended on phasing  
out the old functional tests and replacing them with recorded scripts.

>
> Currently we are running tests by filename - is there a plan to  
> have it run some other way?

To clarify, the _framework_ only thinks about tests in terms of test  
modules. You can reference them by filename but it's all the same to  
the framework.

>
> If we are no longer running recorded tests by filename then how  
> will they be run by tinderbox?  How will developers use the  
> existing --recordedTest command line parameter to try and reproduce  
> a test failure?

The --recordedTest command line runs execute_frame, which uses a  
prebuilt dictionary of tests that are collected as modules using  
get_test_modules. Sure, the command line references them by filename  
but it's really just treated as a dictionary key, the value of which  
is a test module.

Thanks to Jacob, we have a new dynamic menu under the Tools ->  
scripting menu which lists all available tests ( and doesn't observer  
platform exclusions by design ) in which you can run each test inside  
of chandler and we think this will end being used by developers more  
often for debugging than the --recordedTest command line parameter.

>
> I'm guessing that I'm missing information as to why recorded tests  
> cannot use the same framework that the current functional tests use  
> (with appropriate changes for the setup/teardown needs).

The recorded tests have never used the old framework. At this point  
it would be better to run the old tests in the new framework than the  
other way around as we've taken some feature requests and worked them  
in to the new framework. But like I said before, it's always been my  
impression that we are phasing out the old tests as the recorded  
scripts stabilize.

-Mikeal

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20070608/bdc5f264/PGP.pgp