mikeal at osafoundation.org
Fri Jun 8 01:15:42 PDT 2007
> I thought the current test framework already included this
> information within FunctionalTestSuite. This single file method
> can easily be extended to include recorded tests and would also
> have the benefit of being a single place to run both Functional and
> recorded tests. After all, are not recorded tests just another
> functional test?
It's always been my impression that we will be replacing the "old"
functional tests with recorded tests. Running them from the same
framework wasn't really considered and isn't how they run now. There
were many bugs/feature requests for the old framework which we solved
in the new one and as far as I know we've always intended on phasing
out the old functional tests and replacing them with recorded scripts.
> Currently we are running tests by filename - is there a plan to
> have it run some other way?
To clarify, the _framework_ only thinks about tests in terms of test
modules. You can reference them by filename but it's all the same to
> If we are no longer running recorded tests by filename then how
> will they be run by tinderbox? How will developers use the
> existing --recordedTest command line parameter to try and reproduce
> a test failure?
The --recordedTest command line runs execute_frame, which uses a
prebuilt dictionary of tests that are collected as modules using
get_test_modules. Sure, the command line references them by filename
but it's really just treated as a dictionary key, the value of which
is a test module.
Thanks to Jacob, we have a new dynamic menu under the Tools ->
scripting menu which lists all available tests ( and doesn't observer
platform exclusions by design ) in which you can run each test inside
of chandler and we think this will end being used by developers more
often for debugging than the --recordedTest command line parameter.
> I'm guessing that I'm missing information as to why recorded tests
> cannot use the same framework that the current functional tests use
> (with appropriate changes for the setup/teardown needs).
The recorded tests have never used the old framework. At this point
it would be better to run the old tests in the new framework than the
other way around as we've taken some feature requests and worked them
in to the new framework. But like I said before, it's always been my
impression that we are phasing out the old tests as the recorded
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20070608/bdc5f264/PGP.pgp
More information about the chandler-dev