[Windmill-dev] New test collection and dependency strategy
mikeal at osafoundation.org
Mon Dec 3 16:44:17 PST 2007
Those of you who watch the trunk commits have probably noticed the new
collector and dependency code I've been committing.
The largest change is to the way we collect and run json tests. I
wrote a lot of code in functest to allow different consumers to define
their own additions to the test collection mechanisms in functest.
Using that framework I wrote a windmill collector for .json test files.
This means that you won't need to define a test_conf.py with a static
list of test ordering (which is horribly unscalable as we've realized
on cosmo). Instead you make sure that all your directories being with
"test_" and are valid python modules ( they must include a __init__.py
file). This may seem like more work than previously, but this also
means that you can leverage functests setup/teardown dependency system
which uses module heirarchies of setup/teardown for dependency
I've started working on a porting the cosmo tests to this new framework
It's pretty ugly right now since I have copies of all the old tests in
there, but you can start to get an idea of how things work.
At this point I'm ready to integrate the new functest framework run
points in to windmill. This means, among other things, changing the
meaning of certain windmill commands. The old run_json and load_json
commands will mean something a bit different and link to the new way
of running tests in the framework. It also means that keyword
arguments given to the windmill binary will need to be added to
functest.registry to avoid incompatibilities between test repositories
written for windmill/functest.
Another big change is that json tests run like any other test library,
they block and wait for the result of each action unless they are
passed to the service via add_command/add_test. With functest, if you
pass browser_debugging=True to the command line json/python tests will
be loaded rather than block and wait for a return. The new run_ and
load_ commands in windmill will work this way as well.
More information about the Windmill-dev