[cosmo-dev] Windmill Woes
br at osafoundation.org
Fri Dec 14 16:30:58 PST 2007
After the update to the latest Windmill (3.x) with all the new
dependency stuff, I find myself back at the point where Windmill is
more or less unusable for me. I ran into a number of problems:
To create my test, I needed to know what state Windmill is in during
the running of that test. So I was told that the best way to do this
was to just run the test, which will run all its dependencies.
So I do load_test() in the shell and give it a shot. It fails because
I'm on the wrong page ('/chandler' instead of '/')
Then I go to the proper starting URL and try again. It loads (takes a
long time) and I hit run. It's taking a while so I get up to go to the
bathroom. When I get back, the it's done! Yay! Oh wait, it's all tore
down. Gotta start again damn.
So I clear the IDE and hit load_test(). This time load_test() behaves
differently - only the test itself is loaded, not the dependencies.
Damn. So I quit windmill and do it again.
This time I make sure to watch very carefully so I can hit "Stop" and
observe the state of the app after setup. Well, the test fails. Damn.
At this point, I've given up. There's actually more iterations of
these sorts of things, but I didn't log every single thing - it was
frustrating enough as it is.
Luckily, though the usability problems right now are IMO major, the
fixes are relatively simple:
1) Documentation: If there was documentation in the __init__.py
telling test writers what state the app will be in after setup() this
whole thing would've been avoided.
2) Don't be clever (or, do what I say not what you think I want):
The load_test() method loads a test into the IDE with all the
necessary setup and teardown actions. Well, that's how it behaves the
FIRST time you call it. Subsequent invocations only load the test
itself, forcing me to restart windmill. Grrrrrr. Windmill is trying to
be clever here, it's assuming that because I loaded the dependencies
once, they must've ran and I don't need them anymore. Except that it's
not clever enough - the dependencies failed, so I need to try again.
Also, what if i didn't want to load the dependencies at all? What if I
just wanted to load the setup? Or the teardown? There should be
explicit methods to do each of these things that work the same way
3) Actions take too long to load.
4) When there are several suites loaded into the IDE, give me the
option to run one at a time.
5) Make windmill start at the correct url
As for the tests not running after I got past all the hurdles, I'm not
sure what the problem is.
In any event, I just want to acknowledge that the Windmill team has
been extremely responsive to the needs of the Cosmo team, and that any
usability bugs introduced were only because of an attempt to make
things better for us. So, thanks you guys!
More information about the cosmo-dev