[Windmill-dev] js framework "exit on done" work
mikeal at osafoundation.org
Mon Oct 22 19:08:22 PDT 2007
Well I'm glad to hear that we have consensus on getting this feature
in to the jstests.
In terms of getting a similar feature for the service to tell the IDE
when a suite is completed, I don't think it's actually possible.
To backtrack a little, the reason we need the jsTests to tell the
service when they are done is because in some cases the IDE is
launched and the jsTests run headless (nobody is watching, you still
need a GUI and a browser obviously) and when they are completed the
Service needs to kill the IDE and the browser completely, or move on
to the next set of tests. In this case the IDE is a fixture created
by the Service. The reverse case never happens, the Service is never
launched by the IDE and is later destroyed by it.
Although I can see the utility in the IDE knowing when a suite is
done, the Service rarely knows that state. The only time it does is
when test objects are loaded in to the Service for browser debugging
of the tests. It would have to fall on the XMLRPC based test
framework to notify the service of this state change if we were to do
It's very difficult for code based test frameworks to know when
"suites" end (suites are implemented as individual test methods in
most cases) e. I think it's a fairly large burden on new library
developers to have to implement this.
The only time this is easy is when you have markup based tests that
are parsed, if you're executing code line by line it's a pain to pick
up when you move from one set to the next and send yet another method
to the service.
I do think that state changes should be obvious in the IDE, but the
IDE already knows about as much as the Service does about this state.
The IDE knows when the next suite starts, it knows when the service
is out of test objects, that's about all the Service knows.
On Oct 22, 2007, at 6:46 PM, Matthew Eernisse wrote:
> I actually remember asking about this before -- it's a pretty
> standard thing to want to know when an entire suite has finished.
> Of course that would require that we have some kind of a larger
> concept of a 'suite.'
> But it would be nice to be able to call a big blob of tests, and
> then get some kind of summary when the entire thing is done.
> Adam Christian wrote:
>> This sounds fine to me, I was also thinking that this might not be
>> such a bad idea for the non js tests.. that way the IDE can
>> actually display some output like.. "Suite finished running" etc
>> so people don't have to guess on the state.. ideas?
>> On Oct 22, 2007, at 2:50 PM, Mikeal Rogers wrote:
>>> Trying to get the js based tests hooked up I just noticed some
>>> 1) The Service has no idea when the jstests are finished. I can
>>> guess by waiting for the IDE to call next_action() but this is
>>> 2) If we just add a simple "I'm done" method to the Service it
>>> then has no way to differentiate between multiple js test runs.
>>> What I propose is to make some additions to both interfaces.
>>> Currently commands.jsTests only takes one argument, "tests" which
>>> is a list of the js files it's recursively parsed from the given
>>> directory. I'd like it to take another argument, "uuid" which
>>> will be a standard UUID for the test run.
>>> At the end of that test run, the js framework should call a new
>>> method in the Service called "js_test_run_final" with params
>>> "uuid" and "results" (a serialized list of the test results -- i
>>> believe you're already keeping this somewhere).
>>> This way the Service can make multiple test runs and people can
>>> query the server to see the status of any test run. This work can
>>> also be leveraged once we go multi-client with minimal changes.
>>> The Service will be implementing additional methods so that
>>> remote xml/json-rpc clients can start js test runs and get the
>>> results from the Service.
>>> If anyone else would like to add more methods or options to the
>>> js test workflow now would be a good time. Is there anything else
>>> notified of?
>>> Windmill-dev mailing list
>>> Windmill-dev at osafoundation.org
>> Windmill-dev mailing list
>> Windmill-dev at osafoundation.org
> Windmill-dev mailing list
> Windmill-dev at osafoundation.org
More information about the Windmill-dev