[Chandler-dev] Sane defaults for new Chandler testing framework
mikeal at osafoundation.org
Tue May 23 17:02:53 PDT 2006
-----BEGIN PGP SIGNED MESSAGE-----
On May 23, 2006, at 4:01 PM, Heikki Toivonen wrote:
> Mikeal Rogers wrote:
>> QA has written an entirely new test framework and new logger for
>> chandler testing. We also have a variety of new features. Some of the
>> new features require your input as to what sane defaults should be
>> running chandler tests.
> I didn't completely understand how each of the levels and masking etc.
> works, but my preference is to have really minimal output by default.
I'll be writing up documentation on the wiki that explains how the
new logger works in more detail.
> Following Python unittest output I like having just a dot (.) for a
> passed test (or in some cases testname + "ok" on a line). I need to
> think about this a little more...
We need to keep the output a bit more humanly legible than this.
The upcoming test framework (OAF) has visible output decoupled from
the test output, so you will be able to do customized logging in what
ever format you wish :)
> In case an error occurs, outputting everything I need or is available:
> location of error, why the error happened ("expected x, got y"),
> backtrace, ...
Failures print the name and a comment string when specified, since we
didn't actually replace QAUITestAppLIb (we just renamed it and
retrofitted it to use the new logger) we don't have the failure
output integrated with the tool closely enough to say "got this
expected this" in every case. When logging a pass/fail a comment
string can be attached, and we are being good about putting comments
in for each pass/fail report called.
Tracebacks are handled much better now and we output the traceback to
both stdout and the logfile as well as logging a failure. This can't
be turned off so it's in by default no matter what :)
> Heikki Toivonen
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> Open Source Applications Foundation "chandler-dev" mailing list
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)
-----END PGP SIGNATURE-----
More information about the chandler-dev