[Chandler-dev] Sane defaults for new Chandler testing framework

[Mikeal Rogers]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On May 23, 2006, at 4:01 PM, Heikki Toivonen wrote:

> Mikeal Rogers wrote:
>> QA has written an entirely new test framework and new logger for
>> chandler testing. We also have a variety of new features. Some of the
>> new features require your input as to what sane defaults should be  
>> when
>> running chandler tests.
>
> I didn't completely understand how each of the levels and masking etc.
> works, but my preference is to have really minimal output by default.

I'll be writing up documentation on the wiki that explains how the  
new logger works in more detail.

>
> Following Python unittest output I like having just a dot (.) for a
> passed test (or in some cases testname + "ok" on a line). I need to
> think about this a little more...

We need to keep the output a bit more humanly legible than this.

The upcoming test framework (OAF) has visible output decoupled from  
the test output, so you will be able to do customized logging in what  
ever format you wish :)

>
> In case an error occurs, outputting everything I need or is available:
> location of error, why the error happened ("expected x, got y"),
> backtrace, ...

Failures print the name and a comment string when specified, since we  
didn't actually replace QAUITestAppLIb (we just renamed it and  
retrofitted it to use the new logger) we don't have the failure  
output integrated with the tool closely enough to say "got this  
expected this" in every case. When logging a pass/fail a comment  
string can be attached, and we are being good about putting comments  
in for each pass/fail report called.

Tracebacks are handled much better now and we output the traceback to  
both stdout and the logfile as well as logging a failure. This can't  
be turned off so it's in by default no matter what :)


>
> -- 
>   Heikki Toivonen
>
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFEc6KtIy1Ff2Qzo+wRAukxAJ9OG3APXdKbs8Zk4vhsN479SalNfQCcCe7B
MAHUbAJksVc2txkf2JgIGoo=
=xWOT
-----END PGP SIGNATURE-----