[Chandler-dev] Comments requested on new test output format
mikeal at osafoundation.org
Wed Jul 12 17:05:11 PDT 2006
On Jul 12, 2006, at 4:23 PM, Andi Vajda wrote:
> On Wed, 12 Jul 2006, Mikeal Rogers wrote:
>> This output;
>>> Test Report;
>>> *Suite ""ChandlerTestSuite"" Failed :: Total Time
>>> ""0:04:55.434000"" ::
>>> Comment ""None""
>>> **Test ""TestSwitchTimezone"" Failed :: Total Time
>>> ""0:00:00.931000"" ::
>>> Comment ""None
>>> ***Action ""CheckBlockVisibility"" Failed :: Total Time
>>> ""0:00:00"" ::
>>> Comment ""(On EditTimeZone Visibility) || detail view = Fa
>>> lse ; expected value = True""
>>> ****Report ""(On EditTimeZone Visibility) || detail view = False ;
>>> expected value = True"" Failed :: Comment ""None""
>> is only displayed when there is a failure. It's just displayed
>> before the final summary. It does however "spew" to your screen
>> every time there are failures calculated at the end of the suite.
> How about trimming this:
> - remove the Total Time with 6 decimals
> - remove Comment None
So 'None' is obviously just the __str__() from a NoneType object. The
default is None, since comments in report calls are optional. We can
easily have a check for None, and not display anything in the output
string, but this will increase the processing time for each output
line by a little bit.
I'm for it.
> - remove name of the suite (I know which suite I run)
I'm in favor of removing this temporarily. Right now people only run
one Suite, but in the future we intend for a user to be able to run
multiple suites and we wouldn't want to hide this output.
> - remove ::
We need a separation character for easy parsing of the output, but
I'm open to using any other characters.
> - replace *** with spaces
I personally can't tell the different between 2 and 3 spaces "at a
glance" which is what we're shooting for. I'd be in favor of tabs
instead of *, but this would increase the line size of some of the
more nested output and some people really hate that.
> - replace "" with "
The issue is that some of the test names include " and using the same
character would screw up any parsing. But I agree we should use a
different character, triple characters are evil, my bad :)
> - on the other hand, if there is a stack trace, don't hide it
The stack trace can't be masked, and can't be hidden by the debug
values, it's unique in this respect. It always prints and is flushed
right when a failure occurs in python. The test framework then logs a
new failed report stating that a failure in python occured, it cleans
up the framework state and moves on to the next test.
No more swallowing tracebacks :)
More information about the chandler-dev