[Chandler-dev] Comments requested on new test output format

[Mikeal Rogers]

On Jul 12, 2006, at 4:23 PM, Andi Vajda wrote:

>
> On Wed, 12 Jul 2006, Mikeal Rogers wrote:
>
>> This output;
>>> Test Report;
>>> *Suite ""ChandlerTestSuite"" Failed :: Total Time  
>>> ""0:04:55.434000"" ::
>>> Comment ""None""
>>> **Test ""TestSwitchTimezone"" Failed :: Total Time  
>>> ""0:00:00.931000"" ::
>>> Comment ""None
>>> None""
>>> ***Action ""CheckBlockVisibility"" Failed :: Total Time  
>>> ""0:00:00"" ::
>>> Comment ""(On EditTimeZone Visibility)  || detail view = Fa
>>> lse ; expected value = True""
>>> ****Report ""(On EditTimeZone Visibility)  || detail view = False ;
>>> expected value = True"" Failed :: Comment ""None""
>>
>> is only displayed when there is a failure. It's just displayed  
>> before the final summary. It does however "spew" to your screen  
>> every time there are failures calculated at the end of the suite.
>
> How about trimming this:
>   - remove the Total Time with 6 decimals

Excellent idea.

>   - remove Comment None

So 'None' is obviously just the __str__() from a NoneType object. The  
default is None, since comments in report calls are optional. We can  
easily have a check for None, and not display anything in the output  
string, but this will increase the processing time for each output  
line by a little bit.

I'm for it.

>   - remove name of the suite (I know which suite I run)

I'm in favor of removing this temporarily. Right now people only run  
one Suite, but in the future we intend for a user to be able to run  
multiple suites and we wouldn't want to hide this output.

>   - remove ::

We need a separation character for easy parsing of the output, but  
I'm open to using any other characters.

>   - replace *** with spaces

I personally can't tell the different between 2 and 3 spaces "at a  
glance" which is what we're shooting for. I'd be in favor of tabs  
instead of *, but this would increase the line size of some of the  
more nested output and some people really hate that.

>   - replace "" with "

The issue is that some of the test names include " and using the same  
character would screw up any parsing. But I agree we should use a  
different character, triple characters are evil, my bad :)

>   - on the other hand, if there is a stack trace, don't hide it

The stack trace can't be masked, and can't be hidden by the debug  
values, it's unique in this respect. It always prints and is flushed  
right when a failure occurs in python. The test framework then logs a  
new failed report stating that a failure in python occured, it cleans  
up the framework state and moves on to the next test.

No more swallowing tracebacks :)

>
> Andi..