[Chandler-dev] Comments requested on new test output format

Dan Steinicke dan at osafoundation.org
Wed Jul 12 10:33:44 PDT 2006


We are getting close to being able to switch over to the new test 
framework (CATS-0.2).  As it is currently implemented the output from 
the tests will be quite different from the current test output. 

Please take a moment to look over the test output samples below and let 
us know what you think so we can address peoples concerns before the new 
framework goes live, rather than after.

Two sample outputs are shown below, one where a test fails without a 
traceback, another of a failure with a traceback.

For more info on the new framework see :
general docs 
http://wiki.osafoundation.org/bin/view/Projects/ChandlerAutomatedTestSystemZeroPointTwo 

writing new tests 
http://wiki.osafoundation.org/bin/view/Projects/WritingChandlerAutomatedTestsWithCATSZeroPointTwo 


Thanks
Dan

############# Here is sample output from do_tests where one tests fails 
without a traceback:

Test Report;
*Suite ""ChandlerTestSuite"" Failed :: Total Time ""0:04:55.434000"" :: 
Comment ""None""
**Test ""TestSwitchTimezone"" Failed :: Total Time ""0:00:00.931000"" :: 
Comment ""None
None""
***Action ""CheckBlockVisibility"" Failed :: Total Time ""0:00:00"" :: 
Comment ""(On EditTimeZone Visibility)  || detail view = Fa
lse ; expected value = True""
****Report ""(On EditTimeZone Visibility)  || detail view = False ; 
expected value = True"" Failed :: Comment ""None""
***Action ""CheckEditableBlock"" Failed :: Total Time ""0:00:00"" :: 
Comment ""(On EditTimeZone Checking)  || detail view value =
Floating ; expected value = US/Pacific""
****Report ""(On EditTimeZone Checking)  || detail view value = Floating 
; expected value = US/Pacific"" Failed :: Comment ""None"
"
***Action ""CheckBlockVisibility"" Failed :: Total Time ""0:00:00"" :: 
Comment ""(On EditTimeZone Visibility)  || detail view = Fa
lse ; expected value = True""
****Report ""(On EditTimeZone Visibility)  || detail view = False ; 
expected value = True"" Failed :: Comment ""None""
***Action ""CheckEditableBlock"" Failed :: Total Time ""0:00:00"" :: 
Comment ""(On EditTimeZone Checking)  || detail view value =
Floating ; expected value = US/Pacific""
****Report ""(On EditTimeZone Checking)  || detail view value = Floating 
; expected value = US/Pacific"" Failed :: Comment ""None"
"
$Suites run=1, pass=0, fail=1 :: Tests run=26, pass=25, fail=1 :: 
Actions run=367, pass=363, fail=4 :: Reports run=556, pass=552,
fail=4
#TINDERBOX# Testname = ChandlerTestSuite
#TINDERBOX# Time elapsed = 0:04:55.434000 (seconds)
#TINDERBOX# Status = FAILED
- + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
The following tests failed
   
(debug)C:\cygwin\home\Dan\chandler\tools\cats\Functional\FunctionalTestSuite.py 

   
(release)C:\cygwin\home\Dan\chandler\tools\cats\Functional\FunctionalTestSuite.py 



########## here is some sample output of a test failing with a traceback


Test Report;
*Suite ""ChandlerTestSuite"" Failed :: Total Time ""0:00:06.139000"" :: 
Comment ""None""
**Test ""TestCauseTrace"" Failed :: Total Time ""0:00:00.040000"" :: 
Comment ""None
Test Failure due to traceback
Traceback (most recent call last):
 File "C:\cygwin\home\Dan\chandler\tools\cats\framework\runTests.py", 
line 51, in run_tests
   test.runTest()
 File 
"C:\cygwin\home\Dan\chandler\tools\cats\framework\ChandlerTestCase.py", 
line 68, in runTest
   self.startTest()
 File 
"C:\cygwin\home\Dan\chandler\tools\cats\Functional\TestCauseTrace.py", 
line 24, in startTest
   1/0
ZeroDivisionError: integer division or modulo by zero
""
***Action ""Divide by zero "" Failed :: Total Time ""0:00:00.040000"" :: 
Comment ""None""
****Report ""Action Failure due to traceback"" Failed :: Comment ""None""
$Suites run=1, pass=0, fail=1 :: Tests run=3, pass=2, fail=1 :: Actions 
run=5, pass=4, fail=1 :: Reports run=28, pass=27, fail=1
#TINDERBOX# Testname = ChandlerTestSuite
#TINDERBOX# Time elapsed = 0:00:06.139000 (seconds)
#TINDERBOX# Status = FAILED



More information about the chandler-dev mailing list