[Chandler-dev] Extra check()s in functional tests

Bryan Stearns stearns at osafoundation.org
Fri Jun 30 09:17:10 PDT 2006


Heikki (& Andi),

At lunch yesterday, you (Heikki) mentioned that you'd seen a case where 
a repository problem happened after the functional test suite had been 
run (and reported passed all the tests), but failed while closing the 
repository.

I suggested adding a test at the end that did check() on the repository, 
and you (or someone else?) suggested doing the check after every test 
script.

I modified run_tests (called to loop over all the individual test 
scripts run by FunctionalTestSuite.py) to try this out to see what the 
time impact would be; on linuxdev1, adding one check() at the end added 
two and a half seconds, but adding a check after every script added 
almost two minutes.

I'm sensitive to how long the tests take -- I generally have to run all 
the functional & unit tests three or four times for each checkin, and 
sometimes more -- so I don't want to add this overhead unless it's useful.

Heikki, can you explain to Andi what the problem case was?

Andi, do you think there's any utility to running after each test? 
(Seems like running once at the end would tell us if any test corrupted 
the repository; doing it for each test wouldn't add anything, since once 
it's corrupt, it's likely to stay that way.)

...Bryan


More information about the chandler-dev mailing list