[Chandler-dev] Proposal to deal with network errors in tests

[Heikki Toivonen]

We have a handful of tests that try real connections over the web,
against services we have no control over. Occasionally these fail.

We have modified some tests to simply ignore some network errors. This
is of course suboptimal, because it makes it hard to find out if there
are real network coding errors in our code.

We run the tests in several different situations (tbox runs the full
suite, individuals run suite and tests individually), so it seems
whatever the solution it must be contained in each individual test.

That leads me to the actual proposal. Wrap each network call in a block
that tries the call a few times (try once, sleep for a minute, try
again, sleep 2 minutes, and repeating until we still fail after 8
minutes of sleep). I think Twisted actually provides something like this
for tests that are based on twisted. If none of the attempts succeeds,
report a failure.

I believe this would get rid of most false errors. There have still been
cases when flickr etc. have been unavailable for several hours, but I
don't think we can do anything about that.

Of course, the best solution would be to not make any network calls. If
you have a test that does go over the network, see if you could change
the test so that it does not need to do that.

-- 
  Heikki Toivonen


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 251 bytes
Desc: OpenPGP digital signature
Url : http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20060913/5d0538d2/signature.pgp