[Chandler-dev] Making Tbox logs more readable

[John Anderson]

I'd like to see a log that make it obvious what the error was and which 
test caused it.

If the current tinderbox mechanism can't do that -- because of bugs, 
limitations, features, etc. as described below -- perhaps we need to 
think about how to change the way tinderbox works so that it can 
correctly report errors and make them obvious.

John

Heikki Toivonen wrote:
> The issue of Tinderbox log readability has been raised (once again). The
> purpose of this email is to start a discussion to see if we can come to
> a consensus on a suitable format. (We are talking about the full Tbox log.)
>
> To start with, let me describe briefly how the current system works.
>
> A Tinderbox clients pulls chandler source code, do build if code
> changed, then run various tests. All of this activity is logged, and of
> course Chandler itself and the test frameworks provide additional log
> files. At the end of the run, the client sends an email to the Tinderbox
> server with all of the logs concatenated to a single file.
>
> In some cases we can miss entire logs from certain tests if we
> encountered a Python crash. For example, functional tests will not
> output anything until they finish, and if Python crashes before we
> finish there will be no output. The only indication in the log file is
> that there is a line that says something like 'starting functional
> tests' and the next line says exit code=<some non-zero number).
>
> We can also experience a situation where test logs were output properly
> and all tests passed, but then we experience a Python shutdown crash. We
> catch that in some cases (but not all), and this will show up in the
> Tbox log as something like 'all tests passed' followed by line that says
> 'tests failed' (the first one was output by the test framework, the
> second one by the script that started the testsuite and monitored python
> exit value).
>
> The Tinderbox server creates an HTML page from the log. The page
> contains the full log, among with some additional information. At the
> top of the page is a section of links to errors. That is produced by
> parsing the full log, and creating links for each line were certain
> keywords appear, like 'error', 'warning' and 'failed'. This is of course
> not foolproof, and can result in false positive links. The longer the
> log, the more false positive error links there are the top.
>
> Currently the logs are very long and verbose.
>
>
> Now... What should we have in the logs in your opinion, and how should
> they be formatted?
>
>   
> ------------------------------------------------------------------------
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20060914/8a8b069d/attachment.html