[Dev] Dabo, Wax, others?

[John Anderson]

I agree it makes sense for a end user debug version to send info back to 
us and and a developer debug version wouldn't do that. Perhaps we could 
have a a single debug version with a developer command line flag, or a 
launcher that specifies an end user flag.

In past projects I turned asserts and crashes into automated reports and 
it was amazing how much I learned from those reports. Most of the 
problems that weren't found during "in house" testing were related to OS 
configuration differences between machines. Also, asserts found many 
more problems than crashes -- perhaps because crashes were rarer. It was 
not unusual to add extra debugging code to narrow down rare, non 
reproducable problems, and have the info sent back identify the source 
of the bug.

John

Alec Flett wrote:

> The "talkback" feature that heikki describes is more of an end-user 
> feature. Typically used to allow developers to track crashes of their 
> application out in the wild, it sends a stack trace to a central 
> server on a crash.. (with permission of the user, sometimes even with 
> comments that included the steps required to make the crash happen)
>
> The stack trace gets aggregated with other similar stack traces on the 
> server. Then we, as the developers, can look at the data on this 
> server, and look at a "top crash" report to see what crashes are seen 
> most often in the real world, with real users..
>
> In our case, I think heikki is saying that for end users, if we get an 
> exception at the top level, where developers would normally see a 
> dialog box with all sorts of ugly code, end-users should see a 
> user-friendly error message recommending the user quits & restarts 
> chandler. Someday it would be nice if the stack trace were sent back 
> to OSAF to be combined with others so we could see a "top exception" 
> report and fix such errors.
>
> Obviously, developers want the actual stack trace  of the exception 
> (an assert being just one kind of exception) and the exception would 
> NOT be sent to a central talkback server, since developers are 
> constantly throwing exceptions during day-to-day debugging :)
>
> Alec
>
> John Anderson wrote:
>
>> I like the talkback feature and have used it in the past. However, I 
>> perfer using it in the debug version since it gives you feedback 
>> about asserts as well as crashes. I've found as many rare failures in 
>> asserts as crashes using the talkback technique -- and Python errors 
>> are often ignored instead of causing crashes like C++ programs. So 
>> I'd vote for making the debug version available for customers as an 
>> optional install and providing talkback for all errors.
>>
>> John
>>
>> Heikki Toivonen wrote:
>>
>>>Maybe you meant this message to go to dev? You sent to me only, which is
>>>why I am replying in private. Feel free to forward/reply to dev.
>>>
>>>  
>>>
>>>>My personal bias is to avoid the bug by having good testing rather than
>>>>writing code to report bugs to users, expecially because there are so
>>>>many possible bugs.
>>>>
>>>>I'd rather leave this in the debug version and have some of our "power"
>>>>customers using the debug version.
>>>>    
>>>>
>>>
>>>My point is that we can never find all the bugs in testing. Like you
>>>said, "there are so many possible bugs". For example, suppose a bug
>>>shows up in 1 instance of every 1,000 Chandler installation. We'd be
>>>extremely lucky to have enough "power" customers to uncover that. On the
>>>other hand, if we had 10,000 or more users, the likelihood that someone
>>>will run into it is getting significant and if we make it easy to report
>>>it is a pretty good chance we can act on it.
>>>
>>>Also, silently ignoring exceptions is potentially really, really bad.
>>>(It is likely a program written in some compiled language would have
>>>even crashed. And even in our case some users will run into bugs where
>>>they crash the whole application.) We should give the user some kind of
>>>indication that a problem we didn't anticipate cropped up and give them
>>>a chance to do something about it, rather than hiding it under the covers.
>>>
>>>And suppose we did hide the problem. It is likely that at some point in
>>>the future the users start experiencing weird, very noticeable problems.
>>>But if they report those problems to us, we will have a very difficult
>>>time finding out what is causing them. If we fix those symptoms, the
>>>original bug could still go unfixed.
>>>
>>>Mozilla has had huge improvements in stability from the Talkback crash
>>>reporting tool. I can remember many cases where the developers were
>>>unable to reproduce a bug, but thousands of people were experiencing it.
>>>The only way the bugs were found and verified to be fixed was by using
>>>Talkback. It is an optional install, and it is optional to send the
>>>crash reports.
>>>
>>>  
>>>
>>------------------------------------------------------------------------
>>
>>_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>
>>Open Source Applications Foundation "Dev" mailing list
>>http://lists.osafoundation.org/mailman/listinfo/dev
>>  
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/dev/attachments/20050916/5b691cc3/attachment.htm