[Dev] are functional tests broken ?

John Anderson john at osafoundation.org
Mon Feb 13 10:52:14 PST 2006


After looking into this a bit more I think that my new TestBlocks 
functional test, when run at the beginning, reveals a real bug -- which 
I will fix. Longterm it would be nice to be able to run all the 
functional tests separately and together as a way of narrowing down such 
failures.

John

Dan Steinicke wrote:
> This morning I changed the way the logger reports pass/ fail to better 
> report failure when any test fails or entire suite fails to complete 
> (commit r9494).  I also moved John's new TestBlocks to be the last 
> test to run, as it seemed to be leaving Chandler in a state that 
> caused problems with many other tests  On windows at least this seems 
> to work as a temporary solution, I am watching tinderbox to see how 
> the other OS's fair.
>
> In the longer term I would like to see the Functional tests be able to 
> be run in any order.
> Dan
>
> Aparna Kadakia wrote:
>
>> I talked to John about this briefly on Friday and it seems that this  
>> error  may be related to the state of the app carried over from the  
>> execution of the previous tests.
>> I am planning to run the same tests with a new repo copied over for  
>> each test to see if we can reproduce the same error.
>>
>> Aparna
>>
>> On Feb 13, 2006, at 7:24 AM, John Anderson wrote:
>>
>>> That's true, however, when I did that the test still fails, just in  
>>> a different way -- it hangs.
>>>
>>> John
>>>
>>> Andi Vajda wrote:
>>>
>>>>
>>>> On Sun, 12 Feb 2006, John Anderson wrote:
>>>>
>>>>> Yes, the functional test are broken.
>>>>>
>>>>> I spent most of Friday looking into this, but didn't find the  
>>>>> bug. It turned out to be very complicated. The test randomly  
>>>>> fails in different ways and some times doesn't fail at all.  
>>>>> However I learned a lot which we're following up on. If you want  
>>>>> to hear all the details, let me know.
>>>>
>>>>
>>>> I looked around in svn and it appears that rev 9464 introduced the  
>>>> current behaviour. If you refresh to rev 9463, the usual 'show'  
>>>> happens and finishes with a claim that 'all tests passed'.
>>>>
>>>> Andi..
>>>
>>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>>
>>> Open Source Applications Foundation "Dev" mailing list
>>> http://lists.osafoundation.org/mailman/listinfo/dev
>>
>>
>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>
>> Open Source Applications Foundation "Dev" mailing list
>> http://lists.osafoundation.org/mailman/listinfo/dev
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "Dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/dev


More information about the Dev mailing list