[Dev] Functional test experience report

[Ted Leung]

Hi all.

Last week I discovered that Flickr and some of the other example  
parcels were broken by changes in the detail view.    This week I set  
out to write a Functional test (GUI test really, since there isn't a  
functional spec for Flickr to test against) so that we'd have a  
little more confidence that Flickr would keep on operating in the  
face of changes that we need to make to the app.   I think that it is  
important that we expand our scripting based GUI test coverage.   We  
are building out enough application functionality that it is hard to  
test it exhaustively by hand.

When trying to write my test, I ran into several problems, and I  
thought I'd share my experiences in the hope of saving people some  
time if they try to write a test.

1. At the moment we can't script modal dialogs (there's a bug for  
this <https://bugzilla.osafoundation.org/show_bug.cgi?id=4467>) , and  
Flickr's code for creating a collection of photos was tied to the  
code that popped up the dialog.   So I had to refactor the flickr  
code to separate the dependencies.   The code should have been  
written that way from the beginning anyway.

2. I discovered that much of the functionality of QAUITestAppLib is  
hardwired for the basic Kinds in Chandler, and I had to bypass a good  
deal of the functionality there and go "down to the metal".  I logged  
<https://bugzilla.osafoundation.org/show_bug.cgi?id=4569> for this.

3. I needed to force a repaint/idle in order to have all the  
collection changes noticed by the sidebar and summary table.   Katie  
supplied the code below as a solution.  I believe there is a bug for  
this, but couldn't find it in bugzillla.   I didn't log a bug myself  
because I wasn't sure where processNextIdle ought to live.

     def processNextIdle():
         wx.GetApp().Yield()
         ev = wx.IdleEvent()
         wx.GetApp().ProcessEvent(ev)
         wx.GetApp().Yield()

I hope this will save people some time as they try to write their own  
tests.

Ted