[cosmo-dev] Windmill Test Coverage

[Adam Christian]

I completely agree. In fact the main reason that most of the tests in  
there don't have comments, is because when I created them Windmill  
didn't support comments or whitespace in those files, but after many  
requests those were added and have turned out to be more important  
than I thought.

The Windmill test coverage page is a general overview of the things we  
touch, but is no where near detailed enough to where a dev could use  
it to figure out what test to write next. I think that should be the  
goal.

I think what needs to happen here is:
	1. Some of my cycles need to go into commenting what is already there
	2. I need to document in more depth what we have and what we need

After 0.9 ships I will talk to Aparna and get some time allocated to  
these tasks.

Thanks Bobby for the response.
Adam

On Nov 8, 2007, at 4:57 PM, Bobby Rullo wrote:

> Since no one has commented, edited or added to that page let me  
> explain my motivations for that post a bit more:
>
> Cosmo needs more test coverage, and quickly more than QA can churn  
> out in the time it's needed. So, Cosmo developers are going to add  
> Windmill tests.
>
> My problem is, unless it's a brand new feature, it's really  
> difficult to see what has been tested and what needs to be tested -  
> the only document which has this information is AFAIK is Adam's  
> recent email posting, which is a good start.
>
> I would prefer to see something easier to grab than an email,  
> something at a permanent location that can be updated. This could be  
> a wiki page or even a text file checked into the cosmo source tree.
>
> Also, what Adam's email doesn't do is say exactly what is being  
> validated, or how the tests work, it just mentions the functionality  
> being tested. We have no idea however, the depth of testing of that  
> functionality. The only thing we have to go on then is the tests  
> themselves. Windmill tests don't make for easy reading (no  
> disrespect to Windmill, that's just the nature of the beast), so I  
> think we really need to comment all the tests so we know what they  
> are doing. See for example:
>
> http://svn.osafoundation.org/server/cosmo/trunk/cosmo/src/test/functional/windmill/canvas/event/event_status_enable_disable.json
>
> The one thing I would've done differently in this case is put a  
> comment at the top of the file describing what the file tests.
>
> Another thing that would make it easier for developers to write  
> tests is too have a list of specific tests that should be written -  
> the more specific the better. Entries should look like "Ensure that  
> when moving an event from one week to the next that the data has  
> really moved" or "Ensure that when unstamping eventness, the event  
> is no longer on the canvas". Note that these examples say exactly  
> what should be asserted.
>
> I'm not saying that it's not helpful for someone to point out that  
> "we need more coverage in area XXXX", just that it's not sufficient.  
> In an (my) ideal world, there'd be a list of these tests and devs  
> could just go in and claim them as their own.
>
> This may seem like more work for QA but it's actually less. Here's  
> why: I submit that in the same time it takes Adam (for example) to  
> write a windmill test, he could come up with (at least) ten specfic  
> tests that need to be written. Those tests could then be written be  
> developers (and volunteers! this is a great way for people to get  
> involved, and giving people a list of tasks really lowers the  
> barrier to entry). So in the time that it took him to write that one  
> windmill test he effectively was responsible for ten getting created.
>
> It's also less work for us devs - we spend less time rummaging  
> through the tests figuring out what they do and tearing our hair out  
> over what to do next.
>
> So in summation: if you tell us exactly what to do, and what has  
> been done, it's a lot easier for us to do it and everyone saves  
> time. Also, comments are good. Yay!
>
> bobby
>
>
> On Nov 7, 2007, at 2:37 PM, Bobby Rullo wrote:
>
>> In an effort to get the tests we need written and to avoid  
>> duplication of work, I've created a page which we can use to  
>> remember what areas need Windmill coverage. It's very crude right  
>> now, so feel free to do whatever is necessary to make it more useful.
>>
>> see: http://chandlerproject.org/Developers/WindmillTestCoverage
>>
>> bobby
>> _______________________________________________
>> cosmo-dev mailing list
>> cosmo-dev at lists.osafoundation.org
>> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev
>
> _______________________________________________
> cosmo-dev mailing list
> cosmo-dev at lists.osafoundation.org
> http://lists.osafoundation.org/mailman/listinfo/cosmo-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/cosmo-dev/attachments/20071112/7421440e/attachment.html