[Windmill-dev] Direction of the project and plans for 0.4

[Mikeal Rogers]

On Dec 19, 2007, at December 19, 20071:12 AM, Adam Christian wrote:

> Comments inline
> On Dec 19, 2007, at 12:33 AM, Mikeal Rogers wrote:
>
>> I had a couple big emails that replied to the recent thread but I  
>> decided instead to delete them and start fresh.
>>
>> The project has come a long way, and every time we get over one  
>> hurdle and see some adoption everyone starts clamoring to add new  
>> features and bring in more users and add complexity.
>>
>> Here is the situation we're in.
>>
>> - We have limited contributors with limited time.
> Because currently the people that can use the project are limited to  
> 'tech savvy' people, keeping any buzz from happening large enough to  
> expose us to large amounts of possible committers.
>
>>
>> - We have a couple large holes in our functionality that we all  
>> need plugged.
> These holes are SSL, concurrent browser sessions, and what else?

More supported authoring languages.

>
>>
>> - We have a large list of features that we consider for whatever  
>> reason to be "barriers to adoption".
>> Right now we have a lot of solid functionality, the windmill  
>> controller API is particularly nice and far above what anyone else  
>> currently has in competing products. We can't let that cloud the  
>> fact that we still have a long way to go.
>>
>> We have a series of half done features and a lot of areas that need  
>> re-factoring in order to increase the long term maintainability of  
>> the project. This is what we need to focus on _now_, because it's  
>> something that is much harder to get done the more we add complexity.
> If we are going to do these large re-factors that I have been  
> hearing about since 0.1 we should probably do it now because I think  
> 'future possibilities' of re-factoring is a bad reason not to become  
> user friendly to avg joe.

What i'm talking about are the 4 areas I'm suggesting for 0.4, nothing  
more.

>
>
>> We're seeing some great users coming by, yes they are "tech-savy"  
>> but to be frank, we can't feasibly support people who aren't right  
>> now. These are the kinds of users we should have at this stage, and  
>> should try to appeal to more instead of going after people with  
>> less experience that will require more support.
> This sounds terribly backwards to me, selenium found a large contrib  
> base after they had a firefox plugin so that everyone and their  
> brother could write tests. Maybe these tests were just to be opened  
> and played back over and over, but they were the gateway drug making  
> users want more flexibility, and those are the people that turned  
> into contributors.
>
> Our ability to 'support' all these users will still be an issue if  
> we have a large influx of 'tech savy' users. After we remove the  
> barriers to entrance get get users, they can help each other.  
> Without this happening we simply don't have the time to support more  
> than 5 of the most 'tech savvy' users writing to the list about  
> issues.

Selenium gets out of this problem by simply not supporting their  
users, we know this more than anyone.

>
>
>> In all honestly, this project can't sustain the work it will take  
>> to support a large "non-tech-savy" user base. We need to focus on  
>> appealing to more potential contributors, in other words "tech- 
>> savy" people, to make the projects sustainable. For this purpose  
>> it's important to focus on the Python and JavaScript related  
>> features, as the JSON format is targeted at non-developers.
> Again, we can't support any user base on our own really, where are  
> these contributors magically going to come from if this project  
> doesn't start to get out there?

We're seeing a steady and healthy increase in usage. And we seem to be  
retaining most of that usage. I'm not talking about any magic, I'm  
talking about sustaining the growth we currently see and not loading  
ourselves up with more than we can do.

>
>
> I disagree, and feel that the few things keeping us from being a  
> binary download and two button clicks away from recording tests as  
> 'avg joe' is just as important if not more for the community aspect.

Again, I just don't know how we would support this person right now.

Every user I can think of in recent memory found a bug that we fixed  
or a problem with the documentation. After our last major release we  
shipped 4 immediate bugfix releases the proceeding week. What would  
have happened if thousands of "average joe's" were using windmill when  
we shipped that? What if they were all downloading 70meg binaries? I'm  
not even going to go in to the issue of actually building and  
distributing them, updating relevant package managements systems, etc.

>
>
>>
>> Here is what I would like to see for 0.4;
>>
>> 1. Completing JavaScript test authoring.
>> 	We cannot defer this work. Currently there are zero unit tests  
>> checked in for this, there is zero continuous integration support,  
>> and zero XMLRPC support for this framework. This work has to happen  
>> now, if we're going to have 3 supported authoring paths then they  
>> need to be in and continually supported. I would like Adam to take  
>> on portions of this from mde if at all possible.
>> 2. wxWindmill Alpha for Windows.
>> 	There are several caveats. Windows ONLY, we will not produce  
>> binaries for platforms other than windows. We will create the  
>> infrastructure to host and create new builds but we won't be  
>> pushing new binary bits for every bugfix release. And most  
>> importantly, wxWindmill will be moved out of the primary windmill  
>> module in to, more or less, it's own project, in the windmill trunk.
>
> I disagree, I think anybody with a Mac or Linux box who wants to try  
> out windmill with the 10 minutes they have before they catch the  
> train should be able to pull down a binary, fire it up and off they  
> go. I can see a lot of tech savvy people doing this and deciding  
> that it's pretty cool and that there is something they would like to  
> add, so they start digging into the source.

We shipped 4 bugfix releases last week.

We have a hard time keeping up with updating the code we already have  
and pushing it out. How do you plan on supporting 3 platform builds  
every time we need one of these releases? Not to mention you're  
opening a huge can of worms and support on all platforms at once.

Also remember that I didn't write any of the wx code, I'm not the  
primary owner of it, and have no idea how to fix bugs in it. Our  
domain owner for this stuff, Jacob, is about to take off to Australia  
until the summer so I consider it an aggressive goal just to support a  
windows build. If we don't have an active owner there is no way we can  
even begin to consider what you're asking.

I'm willing to put work in to Windows because right now installing  
windmill on Windows is even hard for me. There is no question it's a  
barrier to adoption even by very tech savy users.

We also need to stagger the impact of supporting pre-built binaries  
over a longer period of time. I hope the next release will be done in  
January and that's just not enough time to figure out how we're going  
to do all of this.


>>
>> 3. Improved XMLRPC Support.
>> 	The largest barrier for people creating new language libraries is  
>> the XMLRPC support. It's also made features in windmill hard to  
>> develop as all the tests have moved to being XMLRPC based.
> I agree, this is another important piece of the community puzzle.
>
>>
>> 4. IDE UI re-arch.
>> 	I've heard Adam grumble about this saying it's in need of some  
>> refactoring. We're adding new stuff all the time and if the  
>> architecture becomes a bottle neck this will be very problematic.
> This code is ugly, however every-time we run into something I think  
> is going to be an impossible feat with this code the way it is it  
> turns out that it isn't. There is one big function here that returns  
> a DOM element representing an action and I have been slowly cleaning  
> it up as we go.
>
>> Notice that, other than XMLRPC, none of the external API's are  
>> changing. There are no new features in this list, just solidifying  
>> things we're had for a while. There are no features directly  
>> pointed at end-user adoption. The wxWindmill work is also targetted  
>> at windmill supporting itself more like an importable python  
>> module, with greater facilities for re-use to encourage adoption in  
>> other projects/frameworks.
> If we want to make a list of things that need to be done before we  
> drop the big "become available to the world" bomb, then I think we  
> should write it out, do it, then do the very reasonable workload it  
> will take to make the project available for the masses.
>
> XMLRPC, Binaries, run a folder of JSON tests. I can live with 'sorry  
> about SSL' for now.
>
> There is some Chandler Desktop history that we really need to learn  
> from here. Mass of users is important to an open source project even  
> if it isn't where you want it 'yet', continually tweaking and 're- 
> factoring' can lead to 5 years without really putting the deserved  
> energy into exposure, and probably most importantly -- engineers are  
> never 'happy enough' to pull the trigger. As long as API's aren't  
> changing, we can refactor under the hood as many times as we want..  
> and maybe if we have 100 committers we won't have to do all that  
> work ourselves.

Dude, we aren't Microsoft, we don't have big "available to the world"  
bombs.

We are seeing a steady increase in adoption and we are supporting  
users as they come along very nicely. We're seeing good steady  
improvements in the product in terms of features and stability.

We don't have the time or resources to do more than we already are  
until we get more contributors. And i think this will happen  
naturally, there is no doubt in my mind.

-Mikeal