[Dev] Using TwistedMatix as a core package in Chandler

Lou Montulli lou at montulli.org
Sun Mar 23 20:31:45 PST 2003


Michael McLay wrote:

>On Sunday 23 March 2003 07:15 pm, Paul B. Hill wrote:
>  
>
>>>... While I like BEEP in principle, it look like even the author of BEEP
>>>      
>>>
>>has conceded
>>
>>    
>>
>>>that it isn't getting as much traction as he was hoping to achieve. ...
>>>      
>>>
>>May I ask where you got that information? I've heard a couple of other
>>people making similar statements recently, so I asked Marshall Rose about
>>this last week. He seemed surprised. He would also like to know the source
>>of these rumors because he doesn't recall making negative comments about
>>BEEP and still thinks it meets many people's needs.
>>
>>He did say that he that w.r.t. instant messaging systems he did not see a
>>reason to spend further time developing APEX, since Jabber appears to meet
>>all of his current requirements in this application domain.
>>    
>>
>
>It was his comment about APEX and Jabber on the apex mailing list that led me 
>to the conclusion. Perhaps I didn't phrase my comment about BEEP clearly. 
>Jabber is wining against the BEEP based IM protocol because Jabber comes with 
>working code and an established user base. It is an issue of market share. It 
>will be hard for BEEP to displace Jabber, which is the essence of what I got 
>out of Mashall's email. I like the BEEP architecture, and long term I think 
>it has possibilities. It is just off to a slow start and it still requires 
>fairly low level programming. Suggesting to build Chandler on top of BEEP 
>instead of Twisted would be like saying we should build Chandler on top of 
>TCP socket calls instead of using a library that implements transparent 
>object sharing over a half dozen higher level protocols. By moving to Twisted 
>the project moves up a layer.
>
There are already multiple implementations of beep available that work 
quite well.  (The one we are currently using is called RoadRunner)  So 
we don't need to spend alot of time reinventing the wheel.  Second, when 
it comes to our access to data, especially when it comes to 
serialization and deserialization, speed is an important issue.  
Chandler will be a very data intensive application, so access to data 
needs to be as inexpensive as possible.  Thus having a C implementation 
with a Python wrapper is a big win.    Third, BEEP provides protocol 
primitives that just are not available in other protocols, namely 
asyncronous parrallelism over a single tcp connection.  Other things 
like TLS in C and SASL are also easy with BEEP, so it seems to fit our 
requirements very well. 

>
>At the Python conference last year there was a good deal of discussion about 
>BEEP. It looks promising as a technology that could eliminate some of the 
>inefficiencies of TCP. But BEEP is at a lower level in that respect. Twisted 
>and Zope are object management tools that use network protocols to share 
>objects over the network. Chandler is an application that could be built on 
>top of Twisted or Zope, or it could be built on top of BEEP or raw socket 
>calls. Writing Chandler using Twisted or Zope will require less coding than 
>BEEP or sockets.
>
>The conversation at the conference included the TwistedMatrix developers. They 
>were interested in possibly adding BEEP to Twisted. I expect they will 
>eventually include a BEEP interface into Twisted, but in the mean time, I 
>think Chandler might be able to make good use of the Twisted software because 
>it provides a higher level  integration environment for developing 
>distributed applications.
>

It would be very easy for them to add BEEP support to Twisted, but also 
keep in mind that BEEP is just a protocol framework.  It defines no real 
symantics on its own, so TwistedMatrix would have to add that on top.

>
>I also think that BEEP would have a greater chance of advancing if the BEEP 
>core team used Python, instead of Java, to demonstrate the advantages of 
>BEEP. Python is a more agile language and in combination with BEEP it should 
>be possible to demonstrate some amazingly flexible capabilities. The creation 
>of BEEP was intended to reduce the complexity of building protocols by 
>abstracting away common infrastructure. Why didn't they apply the same 
>philosophy to the choice of programming language used in developing the BEEP 
>reference implementation?  Using low level, statically typed languages like C 
>and Java misses an opportunity to move the industry up a notch in language 
>technology.
>

In my experience, a good C lib takes almost no time to wrap using swig.  
So C creates a much more versital, and faster, implementation.  It takes 
more work to create the C lib, but given that the work is already done, 
it's hard to argue that it should be redone in Python.

>
>Also, since BEEP channels can use any encoding format, it would be interesting 
>to see a demonstration of applications sharing objects over BEEP using the  
>TwistedMatrix Perspective Broker encoding for interacting with objects on the 
>net. That could be a killer app for BEEP.
>
>
>  
>
I agree, I think BEEP has not yet found an application that really takes 
advantage of its capabilities.  I think that our use of BEEP as a 
repository access protocol may also prove to be a very compelling BEEP 
usage.

:lou




More information about the Dev mailing list