[Dev] Services, CPIA, Content Model intercommunication

Tue Nov 9 10:49:28 PST 2004

Comments inline

Brian Kirsch - Email Framework Engineer
Open Source Applications Foundation
543 Howard St. 5th Floor 
San Francisco, CA 94105 
(415) 946-3056 

On Nov 8, 2004, at 1:23 PM, Ted Leung wrote:

> Brian,
>
> One other issue that I just thought of is that we'll probably end up 
> with more attributes on items than we would if we didn't use queries 
> -- examples of this would be the various status flags that are used to 
> determine which query/item collection an item would be in.
>
> Ted
>

At first glance as long as their are not too many additional attribute 
require it does not seem like a substantial performance hit. If we used 
a Notification Manager there would be performance issues their as well 
with the registering and unregistering of listeners and the callback 
iterations.

Am I correct in assuming that the additional attributes will take up 
more disk space in the Repository but will not
adversely effect performance?

> On Nov 8, 2004, at 12:08 PM, Brian Kirsch wrote:
>
>> Ted,
>> Perhaps I misunderstood but I thought the Query notifications were 
>> separate from any work we are talking about on
>> Notification Manager. My understanding was that I could ask to be 
>> notified by your code when a change occurs in an item collection 
>> (adding / removing of Mail). My question was do you see any issues 
>> with using your Query notifications as the
>> main mechanism for inter-layer communication? This would be 
>> independent of any Notification Manager. And in fact if we use item 
>> collection notifications  then the role and need for a dedicated 
>> Notification Manager would need to be revisited.
>>
>>
>>
>> Brian Kirsch - Email Framework Engineer
>> Open Source Applications Foundation
>> 543 Howard St. 5th Floor 
>> San Francisco, CA 94105 
>> (415) 946-3056 
>>
>> On Nov 8, 2004, at 11:33 AM, Ted Leung wrote:
>>
>>>
>>> On Nov 8, 2004, at 11:21 AM, Brian Kirsch wrote:
>>>
>>>> Ted,
>>>> Thanks for the feedback.
>>>>
>>>> As the Query owner do you see any issues or drawbacks to using 
>>>> query notifications as the means of inter layer communication?
>>>
>>> The big issue that I see is the performance of notification.  You're 
>>> looking at substantial overhead vs just calling a method (or a few 
>>> methods).   The big advantage is that use of notification allows us 
>>> to decouple components from one another as long as the subscription 
>>> names are well known.  If subscriptions are items, then we could 
>>> store them in the repository and you could query for the appropriate 
>>> subscription.  This would take the place of looking for a particular 
>>> service via some kind of name service (think of using JNDI to lookup 
>>> the implementations of various functionality in a J2EE context).
>>>
>>> Since we are also considering extending notification to allow 
>>> asynchronous processing of notifications and cross thread 
>>> notification, there may be some additional benefits.
>>>
>>>>
>>>> I agree with you that although the original suggestion of using 
>>>> queries was made to solve the Mail dependencies we are
>>>> really talking about fundamental architecture design that will be 
>>>> the pattern that other Chandler services and Third party parcels 
>>>> will implement. I am more than happy to do the write up but I will 
>>>> need some participation / feedback from yourself and Bryan 
>>>> regarding the CPIA / Query language portions of the write up.
>>>
>>> I'm happy to contribute -- I think it would be good to have Bryan 
>>> take a crack at things first, and then I can review/fix what he 
>>> does.  I'd like to get Bryan's feedback/insights/etc before he gets 
>>> totally brainwashed ;-).   Bryan if you're not comfortable with 
>>> that, just say so, and I'm happy to get involved as necessary.
>>>
>>> Ted
>>>
>>>>
>>>>
>>>>
>>>> Brian Kirsch - Email Framework Engineer
>>>> Open Source Applications Foundation
>>>> 543 Howard St. 5th Floor 
>>>> San Francisco, CA 94105 
>>>> (415) 946-3056 
>>>>
>>>> On Nov 8, 2004, at 11:09 AM, Ted Leung wrote:
>>>>
>>>>> Brian,
>>>>>
>>>>> Thanks for summarizing our meeting -- it's really helpful.   I 
>>>>> wanted to add/accentuate 2 points:
>>>>>
>>>>> 1. It seems to me that Bryan's suggestion for using query 
>>>>> triggered notifications for inter layer/inter parcel communication 
>>>>> is a fundamental architectural pattern for Chandler.  This is 
>>>>> something that hasn't quite sunk into our thinking -- most of us 
>>>>> were familiar with the general notions (especially having been in 
>>>>> the meeting on notification), but it still didn't leap to mind as 
>>>>> the first solution (at least for me).  So I think it's worth 
>>>>> stating/re-stating this principle, as well as documenting it.
>>>>>
>>>>> 2. There's a bit of squishiness in our discussions regarding where 
>>>>> certain kinds of functionality should live.  Taking the example of 
>>>>> Mail, there were questions about where particular parts of the 
>>>>> Mail functionality should live: in the content model, in the 
>>>>> services layer, etc.   We should really be looking at this closely 
>>>>> (We'll probably run into the same issues with ZaoBao, at least) 
>>>>> and try to distill out some guidlines for how this should be done.
>>>>>
>>>>> Other than that, I think that this was a good set of thinking, and 
>>>>> that Brian ought to go ahead on a writeup.
>>>>>
>>>>> Ted
>>>>>
>>>>> On Nov 5, 2004, at 12:34 PM, Brian Kirsch wrote:
>>>>>
>>>>>> Hello,
>>>>>> Ted, John, Bryan, Donn, and myself had an informal meeting 
>>>>>> yesterday to discuss the best way to ensure proper division 
>>>>>> between the CPIA, Services, and Content Model layers. This 
>>>>>> discussion was driven by Donn and my work on mail in .4. We found 
>>>>>> with the current architecture that dependencies arose across 
>>>>>> layers. The Content Model imported code from the CPIA and 
>>>>>> Services layers. The Services layer imported code from the CPIA 
>>>>>> layer.  This is not a good thing and should be addressed in the 
>>>>>> .5 release.
>>>>>>
>>>>>> Based on the meeting we determined that Content Model items 
>>>>>> should have some basic knowledge of how to perform operations on 
>>>>>> themselves. For example, an EmailAddress should be able to format 
>>>>>> and validate itself.
>>>>>>
>>>>>> And that the services layer should know how to take action on a 
>>>>>> Content item. For example, the mail service should  know how to 
>>>>>> convert messages to and from MailMessage items and send and 
>>>>>> receive those items via SMTP  and IMAP (with POP to come in a 
>>>>>> future release).
>>>>>>
>>>>>> Bryan came up with a very good suggestion to use ItemCollections 
>>>>>> and Ted's Query language as the communication mechanism between 
>>>>>> the layers.
>>>>>>
>>>>>> In the case of mail, there would be an In item collection and Out 
>>>>>> item collection for each IMAP and SMTP account. When a user sends 
>>>>>> mail the CPIA layer places the MailMessage in the appropriate Out 
>>>>>> collection. The mail service would register as a listener to the 
>>>>>> Out item collection via Ted's Query API.  The mail service would 
>>>>>> be notified when that MailMessage  was added to the Out 
>>>>>> collection. The mail service then sends the MailMesage item via 
>>>>>> SMTP  and moves  the MailMessage from the Out collection to the 
>>>>>> Sent collection. The CPIA layer gets notified of addition of the 
>>>>>> MailMessage to the Sent collection and displays a status message 
>>>>>> such as 'Your message was sent'. This is a very clean model for 
>>>>>> services / CPIA communication and can be applied to all Chandler 
>>>>>> services (webdav etc).
>>>>>>
>>>>>> This change will require some significant re-factoring of the 
>>>>>> Mail code which will take some time (2 weeks?) to do.
>>>>>>
>>>>>> Another good suggestion was made that perhaps sharing would be a 
>>>>>> good first candidate for implement this new design since much of 
>>>>>> sharing is being reworked anyways and it does not have the 
>>>>>> complex dependencies the mail service does (Twisted, Threaded 
>>>>>> commits,  Repository view setting and unsetting).
>>>>>>
>>>>>> Thoughts on the proposal?
>>>>>>
>>>>>> If everyone agrees this is a good way to go the next step is to 
>>>>>> get a formal write-up together for review.
>>>>>>
>>>>>>
>>>>>> Brian Kirsch - Email Framework Engineer
>>>>>> Open Source Applications Foundation
>>>>>> 543 Howard St. 5th Floor 
>>>>>> San Francisco, CA 94105 
>>>>>> (415) 946-3056 
>>>>>>
>>>>>>
>>>>>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>>>>>
>>>>>> Open Source Applications Foundation "Dev" mailing list
>>>>>> http://lists.osafoundation.org/mailman/listinfo/dev
>>>>>>
>>>>> ----
>>>>> Ted Leung                 Open Source Applications Foundation 
>>>>> (OSAF)
>>>>> PGP Fingerprint: 1003 7870 251F FA71 A59A  CEE3 BEBA 2B87 F5FC 4B42
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
> ----
> Ted Leung                 Open Source Applications Foundation (OSAF)
> PGP Fingerprint: 1003 7870 251F FA71 A59A  CEE3 BEBA 2B87 F5FC 4B42
>
>