[Chandler-dev] [Sum] The Great Architecture Discussion of 2007

[D John Anderson]

On Oct 15, 2007, at 5:31 PM, Katie Capps Parlante wrote:

> Hi all, sorry for jumping in to this thread late.
>
> IMHO, Phillip is correct to point out that we're not really looking  
> at a dichotomy between "powerful generality" and "simple app  
> limited to mail/calendar".
>
> In particular, we want to keep the ability to extend app with new  
> kinds of end-user visible data types (content items: photos,  
> tweets, yadayada). This includes the ability to dump/reload the new  
> types as well as the ability to share the new types. EIM always had  
> these requirements in mind, for example.
>
> It would be nice to have a path to:
> - item history (though as PJE points out, this could be modeled  
> directly)
> - end user modification of the schema (in a limited way -- end user  
> adding fields to stamps, for example)
>
> What would we be losing?
> - Persistence of every aspect of the UI in the repository, block by  
> block

Persisting some UI state is easy (and desirable) -- after all that's  
what most preferences files contain. It would be easy to save state  
someplace other than the repository, e.g. a Python pickle.

> - UI objects (and possibly assorted other lower-case-i-items)  
> looking indistinguishable from user-content-items, generically  
> editable from a generic UI
>
> I'm happy to lose these things in service of better performance and  
> scalability for Chandler. (Not to mention better testability for  
> faster development cycles, cleaner extensibility, etc.)
>
> Cheers,
> Katie
>
> Phillip J. Eby wrote:
>> At 10:39 PM 10/9/2007 -0700, Andi Vajda wrote:
>>> On Tue, 9 Oct 2007, Phillip J. Eby wrote:
>>>
>>>> The thing is, once you look at this from the app layering  
>>>> perspective, the mismatch between the relatively simple things  
>>>> the app is trying to do, and the very powerful generality of the  
>>>> repository, becomes more apparent.
>>>
>>> So, I ask again to the general readership of this list - Phillip,  
>>> I think
>>> your opinion is pretty clear - do we want to compile Chandler into a
>>> "relatively simple" app like Mail.app or do we want to keep the  
>>> "powerful
>>> generality" part of the vision and implementation of Chandler  
>>> when I joined
>>> the project in 2003 ?
>> This is both a false dichotomy and a red herring, because:
>> 1. It is not an either-or choice, and
>> 2. Nothing I have proposed removes any ability of Chandler to have  
>> "powerful generality".
>> In fact, the domain refactoring Grant and I are working on  
>> significantly *improves* the generality of the application, as it  
>> decouples dashboard information from content items, so that it's  
>> possible for plugins to have "content items" generate an arbitrary  
>> assortment of dashboard entries.  The same is true for reminders.
>> (That is, we have a way to fully generalize recurrence and  
>> reminders, in a way that's quite extensible and reusable by  
>> plugins, that the current model does not support.)
>> This is only one example of the many ways in which I expect  
>> Chandler's current level of "powerful generality" to be expanded  
>> as a side-effect of the refactoring.
>> So, it would be helpful if you could clarify precisely what  
>> "generality" you're referring to, that you're afraid we're losing.
>> Note, too, that nothing that is being contemplated rules out the  
>> possibility of having user-defined schema at a later date.   
>> Nothing stops anyone from writing a plugin that contains a static  
>> schema that meta-models dynamic schema elements, for example.
>> However, the reality of the project is that OSAF actually creating  
>> such a plugin is out of scope for the foreseeable future, and was  
>> already declared such two years ago.
>>>>> If that's the route we'd like to take Chandler to, fine. That  
>>>>> should be clearly stated.
>>>>
>>>> I believe Katie has already stated it, even as far back as the  
>>>> creation of the schema API.  But I imagine she will clarify it  
>>>> again if needed.
>>>
>>> Katie, I certainly missed your earlier statement in this direction.
>>> What is your take on this ? Where do you stand on this ?
>> Based on your comments above, I suspect that the "this" which I am  
>> talking about is not the same as what you think you have missed.
>> What I'm referring to is that end-user programmability of Chandler  
>> has not been a primary goal of Chandler development for about two  
>> years now, and there has been a moratorium on new investment in  
>> that direction.  Indeed, even platform improvement was designated  
>> as secondary, with priority to be given only to platform issues  
>> affecting the core PIM application development.
>>> Phillip's opinions in this area are not news to me. I even  
>>> offered him my repository job shortly after he joined OSAF when  
>>> we had a similar conversations. For various reasons, he didn't  
>>> take it then.
>> And one of those reasons is that it's a silly thing to offer.   
>> You're not a samurai, so there's no need to commit hara-kiri just  
>> because some features got re-prioritized.  Put the sword away,  
>> already, nobody is questioning your honor.  :)
>>> I'm ready to
>>> make the same offer again as we're ready - it seems - to abandon  
>>> another -
>>> major this time - part of the early vision.
>> It hasn't been abandoned, but it was re-prioritized and  
>> deferred... even before I ever sent my resume' to OSAF.
>> I know, because I checked out the Chandler wiki before deciding to  
>> apply.  What I read there indicated quite clearly that end-user  
>> programming and "development platform" goals had been deferred, in  
>> favor of a focus on out-of-the-box functionality.
>> (If it had *not* said that, I wouldn't have bothered to apply.   
>> Projects without a clear vision of what to *exclude* from a  
>> current milestone, don't ever finish... like setuptools 0.6.  ;) )
>> To defer or exclude from a release, however, is not the same thing  
>> as to "abandon".
>>>> I mean that by implementing a skiplist *inside* of BerkeleyDB  
>>>> rather than using a native BerkeleyDB structure, we're adding an  
>>>> "interpretation" layer there.
>>>
>>> What is the native Berkeley DB structure that corresponds to a  
>>> skiplist ?
>> A BTree.  And the native Python equivalent is a list.  You may  
>> recall that at one time, I compared the performance of the  
>> skiplist implementation with a simple Python list object, and  
>> found that for lists of up to 50,000 items, Python lists were  
>> faster -- often significantly faster.  This is another good  
>> example of where we are reinventing wheels in Python code that  
>> somebody already wrote a mature implementation of in C.
>>> I don't know where you're getting that the repository is slow in  
>>> doing its
>>> persistence job. It slows down in its mapping layer, in the layer  
>>> between
>>> the persistence layer and the app. We have so many sorted indexes  
>>> because of
>>> the generality of the Sidebar that whenever a trivial change is  
>>> made in an
>>> attribute, they all have to be indexed. I don't see where this  
>>> work comes
>>> for free, either code-wise, or perf-wise, in a relational model.
>> Surely you're not comparing the way the repository does indexes  
>> with the way it works in a relational DB?  There aren't any  
>> "notifications" there, certainly not at the Python level.
>> I'm a little confused by your argument, though.  On the one hand,  
>> you've been saying that the application layer is what's causing  
>> performance problems.  However, if we were to refactor the  
>> application layer for best performance, it would no longer conform  
>> to the repository's view of the world as a persistent object store.
>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>> Open Source Applications Foundation "chandler-dev" mailing list
>> http://lists.osafoundation.org/mailman/listinfo/chandler-dev
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev