[Chandler-dev] [Sum] The Great Architecture Discussion of 2007

[Katie Capps Parlante]

Hi all, sorry for jumping in to this thread late.

IMHO, Phillip is correct to point out that we're not really looking at a 
dichotomy between "powerful generality" and "simple app limited to 
mail/calendar".

In particular, we want to keep the ability to extend app with new kinds 
of end-user visible data types (content items: photos, tweets, 
yadayada). This includes the ability to dump/reload the new types as 
well as the ability to share the new types. EIM always had these 
requirements in mind, for example.

It would be nice to have a path to:
- item history (though as PJE points out, this could be modeled directly)
- end user modification of the schema (in a limited way -- end user 
adding fields to stamps, for example)

What would we be losing?
- Persistence of every aspect of the UI in the repository, block by block
- UI objects (and possibly assorted other lower-case-i-items) looking 
indistinguishable from user-content-items, generically editable from a 
generic UI

I'm happy to lose these things in service of better performance and 
scalability for Chandler. (Not to mention better testability for faster 
development cycles, cleaner extensibility, etc.)

Cheers,
Katie

Phillip J. Eby wrote:
> At 10:39 PM 10/9/2007 -0700, Andi Vajda wrote:
> 
>> On Tue, 9 Oct 2007, Phillip J. Eby wrote:
>>
>>> The thing is, once you look at this from the app layering 
>>> perspective, the mismatch between the relatively simple things the 
>>> app is trying to do, and the very powerful generality of the 
>>> repository, becomes more apparent.
>>
>> So, I ask again to the general readership of this list - Phillip, I think
>> your opinion is pretty clear - do we want to compile Chandler into a
>> "relatively simple" app like Mail.app or do we want to keep the "powerful
>> generality" part of the vision and implementation of Chandler when I 
>> joined
>> the project in 2003 ?
> 
> This is both a false dichotomy and a red herring, because:
> 
> 1. It is not an either-or choice, and
> 2. Nothing I have proposed removes any ability of Chandler to have 
> "powerful generality".
> 
> In fact, the domain refactoring Grant and I are working on significantly 
> *improves* the generality of the application, as it decouples dashboard 
> information from content items, so that it's possible for plugins to 
> have "content items" generate an arbitrary assortment of dashboard 
> entries.  The same is true for reminders.
> 
> (That is, we have a way to fully generalize recurrence and reminders, in 
> a way that's quite extensible and reusable by plugins, that the current 
> model does not support.)
> 
> This is only one example of the many ways in which I expect Chandler's 
> current level of "powerful generality" to be expanded as a side-effect 
> of the refactoring.
> 
> So, it would be helpful if you could clarify precisely what "generality" 
> you're referring to, that you're afraid we're losing.
> 
> Note, too, that nothing that is being contemplated rules out the 
> possibility of having user-defined schema at a later date.  Nothing 
> stops anyone from writing a plugin that contains a static schema that 
> meta-models dynamic schema elements, for example.
> 
> However, the reality of the project is that OSAF actually creating such 
> a plugin is out of scope for the foreseeable future, and was already 
> declared such two years ago.
> 
> 
>>>> If that's the route we'd like to take Chandler to, fine. That should 
>>>> be clearly stated.
>>>
>>> I believe Katie has already stated it, even as far back as the 
>>> creation of the schema API.  But I imagine she will clarify it again 
>>> if needed.
>>
>> Katie, I certainly missed your earlier statement in this direction.
>> What is your take on this ? Where do you stand on this ?
> 
> Based on your comments above, I suspect that the "this" which I am 
> talking about is not the same as what you think you have missed.
> 
> What I'm referring to is that end-user programmability of Chandler has 
> not been a primary goal of Chandler development for about two years now, 
> and there has been a moratorium on new investment in that direction.  
> Indeed, even platform improvement was designated as secondary, with 
> priority to be given only to platform issues affecting the core PIM 
> application development.
> 
> 
>> Phillip's opinions in this area are not news to me. I even offered him 
>> my repository job shortly after he joined OSAF when we had a similar 
>> conversations. For various reasons, he didn't take it then.
> 
> And one of those reasons is that it's a silly thing to offer.  You're 
> not a samurai, so there's no need to commit hara-kiri just because some 
> features got re-prioritized.  Put the sword away, already, nobody is 
> questioning your honor.  :)
> 
> 
>> I'm ready to
>> make the same offer again as we're ready - it seems - to abandon 
>> another -
>> major this time - part of the early vision.
> 
> It hasn't been abandoned, but it was re-prioritized and deferred... even 
> before I ever sent my resume' to OSAF.
> 
> I know, because I checked out the Chandler wiki before deciding to 
> apply.  What I read there indicated quite clearly that end-user 
> programming and "development platform" goals had been deferred, in favor 
> of a focus on out-of-the-box functionality.
> 
> (If it had *not* said that, I wouldn't have bothered to apply.  Projects 
> without a clear vision of what to *exclude* from a current milestone, 
> don't ever finish... like setuptools 0.6.  ;) )
> 
> To defer or exclude from a release, however, is not the same thing as to 
> "abandon".
> 
> 
>>> I mean that by implementing a skiplist *inside* of BerkeleyDB rather 
>>> than using a native BerkeleyDB structure, we're adding an 
>>> "interpretation" layer there.
>>
>> What is the native Berkeley DB structure that corresponds to a skiplist ?
> 
> A BTree.  And the native Python equivalent is a list.  You may recall 
> that at one time, I compared the performance of the skiplist 
> implementation with a simple Python list object, and found that for 
> lists of up to 50,000 items, Python lists were faster -- often 
> significantly faster.  This is another good example of where we are 
> reinventing wheels in Python code that somebody already wrote a mature 
> implementation of in C.
> 
> 
>> I don't know where you're getting that the repository is slow in doing 
>> its
>> persistence job. It slows down in its mapping layer, in the layer between
>> the persistence layer and the app. We have so many sorted indexes 
>> because of
>> the generality of the Sidebar that whenever a trivial change is made 
>> in an
>> attribute, they all have to be indexed. I don't see where this work comes
>> for free, either code-wise, or perf-wise, in a relational model.
> 
> Surely you're not comparing the way the repository does indexes with the 
> way it works in a relational DB?  There aren't any "notifications" 
> there, certainly not at the Python level.
> 
> I'm a little confused by your argument, though.  On the one hand, you've 
> been saying that the application layer is what's causing performance 
> problems.  However, if we were to refactor the application layer for 
> best performance, it would no longer conform to the repository's view of 
> the world as a persistent object store.
> 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev