[Chandler-dev] [Sum] The Great Architecture Discussion of 2007

[D John Anderson]

I think we are better off to evolve the existing repository rather  
than completely replace it with and object relational mapping on top  
of SQL at this stage of Chandler's life.

I say this is because I believe such a replacement is a very large  
task with an uncertain outcome. I'd rather work on eliminating extra  
unnecessary notifications which I think will give us improved  
performance with less work.

Personally I'm more excited about the prospect of getting more users  
and making them happy than doing a major rewrite -- but I'm still in  
favor of incremental architectural improvements.

John

On Oct 9, 2007, at 10:39 PM, Andi Vajda wrote:

>
> On Tue, 9 Oct 2007, Phillip J. Eby wrote:
>
>> The thing is, once you look at this from the app layering  
>> perspective, the mismatch between the relatively simple things the  
>> app is trying to do, and the very powerful generality of the  
>> repository, becomes more apparent.
>
> So, I ask again to the general readership of this list - Phillip, I  
> think
> your opinion is pretty clear - do we want to compile Chandler into a
> "relatively simple" app like Mail.app or do we want to keep the  
> "powerful
> generality" part of the vision and implementation of Chandler when  
> I joined
> the project in 2003 ?
>
>>> If that's the route we'd like to take Chandler to, fine. That  
>>> should be clearly stated.
>>
>> I believe Katie has already stated it, even as far back as the  
>> creation of the schema API.  But I imagine she will clarify it  
>> again if needed.
>
> Katie, I certainly missed your earlier statement in this direction.
> What is your take on this ? Where do you stand on this ?
>
> Phillip's opinions in this area are not news to me. I even offered  
> him my repository job shortly after he joined OSAF when we had a  
> similar conversations. For various reasons, he didn't take it then.  
> I'm ready to
> make the same offer again as we're ready - it seems - to abandon  
> another -
> major this time - part of the early vision.
>
>> I mean that by implementing a skiplist *inside* of BerkeleyDB  
>> rather than using a native BerkeleyDB structure, we're adding an  
>> "interpretation" layer there.
>
> What is the native Berkeley DB structure that corresponds to a  
> skiplist ?
> A skiplist is just a fatter ref collection (fatter because of longer
> reaching links between nodes). Are ref collections also duplicating an
> existing Berkeley DB structure ?
>
>> Yes.  The key distinction vis-a-vis relational vs. repository, is  
>> that we
>> would now be adding another Python layer, in addition to the those  
>> that
>> already exist.  Whereas, if we used a Python ORM, we would have  
>> just the
>> mapping and an all-C backend that we don't have to maintain.  In  
>> fact, the
>> mapping layer might also be maintained by someone else, if we use  
>> one of the
>> many O-R mappers for Python such as SQLObject, Storm, Axiom, DejaVu,
>> SQLAlchemy, Mother,...  and probably others I've forgotten about.
>
> The repository back-end is already mostly C. Many of the places  
> where the
> repository code was spending time were moved to C. There is more C  
> code than
> python code in the repository by now. Most of the time today is  
> spent in
> firing thousands of useless notifications. Loading and committing  
> items is
> pretty fast in comparison. (Merging items is still pretty slow,  
> though)
> I don't know where you're getting that the repository is slow in  
> doing its
> persistence job. It slows down in its mapping layer, in the layer  
> between
> the persistence layer and the app. We have so many sorted indexes  
> because of
> the generality of the Sidebar that whenever a trivial change is  
> made in an
> attribute, they all have to be indexed. I don't see where this work  
> comes
> for free, either code-wise, or perf-wise, in a relational model. If  
> you
> cache the results of sorted queries then you have to maintain these  
> caches
> there as well.
>
> By killing Chandler's use of the existing powerful/general  
> repository you can
> also kill some of Chandler's powerful/general features, thereby  
> fixing the
> perceived performance problems that plague it today. This is  
> certainly a
> way to solve the problem.
>
> OSAF management and developers, is this what we want ?
>
> Andi..
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev