[Dev] Schema circularity and extensibility problems

[John Anderson]

Hi Phillip:

My biggest concern about this proposal is that it will be hard for 
people to understand/learn -- the extra level of complexity doesn't seem 
worth it to me.

Phillip J. Eby wrote:

> During the m5 milestone, there were at least three schema problems 
> that I know of, that shared a common root cause:
>
> * AbstractCollection.subscribers - it was desired that this be a 
> bidirectional reference attribute, but there was then no meaningful 
> place to put the "other end" of the relationship, without creating a 
> dummy abstract class like "Subscriber" to act as a mixin.

We've solved these problems by having classes that care about 
subscribers refer to a global attribute, which works pretty well.

>
> * AbstractCollection.color - it was desirable for certain parts of the 
> system to associate a color with collections, but this required 
> modifying the AbstractCollection base class to add the attribute, even 
> though collections in the abstract don't have color.  :)

I'm not sure that using this technique to add color to 
AbstractCollection would satisfy people who don't want to pollute 
collections with UI data. Alternatively a simple dictionary that maps 
collections to UI items seems like a simpler and better solution to the 
pollution problem.

>
> * sharing.UIDMap.items - this was a collection mapping iCal UIDs to 
> calendar events, but to be a bidirectional reference, it needed an 
> inverse attribute on CalendarEventMixin, thereby creating an 
> undesirable circular dependency between osaf.pim.calendar and 
> osaf.sharing.

I do think we need to come up with a solution to the circular dependency 
problem, however, the proposed solution doesn't seem ideal because the 
way you access the attributes needs to change and it adds a new level of 
complexity to learn.

>
> In addition to these three specific issues, there also have been 
> occasional problems with people getting "cannot be modified after use" 
> errors, or other errors having to do with setting up bidirectional 
> relationships across parcels or between more than two kinds.
>
> The common cause of all of these problems is that there is currently 
> no easy way for a parcel to add attributes to existing kinds, without 
> modifying the code of the existing kind to refer to the new kind.  
> This problem also affects third-party extenders of Chandler.  For 
> example, suppose somebody wants to create an "accessibility" parcel 
> that allows assigning sounds to collections instead of colors?  :)
>
>
> So, here's what I'd like to propose:
>
> 1. Allow defining "anonymous" inverse relationships.  If we wanted 
> 'AbstractCollections.subscribers' to be a bidirectional reference, we 
> would need only do this:
>
>     # create an unordered, many-to-many relationship with Item
>     subscribers = schema.Many(inverse=schema.Many())
>
> Similarly, the sharing.UIDMap.items attribute could be defined with:
>
>     # create an ordered many-to-one relationship
>     items = schema.Sequence(pim.CalendarEventMixin, inverse=schema.One())
>
> 2. In the event that something like an attribute editor (or some other 
> object or API that needs an attribute name) needs to be pointed at one 
> of these "anonymous" attributes, they will be accessible via a "fully 
> qualified" attribute name.  For example, to access the collections an 
> item is subscribed to, you could get its 
> "osaf.pim.AbstractCollection.subscribers.inverse" attribute.  You 
> can't get this attribute statically in Python code; you have to use 
> getattr() or getAttributeValue() or any of the other normal APIs that 
> take attribute names, and pass in the string 
> "osaf.pim.AbstractCollection.subscribers.inverse".
>
> 3. Implement a convenience API that lets you use the .inverse 
> directly, in place of using the long name, e.g. something like this:
>
>     pim.AbstractCollection.subscribers.inverse(someObject)
>
> could perhaps be used to get the attribute in a more type-safe way.  
> If the attribute is frequently used in a given module, it can do 
> something like this at the top of the module to create a shortcut:
>
>     subscribees_of = pim.AbstractCollection.subscribers.inverse
>
> and then just use it directly:
>
>     # returns the collections someObject subscribes to
>     subscribees_of(someObject)
>
> 4. For the case where both ends of a relationship already exist (e.g. 
> AbstractCollection and ColorType), we would allow defining the 
> necessary attributes as follows:
>
>     class CollectionColor(schema.Relationship):
>         collections = schema.Many(pim.AbstractCollection)
>         color = schema.One(blocks.ColorType)
>
> Defining this class would create "anonymous" attributes on the 
> relevant kind(s).  If one side of the relationship is a type (e.g. 
> ColorType), then this would just create an anonymous value attribute 
> on the kind (e.g. a "CollectionColor.color" attribute on the 
> AbstractCollection kind).
>
> If both sides of the relationship are kinds, however, then each gets 
> an attribute that points to the other, creating a bidirectional 
> reference without modifying either kind's class definition.   For 
> example, if a third party parcel wanted to create a "likes" 
> relationship between contacts, it might do:
>
>     class Likes(schema.Relationship):
>         likees = schema.Many(pim.Contact)
>         likers = schema.Many(pim.Contact)
>
> This would create a many-to-many relationship between contacts, 
> *without* requiring the base Contact type to be modified.  However, it 
> will not conflict with any other third-party extension that creates 
> such attributes, nor will it be affected if Chandler later adds 
> "likees" and "likers" attributes to Contact.  This is because the 
> attribute names created by the above code will be 
> "some_parcel.Likes.likees" and "some_parcel.Likes.likers", and these 
> names will therefore not conflict with a "likees" or "likers" that 
> might be defined by some other parcel.
>
> To navigate this relationship, you would simply use the likes and 
> isLikedBy class attributes, as before:
>
>     Likes.likees(somebody)      # get the contacts who somebody likes
>     Likes.likers(somebody)      # get the contacts who like somebody
>     me in Likes.likees(you)     # do you like me?
>
>     Likes.likees(everybody).add(somebody)  # everybody likes somebody!
>     Likes.likees(me).remove(you)           # I don't like you any more
>
> As you can see, this provides a fairly usable API for parcels that 
> create new relationships; it's not quite as convenient as being able 
> to say 'somebody.likees' directly, but it doesn't require the 
> Chandler-supplied schema to anticipate every possible future need, and 
> it doesn't create circular dependencies between parcels.
>
> Some of you may remember ideas like these from my Spike prototype 
> earlier this year, so these are not really anything new.  There's even 
> a documented implementation of them in Spike; see:
>
>     
> http://svn.osafoundation.org/chandler/trunk/internal/Spike/src/spike/schema.txt 
>
>
> and there's some additional discussion in:
>
>     
> http://svn.osafoundation.org/chandler/trunk/internal/Spike/src/spike/overview.txt 
>
>
> But at the time I was creating the Spike-like schema API for Chandler, 
> it was not at all clear to me how I could implement these ideas using 
> the repository, and also the need for them didn't seem to be an 
> immediate issue.  Now, however, the need has popped up repeatedly, and 
> with Andi's help I've figured out a basic idea for how to make more or 
> less the same API work atop the repository's schema mechanisms.
>
> I'd like to hear your comments and questions, to make sure this is 
> going in the right direction.  By the way, I believe these changes 
> should also allow us to get rid of some of the annoying schema errors 
> we currently get that result from circular dependencies, including the 
> dreaded "cannot be modified after use" error, and we might also be 
> able to get rid of the confusing distinction between 'otherName' and 
> 'inverse', as 'otherName' is essentially only needed right now as a 
> workaround for the absence of the API features I've described here.
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "Dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/dev