[Dev] New iterItems() convenience API

[Phillip J. Eby]

At 12:49 PM 6/25/2005 -0700, John Anderson wrote:
>Although I think queries are very important, I think andi's ref 
>collections (and soon sets) can better address many of the problems you'd 
>typically use a more traditional query for. They can make it really easy 
>to "hook up" new parcel pieces to existing pieces of chandler, something 
>that's awkward or slow to do with queries.

Agreed; for use cases where the developer's intention is to connect to a 
*particular* thing, where more than one choice is possible (e.g. membership 
in Chandler menus) then connecting to the item in parcel.xml is the obvious 
thing to do.  That's because it's an explicit part of the use case, "I want 
to add this menu item to that menu".

However, for many Chandler-extension use cases, simply creating an object 
of the appropriate type can and should be sufficient, because there isn't 
more than one choice.  The use cases are things like "I want to add a web 
server", or "I want my class to be called when Chandler starts", or "I want 
a callback every 30 minutes", and in these use cases, there is no explicit 
thing to "hook up" to.

So, in those cases, making the developer specify an additional link is 
redundant, a "dead chicken" in Zope project parlance.  (The name comes from 
the idea that making people do things they don't understand to satisfy an 
API's internal implementation requirements is like asking them to perform 
voodoo rites over their software, by "waving a dead chicken" over it.)


>  There are relatively efficient, scale well, can persist result sets, 
> keep track of ordered lists and provide notifications about changes to 
> the UI. It's important to understand all the tools at our disposal and 
> use the right tool for the task, rather than depending upon on a single 
> tool, e.g. a traditional query.

One difficulty that I have in communicating about this is that you are 
talking about very concrete things, but I'm talking about 
abstractions.  "Query" to you appears to mean a specific implementation 
that you have available now, but to me, the word "Query" is just a short 
way of saying "intentional set" - a set whose membership is determined by a 
rule, rather than by explicitly listing its contents.

So, at this conceptual level, whether this set is maintained as a 
refcollection or by searching through the database is of no significance to 
the API.  That's an implementation detail, and both approaches have 
efficiency tradeoffs (search costs vs. maintenance costs) which are 
well-understood in the database field.  The point is that Chandler often 
wants "all items meeting rule X", whatever that rule is, and however it's 
accomplished.  And for a lot of those use cases, the rule is defined as 
some subset of a type extent -- i.e., some subset of "all items of type X" 
(e.g. "all active POP accounts").

Now, as to *how* we get the items matching rule (in implementation terms), 
it doesn't matter, as long as it's reasonably efficient.  If KindQuery were 
to disappear, I would simply emulate it with a kind-extent biref managed by 
__init__, and it would work for everything except core schema items.  My 
point, however, is that parcel developers shouldn't be forced to perform 
those kind of gymnastics for such simple use cases, even if I have to do 
something like that to protect them from it.

It's the STASCTAP rule of API design: Simple Things Are Simple, Complex 
Things Are Possible.  Getting all items of a particular kind is a "Simple 
Thing" (i.e. a simple and common use case), it should therefore be simple 
for someone to express in the API.  Simplicity of expression of common use 
cases is what this is all about.  The current KindQuery API was way too 
complex for the simplicity of the use case, so I added iterItems().  If the 
implementation of iterItems() has to change to use something else, it 
doesn't really matter to the API, because the use case hasn't changed.

Conversely, if we were to say that because the implementation needs to 
change, we have to go through and change all iterItems() calls to do 
something else, then our design is probably a failure, just as a change to 
the application's implementation should (generally) not require us to 
change the application UI!  Developers are users too, and this is all about 
designing *their* (internal) "UI".