[Dev] Fwd: Feedback on query doc

[Ted Leung]

Chao gave his permission for me to repost these exchanges that we had  
regarding the query system.   I hope this is useful to folks interested  
in the query system.

Begin forwarded message:

> From: Ted Leung <twl at osafoundation.org>
> Date: February 15, 2004 11:48:00 PM PST
> To: Chih-Chao Lam <chao at osafoundation.org>
> Cc: Brian Douglas Skinner <skinner at osafoundation.org>
> Subject: Re: Feedback on query doc
>
> Chao,
>
> Thanks for taking the time to give this feedback -- it really helps!
>
> On Feb 13, 2004, at 6:52 PM, Chih-Chao Lam wrote:
>
>> Hi Ted,
>>
>> Here's some feedback and more questions on the query doc
>> <http://wiki.osafoundation.org/twiki/bin/view/Chandler/ 
>> ChandlerQuerySystem>.
>> Many of my questions probably just reflect my ignorance on your  
>> terminology and concepts.
>>
>> 1) Requirements
>> - Meta question: how were the requirements arrived? Are they end-user  
>> requirements? architecture requirements? optimization requirements?  
>> etc. May be useful to categorize the requirements to provide more  
>> context.
>
> This is a good question -- actually you uncovered my secret agenda --  
> I was trying to make all the requirements that I heard explicit.   
> That's doesn't necessarily guarantee that I captured all the  
> requirements.   There are very few end user requirements here -- I  
> think that partially because I'm not sure how the query functionality  
> is going to be exposed to the end user.  Right now I'm kind of  
> assuming that I'm building a tool for the apps groups, but it's  
> possible that the query language may leak up to the end user.
>
>> - Specific questions about requirements:
>> 	- What's the reasoning behind #3? Queries can be transient (not  
>> stored in repository) too, right?
>
> Here I mean the data that describes what the query itself is.  The  
> result of the query can be either transient or persistent.  But you're  
> right that we can have transient items.   I'm not aware of any systems  
> where the query is stored in the repository like we are planning to  
> do, so that's why I tried to highlight this one.  I probably made it  
> too strong.
>
>> 	- Not sure what #5 means specifically. Can you give examples? Is it  
>> just a corollary of #4?
>
> It's possible to specify a query in a way that gives explicit  
> instructions on how to find the items that satisfies the boolean  
> condition stipulated in 4.   You could read the "FOR i ..." part of a  
> query as actual for loop over some set of items in the repository.    
> That is bad because you typically want the computer (in the form of a  
> query optimizer) to figure out the best/fastest way to produce the  
> answer.  Specifying an actual for loop is bad because when we start to  
> index attributes, we'd have to change the loop to iterate over  
> something that came back from an index.  So while the "FOR i ..."  
> stuff looks like a loop, it won't necessarily be implemented in the  
> obvious way.
>
>> 	- #8: Don't quite understand. Please explain further.
>
> This is something that John asked for.  Assume that you have a query  
> that is run over and over, even between Chandler runs.   What John  
> wanted was a way to ask "how many items did I get the last time I ran  
> this query".  It was a way for him to know how slow/fast a query was  
> going to run, or how likely it was that he would get a certain amount  
> of data.
>
>> 	- #9: What's attribute traversal?
>
> All managers who have an Employee who's Spouse's dog is a Golden  
> Retriever.  If you start with a manager item, you "reach" the dog via  
> traversing attributes of items.
>
>> 	- #12: Even as results are produced incrementally, the total result  
>> set size should be known at the beginning of the iteration, right?  
>> [is this what #8 is referring to?]
>
> Not necessarily.  If you've never run the query against the repository  
> before, you don't really know how many items will satisfy it.
>
>> 	- #13: Not sure what this means. How do items enter/exit a result set
>
> This is another way of saying Brian's Observable queries.  If I have a  
> query that is used to compose a view, say "All mail messages send by  
> John", and we have agents/threads that retrieve mail every n minutes,  
> then we don't want to re-run the query every n minutes.  We want any  
> new messages from John to be "added" to the set of items returned by  
> the query for the view.
>
>> - we may have more requirements from the design side, let me get back  
>> to you on this after we've arrived at a more complete set of user  
>> scenarios
>
> I'm looking forward to getting these.
>
>>
>> 2) New Kinds
>> - Would Collection, Query inherit from Item? In which case, would  
>> "name" just be the same as attribute"Display Name" from Item?
>
> On inheritance, yes.  name could be the DisplayName attribute.
>
>> - Not sure, but we may want UsersCollection/Playlist/ItemCollection  
>> to be a ContentItem [data model digression question: do we support  
>> multiple inheritance in the Chandler data model? such that  
>> ItemCollection can inherit from both Collection and ContentItem? If  
>> not, how do we deal with such issues?]
>
> Good question.  As Brian points out, we have MI support in the data  
> model/repository.  Unfortunately, I think we're going to need it, and  
> not only because of this issue.
>
>> - Should UserCollection/Playlist/ItemCollection contain the Query as  
>> an attribute?
>
> I'm guessing this is how you'd want to do it.  The apps team is  
> probably who should define this answer.
>
>>
>> 3) General questions/comments
>> - Are queries at all related to sorting and sort orders? Or sorting  
>> dealt with somewhere else in the architecture?
>
> Sort order of the result set can be handled by the query - it's not  
> specified in the proposal, but it should be (I planned to do it later)
>
>> - When running python code in the queries, is dynamic scoping  
>> supported (as opposed to lexical scoping) for variables?
>
> I was planning for lexical scoping since that's implied in the Python  
> programming model from 2.2 forward.  Do you have a use case for  
> dynamic scoping?  This is an area of the proposal that needs to be  
> detailed out.
>
>> - A design open issue area: we are considering whether to exposed a  
>> query syntax/language to the end-user. This could be related to the  
>> URL bar. Any ideas how end-user queries should look like?
>
> This is a tough one.  I remember reading something in one of the  
> descriptions of CPIA that described a spreadsheet formula type of  
> notation for blocks, but I've never found spreadsheet formulas to be  
> that readable for anything more complicated than functions of 2 or 3  
> arguments).
>
> You could try to "dress down" the syntax that I provided by adding in  
> English/Natural language words, but I'm not sure how much that would  
> help.   I've just finished reading a book on AppleScript, and I'm both  
> impressed and horrified by the approach that they took there.   
> Actually, most of my horror has to do with the under-specification of  
> AppleScript and the problems that it leads to for today's AppleScript  
> programmers.
>
> I think I always assumed that we'd have some sort of GUI query  
> building tool for end users, since that's typically what you see.
>
>> - When the result set of queries are changed, can the query itself be  
>> notified? Brian had suggested a concept known as Observable Queries?  
>> Is this at all in the scope of your proposal? Or is this  
>> architecturally dealt with somewhere else? I think it's an important  
>> feature to have somewhere in Chandler.
>> http://wiki.osafoundation.org/twiki/bin/view/Jungle/ObservableQueries
>>
>> Hope this helps,
>> chao
>
> It was very helpful.  Could I post my reply (I didn't edit yours out)  
> to the dev list as well?
>
----
Ted Leung                 Open Source Applications Foundation (OSAF)
PGP Fingerprint: 1003 7870 251F FA71 A59A  CEE3 BEBA 2B87 F5FC 4B42