[Chandler-dev] Dashboard date and lastmodified proposal

[Bryan Stearns]

Before I left for OSCON, faced with some dashboard design issues, I 
started a conversation here about "the future of redirectTo". I got some 
good responses (thanks PJE, Grant, John, and Andi!), but I think I did a 
bad job of describing the issues - I probably shouldn't've started with 
redirectTo :-! .

I spent parts of the last few days trying to rephrase the issues, but 
got stuck trying to find a good way to describe the problems. Instead, 
after a great conversation with Jeffrey and Grant, I'm going to propose 
two solutions. :-)

There are a couple of problems I'm trying to deal with:
- The 'Date' column in the dashboard is sorted by something complex: 
first, by 'the next important date' for the item, then its triage 
status, then recency of triage-status change. The really complex part is 
that 'next important date', which the dashboard spec defines as:
> If both the custom tickler and event start dates/times have past, in 
> other words, if there is no NEXT important date, display the Start 
> date and time of the event.
> If the custom tickler date has passed and there is no event date/time, 
> keep displaying the Custom Tickler date.
> If an Item is not an Event, always display the most RECENT date: 
> either the Date last modified or the Date sent which ever one happened 
> most recently.
> If an Item has none of these dates, the date column displays the Date 
> created.
- Note that one of the terms in that definition is "last modified": it's 
the time that a user-meaningful change got made (examples: dragging the 
event in the calendar or changing text in the note body should update 
the last-modified date; however, selecting the item in a different 
block, which would change the selectedIn attribute, would not update the 
last-modified date.

-- First, managing Date indexing --

At various times, it's been suggested that we use onValueChanged to 
update values used for indexing this way; however, there's a concern 
that this would be expensive: just for the date column, it would mean 
that every ContentItem would have an onValueChanged, which would get 
called for every attribute change.

I had suggested using a Calculated attribute (essentially a python 
'property' with schema type and dependency information), but (a) the 
repository doesn't know about them, (b) managing the monitors required 
to make sure it stays up-to-date is tricky, (c) everyone who wants to do 
indexing like this would have to understand the monitor mechanism, and 
(d) I'm not even sure it'll work.

Andi had suggested using a 'compare' index, which requires a method be 
provided to compare two items; this seemed to align with using 
Calculated attributes. However, since we also display the 
nextImportantDate values, it seemed strange to calculate it for indexing 
and again for display; this made me think that we really do want to 
store the value we calculate. Also, 'compare' indexes require loading 
O(logN) whole items during index updates, which seems expensive (again, 
given that Date ordering involves all ContentItems, that lastModified is 
one of the terms involved, and that each collection would have its own 
index).

So, that got me thinking that what I'd really like to be able to do is 
declaratively specify what I want in the schema, which led to this syntax:

class ContentItem(schema.Item):
   nextImportantDate = schema.One(schema.DateTime,
                                  *update='updateNextImportantDate',*
                                  
*basedOn=(CalendarEventAnnotation.startTime,*
                                           *lastModified,*
                                           *nextReminderTime)*)

  def updateNextImportantDate(self, attribute):
    self.date = ...

which means: "This is a DateTime attribute. Its value could change if 
anyone changes CalendarEventAnnotation.startTime, lastModified, or 
nextReminderTime attributes on this item, so if someone does, please 
call my updateNextImportantDate method so I can maybe re-set it." This 
would let me use nextImportantDate as a term in any indexing expression 
- I could use an ordinary attribute index on ('nextImportantDate', 
'triageStatus', 'triageStatusChanged') on the Date column, which 
wouldn't require whole-item loads during indexing.

(In reviewing the emails from the last thread just before sending this, 
I see that Grant suggested something very similar to this, which I 
didn't remember when I wrote the above. So, if this is a good idea, he 
gets the credit.)

The hard part, of course, would be implementing this, and I'm completely 
hand-waving it. Ideally, the repository would take care of managing 
whatever monitors or item-watchers it needs to support this.

-- Second, lastModified --

Then I thought about the lastModified problem (which also comes with 
"lastModifiedBy", the last person to change an item): We know that only 
some changes are "substantive" and should update 
lastModified/lastModifiedBy, but we don't want the repository to have to 
know which are which.

Again, onValueChanged was an option, but it's even less of a good choice 
for lastModified than it was for nextImportantDate, for a few reasons: 
more attribute-changing operations would be involved, and doing the 
"substantive?" decision for each one seems bad. The lastModified case 
also gives us another reason not to use onValueChanged for keeping 
nextImportantDate up to date: it'd be tricky to avoid infinite loops 
given the dependence of 'date' on 'lastModified'. Finally, having 
onValueChanged on ContentItem used for maintaining 'lastModified' as 
well as 'date' would make it complicated for a subkind to also 
participate in the date calculation somehow.

I'm positing that "substantiveness" is handleable on an 
attribute-by-attribute basis, so I again thought about a schema-based 
declarative approach. First, for what should happen when a change is 
made to a "substantive" attribute:

class ContentItem(schema.Item):
    schema.kindInfo(*onSubstantiveChange='onSubstantiveChange'*)
   
    def onSubstantiveChange(self, attribute):
        self.lastModified = datetime.now()
        self.lastModifiedBy = me

Which says: "When substantive attributes on this item change, call this 
method." (I don't expect there to be multiple implementations of 
onSubstantiveChange - the purpose is to isolate Chandlerness from the 
repository, not to provide Chandler item types with a generalized change 
mechanism they can override.)

So, what triggers calls to onSubstantiveChange? By default, all 
attributes would be considered substantive, unless they specify 
"substantive=False" in their definitions:

class CalendarEvent(ContentItem):
    startTime = schema.One(schema.DateTime) # changing startTime would 
call onSubstantiveChange
    selectedIn = schema.One('Block',
                            *substantive=False*) # changing selectedIn 
would NOT call onSubstantiveChange

(I figured the default should be "substantive", because it's more likely 
that most outside developers' additions will be user-visible and not 
internal, so that's simpler for them. Of course,  lastModified's 
definition would include substantive=False - this solves the 
infinite-loop problem.

It appears that recurrence and Travis' IMAP stuff also wants the (same?) 
notion of what a substantive change is, so the same schema flag might be 
repurposable for that.

Thoughts? (I know my naming is arguably bad, but for now, please focus 
on the mechanisms, and we can debate the names later. :-) )

Thanks,
...Bryan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osafoundation.org/pipermail/chandler-dev/attachments/20060809/609cae92/attachment.htm