[Ietf-caldav] polling

[Dan Winship]

On Mon, 2004-08-16 at 14:36 -0700, Lisa Dusseault wrote: 
> Dan Winship wrote:
> > I have some thoughts on a "polling for changes" REPORT that I'll post
> > tomorrow.
> 
> Dan, if this could be used for generic WebDAV collections, I know of 
> other groups who would be interested in this as a completely 
> calendaring-independent WebDAV feature.

Yes, it could be used generically in WebDAV.

(The exact problem that the following protocol is trying to solve is
"make it possible for a client to find out about all modifications to,
additions to, and deletions from a collection since an arbitrary time in
the past, without requiring the server to keep an unbounded amount of
historical data or per-client state".)

Modify/addition polling is easy; you just need a uid and last-modified-
timestamp for every object in the collection, and when the client asks
the server "what's changed since TIME", it returns the uids of all
objects with modtimes > TIME. (And then the client requests whatever
additional data it needs for those UIDs. Or alternately, it could
specify as part of the REPORT request what properties it cares about,
and they could be returned as part of the report.)

To allow polling for deletions, the objects in the collection need to
have IMAP-style sequence numbers (meaning "A relative position from 1 to
the number of messages in the mailbox. As each new message is added, it
is assigned a message sequence number that is 1 higher than the number
of messages in the mailbox before that new message was added. ... When a
message is removed from the mailbox, the message sequence number for all
subsequent messages is decremented." This seems like it would have
something to do with WebDAV Ordered Collections, except that it would be
a "server-maintained ordering", which is unspecified by that
document...). In addition to the sequence numbers, each object needs a
"predecessor deleted" timestamp that records the last time its immediate
predecessor-in-sequence-number was deleted. When the client polls for
"what's changed since TIME", the deleted objects report contains the uid
and sequence number of each object whose predecessor-deleted timestamp
is after TIME, and then the client deletes objects from its cache as
needed to make the sequence numbers match up.


I know that's not terribly clear, so here's an example. At noon, I
create a calendar with a bunch of items in it:

        Seq UID   LastMod  PredDel  Summary            DTSTART, etc
         1  306A   12:00     1970   Lunch              12:00 ...
         2  9294   12:00     1970   Conference call    11:00 ...
         3  5798   12:00     1970   Birthday party     17:00 ...
         4  0FEE   12:00     1970   Planning meeting   14:00 ...
         5  53A1   12:00     1970   Company holiday    2004-09-06 ...

(where "1970" means "a timestamp which is earlier than the oldest item
in the collection".)

Then, *from another client*, I make some changes. First the conference
call gets rescheduled, so I update its DTSTART, and the server updates
its modtime:

         2  9294  *12:15*    1970   Conference call    *13:00* ...

Then the birthday party is canceled, so I deleted it, and the server
updates the predecessor-deleted time of the following item, and
decrements the sequence numbers of all following items:

        *3* 0FEE   12:00   *12:30*  Planning meeting   14:00 ...
        *4* 53A1   12:00     1970   Company holiday    2004-09-06 ...

Then I add a new item:

         5  99BB   12:45     1970   Vacation           2004-09-20 ...

Then the planning meeting gets canceled too:

        *3* 53A1   12:00    *1:00*  Company holiday    2004-09-06 ...
        *4* 99BB   12:45     1970   Vacation           2004-09-20 ...

So now the server has:

        Seq UID   LastMod  PredDel  Summary            DTSTART, etc
         1  306A   12:00     1970   Lunch              12:00 ...
         2  9294   12:15     1970   Conference call    13:00 ...
         3  53A1   12:00     1:00   Company holiday    2004-09-06 ...
         4  99BB   12:45     1970   Vacation           2004-09-20 ...

while the original client still has the original version of the data
cached. So it asks, what's changed since 12:00, and the server responds
(in some appropriately XMLish way):

          4 objects in collection
        
          Modified/Added uids: 
            9294
            99BB
        
          Removed objects before:
            seq 3, uid 53A1

To handle the modified/added part, the client asks for whatever
properties it cares about on uids 9294 and 99BB. To handle the removed
part, it finds uid 53A1 in the cache, sees that it has sequence number 5
there, and so deletes its two immediate predecessors so it will have
sequence number 3 like in the report. At that point, it will be
synchronized with the server. (If there were multiple objects mentioned
in the removed part, they have to be synchronized in sequence number
order for things to work out correctly.)

The "4 objects" part of the response isn't needed in this example, but
would have been needed if one or more objects had been deleted from the
end of the collection.

-- Dan