[Dev] Keeping Users' Data

[Phillip J. Eby]

At some point, Chandler is going to be keeping users' real data, and be the 
only place where that data is kept.  When that happens, we will be 
responsible for ensuring that their data remains intact across 
upgrades.  Even if this is done through some kind of import/export 
mechanism, we will need to keep track of the changes in our schema so that 
this *can* be done.

There are several key questions about this that remain open:

* When will we be committing to keeping users' data safe across 
upgrades?  Is it 0.7?  0.8?  1.0?

* How will we ensure (procedurally or otherwise) that each version of 
Chandler will successfully upgrade from older versions?

* What support will we provide for parcel developers to ensure that *their* 
schemas upgrade safely, as well as the Chandler "out of the box" schemas?

* When an upgrade has to be reverted, what guarantees should we give the 
user for being able to revert safely without losing data?

While I'm taking responsibility for driving the technical implementation of 
schema evolution features in 0.7, there is a lot here that is more on the 
organizational commitment level.  Really, the technical implementation of 
schema evolution features is only the tip of the iceberg of effort 
here.  Tracking changes, writing upgrade code, and testing will be the 
biggest resource investments here.

If you want more technical information about the issues involved at the 
implementation level, a good place to start is this post I wrote in October:

http://lists.osafoundation.org/pipermail/dev/2005-October/003994.html

You will probably want to skip most of the first half (which deals with 
developer-only code-upgrading ideas) and go straight to the section 
entitled "Updating Chandler Schema".  The text that follows addresses 
details of various aspects of upgrading Chandler schemas, given the 
existing repository and schema APIs.