[Cosmo] preserving data during cosmo 0.3 upgrade

[Brian Moseley]

as part of the proposed cosmo 0.3 release, we'd like to guarantee that
upgrading from 0.2 will not require the existing data to be wiped.

since the repository schema has changed greatly in 0.3, we'll need to
provide a tool that can convert an existing repository from the 0.2
format to the 0.3 format. such a tool could be be run while the server
is offline or as a component of the application server that executes
at startup time. i argue that the offline approach is the right
choice, since the tool would be more easily scriptable by deployers
and won't require any extra configuration of the server environment.

complicating factors include:

 * user account and security info are stored in an embedded hsql db in
0.2 but are stored in the repository in 0.3

 * the repository schema has changed significantly in 0.3 - the dav
and caldav node types have become mixins, and some property
definitions have changed - jackrabbit doesn't have a good mechanism
for changing existing schema definitions like sql's ALTER TABLE
command

here is my proposal for the offline upgrade tool:

 * it will be written in java since it needs to use the jcr api to
access the repositories

 * it will be packaged together with all dependencies and run via shell script

 * it will not attempt to convert a 0.2 repository "in place" but
rather will copy the data out the 0.2 repository and user db into a
new, blank 0.3 repository

 * it will accept as input (via command line options) the locations of
the data directories and config files for the source (0.2) repository
and destination (0.3) repository

 * at the end of the process, the 0.2 repository and user db will have
exactly the same data as they did at the beginning

the upgrade process will be broken into the following tasks:

 1) get the root user's details from the 0.2 user db and update the
out-of-the-box root user's details in the 0.3 repository

 2) get the rest of the user details from the 0.2 user db and add a
user+homedir node for each into the 0.3 repository

 3) iterate through the homedir nodes in the 0.2 repository, copying
all content for each (calendars, events, collections, resources,
tickets) to the corresponding homedir node in the 0.3 repository

since passwords are stored in an encrypted format in the 0.2 user db,
we'll have to extend the 0.3 internal apis to allow the creation of a
user with an encrypted password (the existing api only allows a user
to be created with a plaintext password).

in the long term i think we'll want an upgrade tool that can convert
from an arbitrary older version, so that one could upgrade a 0.2
server to 0.6 with the same tool rather than requiring 4 separate
tools. this implies an upgrade framework of some sort. since we really
want to release 0.3 in two weeks or less, i think we should punt on
this and then revisit it as a primary tenet for a later release.

i estimate that building the upgrade tool as proposed, testing on
copies of the foxcloud and/or cosmo-demo repositories, and fixing
found bugs will take 3-4 days.

thoughts on the proposal?