[Dev] ZODB is not a Storage Technology (Re: other formats )

[Chris McDonough]

> >From a design standpoint it probably does. I'm not a ZODB expert by any
> means, but it is my understanding that Python pickles are what ZODB stores.
> I'm not sure how this changes as you try and use a Berkley DB as a backing
> store, but it does bring up a big design point. That is, should the file
> format be portable across operating systems? I would say yes, but that means
> that you need the same Berkley DB format to be supported on
> Win9x/ME/NT/2K/XP, Linux, Mac OS X, and whatever else you want to support.
> Does that portable Berkley DB code and format exist, I don't think so? If

Not sure about bsddb files, but FileStorage files are portable across
platforms (even ones with different endianness).

> you just use pickles, the data will be portable, but you need to have a safe
> unpickler to protect against trojan classes in the pickle; the method called
> when the data is unpickle is the big problem AFAIK. There is also the
> possibility that the pickle format could change with a future version of
> Python.
> 
> http://www.python.org/doc/current/lib/node59.html

Possible, but not very likely without a big stink.  The pickle format
hasn't changed in any meaningful way in at least the last four years at
least.

> I seem to remember some discussion a long time ago about data corruption
> when using Berkley DB as a backing store, but all the references on the
> ZODB-dev list seem old, so this might be a non-issue. Someone working on the
> data storage part of Chandler might want to do some searches with Google
> groups in comp.lang.python or some of the ZODB and Zope mailing lists.

bsddbStorage is less mature and well-tested than FileStorage.

> Finally, this is another point that should probably be discussed on the
> design list, but if you want other languages such as Java to be able to read
> the file format used by Chandler, ZODB could be problematic.

Yes.  Unfortunately, there's not much to mitigate this except some sort
of RPC and a server.

- C