[Commits] (vajda) All accumulated repository code changes since 3 weeks before the 0.3 code

commits at osafoundation.org commits at osafoundation.org
Mon Mar 8 14:57:29 PST 2004


Commit by: vajda
Modified files:
osaf/chandler/Chandler/repository/__hardhat__.py 1.2 1.3
osaf/chandler/Chandler/repository/item/Item.py 1.109 1.110
osaf/chandler/Chandler/repository/item/ItemRef.py 1.60 1.61
osaf/chandler/Chandler/repository/persistence/DBContainer.py 1.6 1.7
osaf/chandler/Chandler/repository/persistence/Repository.py 1.67 1.68
osaf/chandler/Chandler/repository/persistence/XMLRepository.py 1.63 1.64
osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py 1.29 1.30
osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py 1.13 1.14
osaf/chandler/Chandler/repository/tests/data/.cvsignore 1.1 1.2
osaf/chandler/Chandler/repository/util/LinkedMap.py 1.11 1.12
osaf/chandler/Chandler/repository/util/Path.py 1.7 1.8
osaf/chandler/Chandler/repository/util/SAX.py 1.1 1.2
osaf/chandler/Chandler/repository/util/Streams.py 1.9 1.10
osaf/chandler/Chandler/repository/util/ThreadLocal.py 1.1 1.2
osaf/chandler/Chandler/repository/util/UUID.py 1.10 1.11

Log message:
All accumulated repository code changes since 3 weeks before the 0.3 code
freeze:
   - implemented simple case (literal->other) of item version merging
   - fixed subtle itemref unloading bug
   - fixed bug in check() causing it to not return False on failure
   - implemented simple case (other->literal) of item version merging
   - implemented simple case (ref->other) of item version merging
   - implemented simple case (other->ref) of item version merging
   - resolved bug 804
   - epydoc'ed all public methods in Item.py
   - fixed bug 1298
   - added repository cache pruning
   - reworked repository deadlock recovery code
   - fixed bug 1338
   - epydoc'ed UUID.py and Path.py


ViewCVS links:
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/__hardhat__.py.diff?r1=text&tr1=1.2&r2=text&tr2=1.3
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/item/Item.py.diff?r1=text&tr1=1.109&r2=text&tr2=1.110
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/item/ItemRef.py.diff?r1=text&tr1=1.60&r2=text&tr2=1.61
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/persistence/DBContainer.py.diff?r1=text&tr1=1.6&r2=text&tr2=1.7
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/persistence/Repository.py.diff?r1=text&tr1=1.67&r2=text&tr2=1.68
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/persistence/XMLRepository.py.diff?r1=text&tr1=1.63&r2=text&tr2=1.64
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py.diff?r1=text&tr1=1.29&r2=text&tr2=1.30
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py.diff?r1=text&tr1=1.13&r2=text&tr2=1.14
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/tests/data/.cvsignore.diff?r1=text&tr1=1.1&r2=text&tr2=1.2
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/util/LinkedMap.py.diff?r1=text&tr1=1.11&r2=text&tr2=1.12
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/util/Path.py.diff?r1=text&tr1=1.7&r2=text&tr2=1.8
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/util/SAX.py.diff?r1=text&tr1=1.1&r2=text&tr2=1.2
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/util/Streams.py.diff?r1=text&tr1=1.9&r2=text&tr2=1.10
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/util/ThreadLocal.py.diff?r1=text&tr1=1.1&r2=text&tr2=1.2
http://cvs.osafoundation.org/index.cgi/osaf/chandler/Chandler/repository/util/UUID.py.diff?r1=text&tr1=1.10&r2=text&tr2=1.11

Index: osaf/chandler/Chandler/repository/tests/data/.cvsignore
diff -u osaf/chandler/Chandler/repository/tests/data/.cvsignore:1.1 osaf/chandler/Chandler/repository/tests/data/.cvsignore:1.2
--- osaf/chandler/Chandler/repository/tests/data/.cvsignore:1.1	Tue Jan  6 19:55:48 2004
+++ osaf/chandler/Chandler/repository/tests/data/.cvsignore	Mon Mar  8 14:56:55 2004
@@ -1 +1,2 @@
-rssfeeds
+rssfeeds 
+rssfeeds.syndic8

Index: osaf/chandler/Chandler/repository/persistence/Repository.py
diff -u osaf/chandler/Chandler/repository/persistence/Repository.py:1.67 osaf/chandler/Chandler/repository/persistence/Repository.py:1.68
--- osaf/chandler/Chandler/repository/persistence/Repository.py:1.67	Fri Feb  6 13:01:05 2004
+++ osaf/chandler/Chandler/repository/persistence/Repository.py	Mon Mar  8 14:56:53 2004
@@ -1,10 +1,10 @@
 
-__revision__  = "$Revision: 1.67 $"
-__date__      = "$Date: 2004/02/06 21:01:05 $"
+__revision__  = "$Revision: 1.68 $"
+__date__      = "$Date: 2004/03/08 22:56:53 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
-import sys, os, os.path, threading, logging
+import sys, os, os.path, threading, logging, heapq
 import libxml2
 
 from repository.util.UUID import UUID
@@ -71,6 +71,7 @@
             self.logger.setLevel(logging.DEBUG)
         else:
             self.logger.setLevel(logging.INFO)
+
         if '-stderr' in sys.argv or not self.logger.root.handlers:
             if not self.logger.handlers:
                 self.logger.addHandler(logging.StreamHandler())
@@ -86,6 +87,9 @@
     def close(self, purge=False):
         pass
 
+    def prune(self, size):
+        self.view.prune(size)
+
     def closeView(self, purge=False):
         self.view.close()
 
@@ -253,11 +257,14 @@
         self._roots.clear()
         self._deletedRegistry.clear()
         self._childrenRegistry.clear()
-
         self._status &= ~RepositoryView.OPEN
 
         self.repository.store.detachView(self)
 
+    def prune(self, size):
+
+        pass
+
     def isOpen(self):
 
         return ((self._status & RepositoryView.OPEN) != 0 and
@@ -802,6 +809,19 @@
 
         return None
 
+    def prune(self, size):
+
+        registry = self._registry
+
+        if len(registry) > size * 1.1:
+            heap = [(item._access, item._uuid)
+                    for item in registry.itervalues()
+                    if not item._status & item.SCHEMA]
+            heapq.heapify(heap)
+            count = len(heap) - int(size * 0.9)
+            self.logger.info('pruning %d items', count)
+            for i in xrange(count):
+                registry[heapq.heappop(heap)[1]]._unloadItem()
 
 
 class RepositoryNotifications(dict):

Index: osaf/chandler/Chandler/repository/util/SAX.py
diff -u osaf/chandler/Chandler/repository/util/SAX.py:1.1 osaf/chandler/Chandler/repository/util/SAX.py:1.2
--- osaf/chandler/Chandler/repository/util/SAX.py:1.1	Fri Dec 12 22:55:32 2003
+++ osaf/chandler/Chandler/repository/util/SAX.py	Mon Mar  8 14:56:57 2004
@@ -1,12 +1,12 @@
 
-__revision__  = "$Revision: 1.1 $"
-__date__      = "$Date: 2003/12/13 06:55:32 $"
+__revision__  = "$Revision: 1.2 $"
+__date__      = "$Date: 2004/03/08 22:56:57 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
 
 from libxml2mod import xmlEncodeSpecialChars as escape
-from libxml2 import SAXCallback
+from libxml2 import SAXCallback, createPushParser
 
 
 class ContentHandler(SAXCallback):
@@ -23,6 +23,10 @@
         self.out = out
         self.encoding = encoding
 
+    def write(self, data):
+
+        self.out.write(data)
+
     def startDocument(self):
 
         self.out.write('<?xml version="1.0" encoding="%s"?>' %(self.encoding))
@@ -68,3 +72,76 @@
         self.out.write('<![CDATA[')
         self.out.write(data)
         self.out.write(']]>')
+
+
+class XMLFilter(ContentHandler):
+
+    def __init__(self, generator, *tags):
+
+        self.generator = generator
+        self.tags = tags
+        self.foundTag = 0
+        self.cdata = False
+
+    def output(self):
+
+        raise NotImplementedError, 'XMLFilter.output'
+
+    def parse(self, xml):
+
+        createPushParser(self, xml, len(xml), 'filter').parseChunk('', 0, 1)
+        
+    def endDocument(self):
+
+        if self.cdata:
+            self.generator.write(']]>')
+            self.cdata = False
+
+    def startElement(self, tag, attrs):
+
+        if self.cdata:
+            self.generator.write(']]>')
+            self.cdata = False
+
+        if tag in self.tags:
+            self.foundTag += 1
+        if self.output():
+            self.generator.startElement(tag, attrs)
+
+    def endElement(self, tag):
+
+        if self.output():
+            if self.cdata:
+                self.generator.write(']]>')
+                self.cdata = False
+            self.generator.endElement(tag)
+        if tag in self.tags:
+            self.foundTag -= 1
+
+    def characters(self, data):
+
+        if self.output():
+            self.generator.characters(data)
+
+    def cdataBlock(self, data):
+        
+        if self.output():
+            if not self.cdata:
+                self.generator.write('<![CDATA[')
+                self.cdata = True
+            self.generator.write(data)
+
+class XMLOffFilter(XMLFilter):
+
+    def output(self):
+        return not self.foundTag
+
+class XMLOnFilter(XMLFilter):
+
+    def output(self):
+        return self.foundTag
+
+class XMLThruFilter(XMLFilter):
+
+    def output(self):
+        return True

Index: osaf/chandler/Chandler/repository/util/Path.py
diff -u osaf/chandler/Chandler/repository/util/Path.py:1.7 osaf/chandler/Chandler/repository/util/Path.py:1.8
--- osaf/chandler/Chandler/repository/util/Path.py:1.7	Fri Jan 16 12:42:45 2004
+++ osaf/chandler/Chandler/repository/util/Path.py	Mon Mar  8 14:56:57 2004
@@ -1,44 +1,29 @@
 
-__revision__  = "$Revision: 1.7 $"
-__date__      = "$Date: 2004/01/16 20:42:45 $"
+__revision__  = "$Revision: 1.8 $"
+__date__      = "$Date: 2004/03/08 22:56:57 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
 
 class Path(object):
-    '''A path to an item in a repository.
+    """
+    A path to an item in a repository.
 
-    A path can be absolute to a repository in which case it starts with //.
-    A path can be absolute to a repository root in which case it start with /.
-    A path is relative to an item when it doesn't begin with '/' and is used in
-    conjunction with an item, as in an item's find method.'''
+    A path can be absolute to a repository; it then starts with C{//}.
+    A path can be absolute to a repository root; it then starts with C{/}.
+    A path can be relative to an item when it doesn't start with C{/}.
+    """
     
     def __init__(self, *args):
-        '''Construct a path.
+        """
+        Construct a path.
 
-        Any number of arguments are combined to for a list of names, a path.
-        Individual Arguments are split along '/' characters allowing for paths
-        to be constructed from path strings.
-        Ending '/' are stripped.
-        A path can be used as an iterator over its constituent names.'''
+        See L{set} method for more details on path construction.
+        A path can be used as an iterator over its constituent names.
+        """
         
         super(Path, self).__init__()
-        
-        self._names = []
-
-        for arg in args:
-            if arg.startswith('//'):
-                self._names.append('//')
-                arg = arg[2:]
-            elif arg[0] == '/':
-                self._names.append('/')
-                arg = arg[1:]
-
-            if arg.endswith('/'):
-                arg = arg[:-1]
-
-            if not arg == '':
-                self._names.extend(arg.split('/'))
+        self.set(*args)
 
     def __repr__(self):
 
@@ -70,24 +55,77 @@
         return self._names.__iter__()
 
     def set(self, *args):
+        """
+        Any number of arguments are combined to form a list of names, a path.
+        Individual Arguments are split along C{/} characters allowing for paths
+        to be constructed from path strings.
+        Ending C{/} characters are stripped.
+        """
+        
+        self._names = []
+        for arg in args:
+            if isinstance(arg, Path):
+                self._names.extend(arg._names)
+                continue
+            
+            if arg.startswith('//'):
+                self._names.append('//')
+                arg = arg[2:]
+            elif arg[0] == '/':
+                self._names.append('/')
+                arg = arg[1:]
 
-        self._names[:] = args
+            if arg.endswith('/'):
+                arg = arg[:-1]
+
+            if not arg == '':
+                self._names.extend(arg.split('/'))
 
     def append(self, name):
-        'Extend this path appending name it.'
+        """
+        Add a name to this path.
+
+        C{name} should be a string without C{/} characters.
+
+        @param name: the name to add
+        @type name: a string
+        """
         
         self._names.append(name)
 
     def extend(self, path):
-        'Concatenate two paths. Leading '/' are not stripped.'
+        """
+        Concatenate two paths.
+
+        This path is augmented with C{path}'s names.
+        Leading '/' are not stripped.
+
+        @param path: the path to extend this path with
+        @type path: a C{Path} instance
+        """
 
         self._names.extend(path._names)
 
-    def pop(self, i=-1):
+    def pop(self, index=-1):
+        """
+        Remove a name from this path.
+
+        @param index: the optional index of the name to rename to remove; by
+        default, the last name is removed.
+        @type index: integer
+        @return: the name removed
+        """
 
-        return self._names.pop(i)
+        return self._names.pop(index)
 
     def normalize(self):
+        """
+        Create a normalized path from this path.
+
+        Redundant C{..} and C{.} names are removed.
+
+        @return: a new path instance
+        """
 
         names = []
         for name in self._names:

Index: osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py
diff -u osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py:1.13 osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py:1.14
--- osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py:1.13	Wed Feb 11 23:10:25 2004
+++ osaf/chandler/Chandler/repository/tests/TestPerfWithRSS.py	Mon Mar  8 14:56:54 2004
@@ -2,14 +2,15 @@
 Simple Performance tests for Chandler repository
 """
 
-__revision__  = "$Revision: 1.13 $"
-__date__      = "$Date: 2004/02/12 07:10:25 $"
+__revision__  = "$Revision: 1.14 $"
+__date__      = "$Date: 2004/03/08 22:56:54 $"
 __copyright__ = "Copyright (c) 2003 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
 import os, os.path, sys, unittest
 
 from bsddb.db import DBNoSuchFileError
+from repository.util.Path import Path
 from repository.item.Query import KindQuery
 from repository.persistence.XMLRepository import XMLRepository
 import repository.parcel.LoadParcels as LoadParcels
@@ -21,7 +22,7 @@
 import feedparser
 
 # get all the RSS files in RSS_HOME (repository/tests/data/rssfeeds)
-# You can obtain the files from http://aloha.osafoundation.org/~twl/rssfeeds.tar.gz
+# You can obtain the files from http://aloha.osafoundation.org/~twl/RSSdata/rssfeeds.syndic8.tar.bz2
 RSS_HOME=os.path.join(_chandlerDir,'Chandler','repository','tests','data','rssfeeds/')
 if os.path.exists(RSS_HOME):
     _rssfiles = os.listdir(RSS_HOME)
@@ -29,9 +30,9 @@
     _rssfiles = []
 
 # make them file URL's
-_defaultBlogs = [ "file://"+RSS_HOME+f for f in _rssfiles ]
+_defaultBlogs = [ "%s%s%s" %("file://", RSS_HOME, f) for f in _rssfiles ]
 
-BASE_PATH = '//parcels/OSAF/examples/zaobao'
+BASE_PATH = Path('//parcels/OSAF/examples/zaobao')
 
 class TestPerfWithRSS(unittest.TestCase):
     """ Simple performance tests """
@@ -59,19 +60,23 @@
         repository = self.rep
 
         itemCount = 0
+        feedCount = 0
         feeds = self.__getFeeds()
 
         if feeds == []:
-            self.rep.logger.info("got 0 feeds")
+            self.rep.logger.info("got no feeds")
             print "If you haven't installed the feed data, you can retreive it from"
             print "http://aloha.osafoundation.org/~twl"
             print "select a tarball, download it, and unpack it in repository/tests/data"
             print "The data will be in a new directory called rssfeeds"
             print "You can now run the tests"
         else:
-            self.rep.logger.info('got %d feeds' %(len(feeds)))
+            self.rep.logger.info('committing %d feeds', len(feeds))
+            self.rep.commit()
+            self.rep.logger.info('committed %d feeds', len(feeds))
 
-        for feed in feeds[:]:
+        for feed in feeds:
+            feed = self.rep.find(feed)
             self.rep.logger.debug(feed.url)
             etag = feed.getAttributeValue('etag', default=None)
             lastModified = feed.getAttributeValue('lastModified', default=None)
@@ -82,44 +87,48 @@
             try:
                 data = feedparser.parse(feed.url, etag, modified)
                 itemCount += len(data['items'])
+                feedCount += 1
                 feed.Update(data)
                 if commitInsideLoop:
+                    self.rep.logger.info('%0.5d committing %s, %0.6d',
+                                         feedCount, feed.url, itemCount)
                     repository.commit()
-            except Exception, e:
-                self.rep.logger.error("%s in %s" % (e,feed.url))
+            except Exception:
+                self.rep.logger.exception('While processing %s', feed.url)
+                self.rep.cancel()
 
         try:
 #            profiler = hotshot.Profile('/tmp/TestPerfWithRss.stressTest.hotshot')
 #            profiler.runcall(repository.commit)
 #            profiler.close()
             repository.commit()
-        except Exception, e:
-            print e
-            self.rep.logger.error("Final commit:")
+        except Exception:
+            self.rep.logger.exception("Final commit:")
             self.fail()
-        self.rep.logger.info("Processed %d items" % itemCount)
+
+        self.rep.logger.info('Processed %d items', itemCount)
         self.assert_(True)
         
     def __getFeeds(self):
         """Return a list of channel items"""
         repository = self.rep
-        chanKind = repository.find(BASE_PATH + '/RSSChannel')
+        chanKind = repository.find(Path(BASE_PATH, 'RSSChannel'))
 
         feeds = []
         parent = repository.find(BASE_PATH)
 
         for url in _defaultBlogs:
             urlhash = str(hash(url))
-            item = repository.find(BASE_PATH + '/' + urlhash)
+            item = repository.find(Path(BASE_PATH, urlhash))
             if not item:
                 item = chanKind.newItem(urlhash, parent)
                 item.url = url
-            feeds.append(item)
+            feeds.append(item.getUUID())
 
         return feeds
 
-    def testCommitAtEnd(self):
-        self._stressTest()
+#    def testCommitAtEnd(self):
+#        self._stressTest()
 
     def testCommitInsideLoop(self):
         self._stressTest(True)
@@ -129,13 +138,13 @@
         for i in items:
             assert(i.getItemName() is not None)
 
-    def testReadBackRSS(self):
-        self._stressTest()
-        self.rep.close()
-        self.rep = XMLRepository(os.path.join(self.testdir, '__repository__'))
-        self.rep.open()
-        RSSItem = self.rep.find('//parcels/OSAF/examples/zaobao/RSSItem')
-        self._readItems(RSSItem.kind)
+#    def testReadBackRSS(self):
+#        self._stressTest()
+#        self.rep.close()
+#        self.rep = XMLRepository(os.path.join(self.testdir, '__repository__'))
+#        self.rep.open()
+#        RSSItem = self.rep.find('//parcels/OSAF/examples/zaobao/RSSItem')
+#        self._readItems(RSSItem.kind)
 #        profiler = hotshot.Profile('/tmp/TestPerfWithRss.readBack.hotshot')
 #        profiler.runcall(TestPerfWithRSS._readItems, self, RSSItem.kind)
 #        profiler.close()

Index: osaf/chandler/Chandler/repository/item/ItemRef.py
diff -u osaf/chandler/Chandler/repository/item/ItemRef.py:1.60 osaf/chandler/Chandler/repository/item/ItemRef.py:1.61
--- osaf/chandler/Chandler/repository/item/ItemRef.py:1.60	Sat Feb 14 00:18:54 2004
+++ osaf/chandler/Chandler/repository/item/ItemRef.py	Mon Mar  8 14:56:52 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.60 $"
-__date__      = "$Date: 2004/02/14 08:18:54 $"
+__revision__  = "$Revision: 1.61 $"
+__date__      = "$Date: 2004/03/08 22:56:52 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -87,7 +87,7 @@
         else:
             other._removeRef(otherName)
 
-        other.setDirty(attribute=otherName)
+        other.setDirty(attribute=otherName, dirty=item.RDIRTY)
 
     def reattach(self, item, name, old, new, otherName):
 
@@ -96,10 +96,14 @@
 
     def _unload(self, item):
 
-        if item is self.getItem():
-            self._item = UUIDStub(self.getOther(), item)
-        elif item is self.getOther():
-            self._other = UUIDStub(self.getItem(), item)
+        # using direct compares instead of accessors to avoid re-loading
+        
+        if item is self._item:
+            if self._other._isItem():
+                self._item = UUIDStub(self._other, item)
+        elif item is self._other:
+            if self._item._isItem():
+                self._other = UUIDStub(self._item, item)
         else:
             raise ValueError, "%s doesn't reference %s" %(self, item)
 
@@ -121,13 +125,16 @@
             other = self.other(item)
         except DanglingRefError, e:
             logger.error('DanglingRefError: %s', e)
+            return False
         except ValueError, e:
             logger.error('ValueError: %s', e)
+            return False
         else:
             if other.isStale():
                 logger.error('Found stale item %s at %s of kind %s',
                              other, other.getItemPath(),
                              other._kind.getItemPath())
+                return False
             else:
                 otherName = item.getAttributeAspect(name, 'otherName',
                                                     default=None)
@@ -137,8 +144,9 @@
                 if otherOtherName != name:
                     logger.error("otherName for attribute %s.%s, %s, does not match otherName for attribute %s.%s, %s",
                                  item._kind.getItemPath(), name, otherName,
-                                 other._item._kind.getItemPath(), otherName,
+                                 other._kind.getItemPath(), otherName,
                                  otherOtherName)
+                    return False
 
         return True
 
@@ -258,6 +266,10 @@
 
         return other
 
+    def _isItem(self):
+
+        return False
+
 
 class UUIDStub(Stub):
 
@@ -280,6 +292,10 @@
 
         return other
     
+    def _isItem(self):
+
+        return False
+    
 
 class RefArgs(object):
     'A wrapper around arguments necessary to make and store an ItemRef'
@@ -658,7 +674,7 @@
 
     def _changeRef(self, key):
 
-        self._item.setDirty(attribute=self._name)
+        self._item.setDirty(attribute=self._name, dirty=self._item.RDIRTY)
 
     def _getRef(self, key, load=True):
 
@@ -785,14 +801,17 @@
             except DanglingRefError, e:
                 logger.error('DanglingRefError on %s.%s: %s',
                              self._item.getItemPath(), self._name, e)
+                return False
             except KeyError, e:
                 logger.error('KeyError on %s.%s: %s',
                              self._item.getItemPath(), self._name, e)
+                return False
             else:
                 if other.isStale():
                     logger.error('Found stale item %s at %s of kind %s',
                                  other, other.getItemPath(),
                                  other._kind.getItemPath())
+                    return False
                 else:
                     name = other.getAttributeAspect(self._otherName, 'otherName', default=None)
                     if name != self._name:
@@ -801,6 +820,7 @@
                                      self._otherName, name,
                                      self._item._kind.getItemPath(),
                                      self._name, self._otherName)
+                        return False
                         
             l -= 1
             key = self.nextKey(key)
@@ -808,6 +828,7 @@
         if l != 0:
             logger.error("Iterator on %s.%s doesn't match length (%d left for %d total)",
                          self._item.getItemPath(), self._name, l, len(self))
+            return False
 
         return True
 

Index: osaf/chandler/Chandler/repository/persistence/DBContainer.py
diff -u osaf/chandler/Chandler/repository/persistence/DBContainer.py:1.6 osaf/chandler/Chandler/repository/persistence/DBContainer.py:1.7
--- osaf/chandler/Chandler/repository/persistence/DBContainer.py:1.6	Sun Feb  1 19:08:34 2004
+++ osaf/chandler/Chandler/repository/persistence/DBContainer.py	Mon Mar  8 14:56:53 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.6 $"
-__date__      = "$Date: 2004/02/02 03:08:34 $"
+__revision__  = "$Revision: 1.7 $"
+__date__      = "$Date: 2004/03/08 22:56:53 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -70,7 +70,6 @@
         self.store.repository.logger.info('detected deadlock: %d', n)
         
 
-
 class RefContainer(DBContainer):
 
     def loadRef(self, version, key, cursorKey):
@@ -88,41 +87,47 @@
                 except DBNotFoundError:
                     return None
                 except DBLockDeadlockError:
-                    self._logDL(1)
-                    continue
+                    if txnStarted:
+                        self._logDL(1)
+                        continue
+                    else:
+                        raise
 
-                while value is not None and value[0].startswith(cursorKey):
-                    refVer = ~unpack('>l', value[0][48:52])[0]
+                try:
+                    while value is not None and value[0].startswith(cursorKey):
+                        refVer = ~unpack('>l', value[0][48:52])[0]
                 
-                    if refVer <= version:
-                        value = value[1]
-                        offset = 0
+                        if refVer <= version:
+                            value = value[1]
+                            offset = 0
 
-                        len, uuid = self._readValue(value, offset)
-                        offset += len
+                            len, uuid = self._readValue(value, offset)
+                            offset += len
                     
-                        if uuid is None:
-                            return None
+                            if uuid is None:
+                                return None
 
-                        else:
-                            len, previous = self._readValue(value, offset)
-                            offset += len
+                            else:
+                                len, previous = self._readValue(value, offset)
+                                offset += len
 
-                            len, next = self._readValue(value, offset)
-                            offset += len
+                                len, next = self._readValue(value, offset)
+                                offset += len
 
-                            len, alias = self._readValue(value, offset)
-                            offset += len
+                                len, alias = self._readValue(value, offset)
+                                offset += len
 
-                            return (key, uuid, previous, next, alias)
+                                return (key, uuid, previous, next, alias)
+
+                        else:
+                            value = cursor.next()
 
+                except DBLockDeadlockError:
+                    if txnStarted:
+                        self._logDL(2)
+                        continue
                     else:
-                        while True:
-                            try:
-                                value = cursor.next()
-                                break
-                            except DBLockDeadlockError:
-                                self._logDL(2)
+                        raise
 
                 return None
 
@@ -190,74 +195,95 @@
 
     def getDocVersion(self, uuid, version=0):
 
-        cursor = None
-        txnStarted = False
-        try:
-            txnStarted = self.store._startTransaction()
-            cursor = self.cursor()
-                
-            try:
-                key = uuid._uuid
-                value = cursor.set_range(key, flags=DB_DIRTY_READ)
-            except DBNotFoundError:
-                return None
+        while True:
+            txnStarted = False
+            cursor = None
 
-            while True:
-                if value[0].startswith(key):
-                    docVersion = ~unpack('>l', value[0][16:20])[0]
-                    if version == 0 or docVersion <= version:
-                        return docVersion
-                else:
+            try:
+                txnStarted = self.store._startTransaction()
+                cursor = self.cursor()
+                
+                try:
+                    key = uuid._uuid
+                    value = cursor.set_range(key, flags=DB_DIRTY_READ)
+                except DBNotFoundError:
                     return None
+                except DBLockDeadlockError:
+                    if txnStarted:
+                        self._logDL(6)
+                        continue
+                    else:
+                        raise
+
+                try:
+                    while True:
+                        if value[0].startswith(key):
+                            docVersion = ~unpack('>l', value[0][16:20])[0]
+                            if version == 0 or docVersion <= version:
+                                return docVersion
+                        else:
+                            return None
 
-                while True:
-                    try:
                         value = cursor.next()
-                        break
-                    except DBLockDeadlockError:
+
+                except DBLockDeadlockError:
+                    if txnStarted:
                         self._logDL(4)
+                        continue
+                    else:
+                        raise
 
-        finally:
-            if cursor:
-                cursor.close()
-            if txnStarted:
-                self.store._abortTransaction()
+            finally:
+                if cursor:
+                    cursor.close()
+                if txnStarted:
+                    self.store._abortTransaction()
 
     def getDocId(self, uuid, version):
 
-        cursor = None
-        txnStarted = False
-        try:
-            txnStarted = self.store._startTransaction()
-            cursor = self.cursor()
+        while True:
+            txnStarted = False
+            cursor = None
 
             try:
-                key = uuid._uuid
-                value = cursor.set_range(key, flags=DB_DIRTY_READ)
-            except DBNotFoundError:
-                return None
+                txnStarted = self.store._startTransaction()
+                cursor = self.cursor()
 
-            else:
-                while value is not None and value[0].startswith(key):
-                    docVersion = ~unpack('>l', value[0][16:20])[0]
+                try:
+                    key = uuid._uuid
+                    value = cursor.set_range(key, flags=DB_DIRTY_READ)
+                except DBNotFoundError:
+                    return None
+                except DBLockDeadlockError:
+                    if txnStarted:
+                        self._logDL(7)
+                        continue
+                    else:
+                        raise
 
-                    if docVersion <= version:
-                        return unpack('>l', value[1])[0]
+                try:
+                    while value is not None and value[0].startswith(key):
+                        docVersion = ~unpack('>l', value[0][16:20])[0]
+
+                        if docVersion <= version:
+                            return unpack('>l', value[1])[0]
                         
-                    while True:
-                        try:
-                            value = cursor.next()
-                            break
-                        except DBLockDeadlockError:
-                            self._logDL(5)
+                        value = cursor.next()
+
+                except DBLockDeadlockError:
+                    if txnStarted:
+                        self._logDL(5)
+                        continue
+                    else:
+                        raise
                         
                 return None
 
-        finally:
-            if cursor:
-                cursor.close()
-            if txnStarted:
-                self.store._abortTransaction()
+            finally:
+                if cursor:
+                    cursor.close()
+                if txnStarted:
+                    self.store._abortTransaction()
 
     def deleteVersion(self, uuid):
 
@@ -278,15 +304,15 @@
     # has to run within the commit transaction
     def apply(self, fn, oldVersion, newVersion):
 
-        cursor = self.cursor()
-
         try:
-            value = cursor.set_range(pack('>l', oldVersion + 1),
-                                     flags=DB_DIRTY_READ)
-        except DBNotFoundError:
-            return
+            cursor = self.cursor()
+
+            try:
+                value = cursor.set_range(pack('>l', oldVersion + 1),
+                                         flags=DB_DIRTY_READ)
+            except DBNotFoundError:
+                return
 
-        try:
             while value is not None:
                 version, uuid = unpack('>l16s', value[0])
                 if version > newVersion:
@@ -301,12 +327,7 @@
 
                 fn(UUID(uuid), version, docId, status, parentId)
 
-                while True:
-                    try:
-                        value = cursor.next()
-                        break
-                    except DBLockDeadlockError:
-                        self._logDL(3)
-                        
+                value = cursor.next()
+
         finally:
             cursor.close()

Index: osaf/chandler/Chandler/repository/item/Item.py
diff -u osaf/chandler/Chandler/repository/item/Item.py:1.109 osaf/chandler/Chandler/repository/item/Item.py:1.110
--- osaf/chandler/Chandler/repository/item/Item.py:1.109	Tue Feb  3 19:37:50 2004
+++ osaf/chandler/Chandler/repository/item/Item.py	Mon Mar  8 14:56:52 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.109 $"
-__date__      = "$Date: 2004/02/04 03:37:50 $"
+__revision__  = "$Revision: 1.110 $"
+__date__      = "$Date: 2004/03/08 22:56:52 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -18,6 +18,7 @@
 from repository.util.Path import Path
 from repository.util.LinkedMap import LinkedMap
 from repository.util.SAX import XMLGenerator
+from repository.util.SAX import XMLOffFilter, XMLOnFilter, XMLThruFilter
 
 
 class Item(object):
@@ -47,6 +48,7 @@
 
         self._status = Item.NEW
         self._version = 0L
+        self._access = 0L
         self._uuid = UUID()
 
         self._values = Values(self)
@@ -67,6 +69,7 @@
         self._root = None
         self._status = 0
         self._version = kwds['version']
+        self._access = 0L
 
         kwds['values']._setItem(self)
         self._values = kwds['values']
@@ -97,6 +100,8 @@
           - C{optional status} is displayed when the item is stale or deleted
           - C{name} is the item's name
           - C{uuid} is the item's UUID
+
+        @return: a string representation of an item.
         """
 
         if self._status & Item.RAW:
@@ -117,6 +122,7 @@
         This method is called by python when looking up a Chandler attribute.
         @param name: the name of the attribute being accessed.
         @type name: a string
+        @return: an attribute value
         """
 
         return self.getAttributeValue(name)
@@ -129,15 +135,16 @@
         python attribute and dispatches to the relevant methods.
         @param name: the name of the attribute being set.
         @type name: a string
-        @param value: the value being set.
+        @param value: the value being set
         @type value: anything
+        @return: the value actually set.
         """
 
         if name[0] != '_':
-            if self._values.has_key(name):
+            if name in self._values:
                 return self.setAttributeValue(name, value,
                                               _attrDict=self._values)
-            elif self._references.has_key(name):
+            elif name in self._references:
                 return self.setAttributeValue(name, value,
                                               _attrDict=self._references)
             elif self._kind is not None and self._kind.hasAttribute(name):
@@ -153,11 +160,12 @@
         python attribute and dispatches to the relevant methods.
         @param name: the name of the attribute being cleared.
         @type name: a string
+        @return: C{None}
         """
 
-        if self._values.has_key(name):
+        if name in self._values:
             self.removeAttributeValue(name, _attrDict=self._values)
-        elif self._references.has_key(name):
+        elif name in self._references:
             self.removeAttributeValue(name, _attrDict=self._references)
         else:
             super(Item, self).__delattr__(name)
@@ -180,12 +188,13 @@
         """
         Tell whether an attribute as value set for the aspect.
 
-        See the L{getAttributeAspect <getAttributeAspect>} method for
-        more information on attribute aspects.
+        See the L{getAttributeAspect} method for more information on
+        attribute aspects.
         @param name: the name of the attribute being queried
         @type name: a string
         @param aspect: the name of the aspect being queried
         @type aspect: a string
+        @return: C{True} or C{False}
         """
 
         if self._kind is not None:
@@ -264,6 +273,7 @@
         @param kwds: optional keywords of which only C{default} is
         supported and used to return a default value for an aspect that has
         no set value for this attribute.
+        @return: a value
         """
 
         if self._kind is not None:
@@ -281,12 +291,13 @@
         assignment syntax is unnecessary.
         @param name: the name of the attribute.
         @type name: a string.
-        @param value: the value to set.
-        @type value: any type as specified by the attribute's C{type} aspect.
+        @param value: the value being set
+        @type value: anything compatible with the attribute's type
+        @return: the value actually set.
         """
 
         self.setDirty(attribute=name)
-
+        
         isItem = isinstance(value, Item)
         isRef = not isItem and (isinstance(value, ItemRef) or
                                 isinstance(value, RefDict))
@@ -390,19 +401,43 @@
         return value
 
     def getAttributeValue(self, name, _attrDict=None, **kwds):
-        """Return the named Chandler attribute value.
+        """
+        Return a Chandler attribute value.
 
-        If the attribute is not set then attempt to inherit a value if the
-        attribute's inheritFrom aspect is set, attempt to return the value
-        of the optional 'default' keyword passed to this method, attempt to
-        return the value of its defaultValue aspect if set, or finally raise 
-        AttributeError. 
-        Calling this method is only required when there is a name ambiguity
-        between a python and a Chandler attribute, a situation best avoided."""
+        Unless the optional keywords described below are used, calling this
+        method instead of using regular python attribute access syntax is
+        not necessary as python calls this method, via
+        L{__getattr__} when a non Chandler attribute of this name is not
+        found.
+
+        If the attribute has no value set then the following attempts are
+        made at infering one (in this order):
+            1. If the attribute has an C{initialValue} aspect set (see
+               L{getAttributeAspect} for more information on attribute
+               aspects) then this value is set for the attribute and
+               returned.
+            2. If the attribute has an C{inheritFrom} aspect set then the
+               value inherited along C{inheritFrom} is the value returned.
+            3. If the C{default} keyword is passed to this method then its
+               value is returned.
+            4. If the attribute has a C{defaultValue} aspect set then it is
+               returned. If this default value is a collection then it is
+               read-only.
+            5. And finally, if all of the above failed, an C{AttributeError}
+               is raised.
+
+        @param name: the name of the attribute
+        @type name: a string
+        @param kwds: an optional C{default} key/value pair
+        @type kwds: the value for the C{default} keyword can be of any type
+        @return: a value
+        """
 
         if self._status & Item.STALE:
             raise ValueError, "item is stale: %s" %(self)
 
+        self._access = Item._countAccess()
+
         try:
             if (_attrDict is self._values or
                 _attrDict is None and name in self._values):
@@ -447,7 +482,17 @@
                                                            name)
 
     def removeAttributeValue(self, name, _attrDict=None):
-        "Remove a Chandler attribute's value."
+        """
+        Remove a value for a Chandler attribute.
+
+        Calling this method instead of using python's C{del} operator is not
+        necessary as python calls this method, via
+        L{__delattr__}.
+
+        @param name: the name of the attribute
+        @type name: a string
+        @return: C{None}
+        """
 
         self.setDirty(attribute=name)
 
@@ -473,6 +518,19 @@
                 raise ValueError, value
 
     def hasChild(self, name, load=True):
+        """
+        Tell whether this item has a child of that name.
+
+        By setting the optional C{load} argument to C{False}, this method
+        can be restricted to only check among the currently loaded children
+        of this item.
+
+        @param name: the name of the child to verify
+        @type name: a string
+        @param load: whether to check only among loaded children
+        @type load: a boolean, C{True} by default
+        @return: C{True} or C{False}
+        """
 
         return ('_children' in self.__dict__ and
                 not ('_notChildren' in self.__dict__ and
@@ -480,14 +538,28 @@
                 self._children.has_key(name, load))
 
     def hasChildren(self):
+        """
+        Tell whether this item has any children.
+
+        @return: C{True} or C{False}
+        """
 
         return (self.__dict__.has_key('_children') and
                 self._children._firstKey is not None)
 
     def placeChild(self, child, after):
-        """Place a child after another one in this item's children collection.
+        """
+        Place a child after another one in this item's children collection.
 
-        To place a children in first position, pass None for after."""
+        To place a child in first position, pass C{None} for C{after}.
+        See also L{move} to change this item's parent.
+
+        @param child: the child item to place
+        @type child: an item
+        @param after: the sibling of C{child} to precede it
+        @type after: an item
+        @return: C{None}
+        """
 
         if not (child.getItemParent() is self):
             raise ValueError, '%s not a child of %s' %(child, self)
@@ -503,7 +575,15 @@
         self._children.place(key, afterKey)
 
     def dir(self, recursive=True):
-        'Print out a listing of each child under this item, recursively.'
+        """
+        Print out a listing of each child under this item.
+
+        By default, this method recurses down children of this item.
+
+        @param recursive: whether to recurse down the children or not
+        @type recursive: a boolean, C{True} by default
+        @return: C{None}
+        """
         
         for child in self:
             print child.getItemPath()
@@ -511,7 +591,20 @@
                 child.dir(True)
 
     def iterChildren(self, load=True):
+        """
+        Return a python generator used to iterate over the children of this
+        item.
 
+        This method is invoked by python's L{__iter__}, usually
+        from a C{for} loop construct. Optionally, this method can be used to
+        iterate over only the currently loaded children of this item.
+
+        @param load: C{True}, the default, to iterate over all the children
+        of this item. C{False} to iterate over only its currently loaded
+        children items.
+        @type load: boolean
+        """
+        
         if self._status & Item.STALE:
             raise ValueError, "item is stale: %s" %(self)
 
@@ -525,10 +618,17 @@
                 yield child
 
     def iterAttributes(self, valuesOnly=False, referencesOnly=False):
-        """Get a generator of (name, value) tuples for attributes of this item.
+        """
+        Return a generator of C{(name, value)} tuples for iterating over
+        Chandler attribute values of this item. 
 
-        By setting valuesOnly to True, no item references are returned.
-        By setting referencesOnly to True, only references are returned."""
+        @param valuesOnly: if C{True}, iterate over literal values
+        only. C{False} by default.
+        @type valuesOnly: boolean
+        @param referencesOnly: if C{True}, iterate over item reference
+        values only. C{False} by default.
+        @type referencesOnly: boolean
+        """
 
         if not referencesOnly:
             for attr in self._values.iteritems():
@@ -542,6 +642,23 @@
                     yield ref
 
     def check(self, recursive=False):
+        """
+        Run consistency checks on this item.
+
+        Currently, this method verifies that:
+            - each literal attribute value is of a type compatible with its
+              C{type} aspect (see L{getAttributeAspect}).
+            - each attribute value is a of a cardinality compatible with its
+              C{cardinality} aspect.
+            - each reference attribute value's endpoints are compatible with
+              each other, that is their C{otherName} aspects match the
+              other's name.
+
+        @param recursive: if C{True}, check this item and its children
+        recursively. If C{False}, the default, check only this item.
+        @return: C{True} if no errors were found, C{False} otherwise. Errors
+        are logged in the Chandler execution log.
+        """
 
         logger = self.getRepository().logger
         result = True
@@ -599,13 +716,28 @@
         return result
         
     def getValue(self, attribute, key, default=None, _attrDict=None):
-        'Get a value from a multi-valued attribute.'
+        """
+        Return a value from a Chandler collection attribute.
 
-        if _attrDict is None:
-            value = (self._values.get(attribute, Item.Nil) or
-                     self._references.get(attribute, Item.Nil))
-        else:
-            value = _attrDict.get(attribute, Item.Nil)
+        The collection is obtained using
+        L{getAttributeValue} and the return value is extracted by using the
+        C{key} argument. If the collection does not exist or there is no
+        value for C{key}, C{default} is returned, C{None} by default. Unless
+        this defaulting behavior is needed, there is no reason to use this
+        method instead of the regular python syntax for accessing instance
+        attributes and collection elements.
+
+        @param attribute: the name of the attribute
+        @type attribute: a string
+        @param key: the key into the collection
+        @type key: integer for lists, anything for dictionaries
+        @param default: an optional C{default} value, C{None} by default
+        @type default: anything
+        @return: a value
+        """
+
+        value = self.getAttributeValue(attribute, default=Item.Nil,
+                                       _attrDict=_attrDict)
             
         if value is Item.Nil:
             return default
@@ -622,13 +754,25 @@
         raise TypeError, "%s is not multi-valued" %(attribute)
 
     def setValue(self, attribute, value, key=None, alias=None, _attrDict=None):
-        """Set a value for a multi-valued attribute, for an optional key.
+        """
+        Set a value into a Chandler collection attribute.
 
-        When the cardinality of the attribute is 'list' and its type is a
-        literals, key must be an integer.
-        When the cardinality of the attribute is 'list' and its values are
-        references, key may be an integer or the refName of the item value
-        to set."""
+            - If the attribute doesn't yet have a value, the proper
+              collection is created for it.
+            - If the collection is a list and C{key} is an integer out of the
+              list's range, an exception is raised.
+            - If the attribute is not a collection attribute, the value is
+              simply set.
+        
+        @param attribute: the name of the attribute
+        @type attribute: a string
+        @param key: the key into the collection, not used when the
+        collection is a collection of item references
+        @type key: integer for lists, anything for dictionaries
+        @param value: the value to set
+        @type value: anything compatible with the attribute's type
+        @return: the collection that was changed or created
+        """
 
         self.setDirty(attribute=attribute)
 
@@ -686,7 +830,23 @@
         return attrValue
 
     def addValue(self, attribute, value, key=None, alias=None, _attrDict=None):
-        "Add a value for a multi-valued attribute for a given optional key."
+        """
+        Add a value to a Chandler collection attribute.
+
+            - If the attribute doesn't yet have a value, the proper
+              collection is created for it.
+            - If the collection is a list, C{key} is not used.
+            - If the attribute is not a collection attribute, the value is
+              simply set.
+        
+        @param attribute: the name of the attribute
+        @type attribute: a string
+        @param key: the key into the collection, not used with lists
+        @type key: anything
+        @param value: the value to set
+        @type value: anything compatible with the attribute's type
+        @return: the collection that was changed or created
+        """
 
         if _attrDict is None:
             if self._values.has_key(attribute):
@@ -708,7 +868,7 @@
                 attrValue = self.setAttributeValue(attribute, attrValue)
 
         if attrValue is Item.Nil:
-            self.setValue(attribute, value, key, alias, _attrDict)
+            return self.setValue(attribute, value, key, alias, _attrDict)
 
         else:
             self.setDirty(attribute=attribute)
@@ -722,19 +882,28 @@
             elif isinstance(attrValue, list):
                 attrValue.append(value)
             else:
-                self.setAttributeValue(attribute, value, _attrDict)
+                return self.setAttributeValue(attribute, value, _attrDict)
 
-    def hasKey(self, attribute, key):
-        """Tell where a multi-valued attribute has a value for a given key.
+            return attrValue
+
+    def hasKey(self, attribute, key, _attrDict=None):
+        """
+        Tell if a Chandler collection attribute has a value for a given key.
 
-        When the cardinality of the attribute is 'list' and its type is a
-        literal, key must be an integer.
-        When the cardinality of the attribute is 'list' and its values are
-        references, key must be an integer or the refName of the item value
-        to remove."""
+        The collection is obtained using L{getAttributeValue}.
 
-        value = (self._values.get(attribute, Item.Nil) or
-                 self._references.get(attribute, Item.Nil))
+        If the collection is a list of literals, C{key} must be an
+        integer and C{True} is returned if it is in range.
+
+        @param attribute: the name of the attribute
+        @type attribute: a string
+        @param key: the key into the collection, not used with lists
+        @type key: anything
+        @return: C{True} or C{False}
+        """
+
+        value = self.getAttributeValue(attribute, default=Item.Nil,
+                                       _attrDict=_attrDict)
 
         if value is not Item.Nil:
             if isinstance(value, dict):
@@ -747,19 +916,22 @@
         return False
 
     def hasValue(self, attribute, value, _attrDict=None):
-        """Tell whether an attribute contains a given value.
-
-        If the attribute is multi-valued the value argument is checked for
-        appartenance in the attribute's value collection.
-        If the attribute is single-valued the value argument is checked for
-        equality with the attribute's value."""
+        """
+        Tell if a Chandler collection attribute has a given value.
 
-        if _attrDict is not None:
-            attrValue = _attrDict.get(attribute, Item.Nil)
-        else:
-            attrValue = (self._values.get(attribute, Item.Nil) or
-                         self._references.get(attribute, Item.Nil))
+        The collection is obtained using L{getAttributeValue}.
+        If the attribute is not a collection, C{True} is returned if
+        C{value} is the same as attribute's value.
+
+        @param attribute: the name of the attribute
+        @type attribute: a string
+        @param value: the value looked for
+        @type value: anything
+        @return: C{True} or C{False}
+        """
 
+        attrValue = self.getAttributeValue(attribute, default=Item.Nil,
+                                           _attrDict=_attrDict)
         if attrValue is Item.Nil:
             return False
 
@@ -776,16 +948,21 @@
 
         return False
 
-    def removeValue(self, attribute, key=None, value=None, _attrDict=None):
-        """Remove the value from a multi-valued attribute for a given key.
+    def removeValue(self, attribute, key=None, _attrDict=None):
+        """
+        Remove a value from a Chandler collection attribute, for a given key.
 
-        When the cardinality of the attribute is 'list' and its type is a
-        literal, key must be an integer and value None.
-        When the cardinality of the attribute is 'dict' and its type is a
-        literal, key must be an existing key and value is ignored.
-        When the cardinality of the attribute is 'list' and its
-        values are references, key is ignored and value must be the
-        referenced item to remove from the collection."""
+        This method only operates on collections actually owned by this
+        attribute, not on collections inherited or otherwise defaulted via
+        L{getAttributeValue}.
+
+        If there is no value for the provided key, C{KeyError} is raised.
+
+        @param attribute: the name of the attribute
+        @type attribute: a string
+        @param key: the key into the collection
+        @type key: integer for lists, anything for dictionaries
+        """
 
         if _attrDict is None:
             if self._values.has_key(attribute):
@@ -810,7 +987,13 @@
         del self._references[name]
 
     def hasAttributeValue(self, name, _attrDict=None):
-        'Check for existence of a value for a given Chandler attribute.'
+        """
+        Tell if a Chandler attribute has a locally defined value.
+
+        @param name: the name of the attribute
+        @type name: a string
+        @return: C{True} or C{False}
+        """
 
         if _attrDict is None:
             return name in self._values or name in self._references
@@ -829,18 +1012,51 @@
             self._status &= ~Item.ATTACHING
 
     def isDeleting(self):
+        """
+        Tell whether this item is in the process of being deleted.
 
+        @return: C{True} or C{False}
+        """
+        
         return (self._status & Item.DELETING) != 0
     
     def isNew(self):
+        """
+        Tell whether this item is new.
+
+        A new item is defined as an item that was before committed to the
+        repository.
+        
+        @return: C{True} or C{False}
+        """
 
         return (self._status & Item.NEW) != 0
     
     def isDeleted(self):
+        """
+        Tell whether this item is deleted.
+
+        @return: C{True} or C{False}
+        """
 
         return (self._status & Item.DELETED) != 0
     
     def isStale(self):
+        """
+        Tell whether this item pointer is out of date.
+
+        A stale item pointer is defined as an item pointer that is no longer
+        valid. When an item is unloaded, the item pointer is marked
+        stale. The item pointer can be refreshed by reloading the item via the
+        L{find} method, passing it the item's C{uuid} obtained with the
+        L{getUUID} method.
+        
+        Stale items are encountered when item pointers are kept across
+        transaction boundaries. It is recommended to keep the item's
+        C{uuid} instead.
+
+        @return: C{True} or C{False}
+        """
 
         return (self._status & Item.STALE) != 0
     
@@ -849,23 +1065,51 @@
         self._status |= Item.STALE
 
     def isDirty(self):
+        """
+        Tell whether this item was changed and needs to be committed.
 
+        @return: C{True} or C{False}
+        """
+        
         return (self._status & Item.DIRTY) != 0
 
     def getDirty(self):
+        """
+        Return the dirty flags currently set on this item.
+
+        @return: an integer
+        """
 
         return self._status & Item.DIRTY
 
     def setDirty(self, dirty=None, attribute=None):
-        """Set a dirty bit on the item so that it gets persisted.
+        """
+        Mark this item to get committed with the current transaction.
 
-        Returns True if the dirty bit was changed from unset to set.
-        Returns False otherwise."""
+        Returns C{True} if the dirty bit was changed from unset to set.
+        Returns C{False} otherwise.
+
+        If C{attribute} is used and denotes a transient attribute (whose
+        C{persist} aspect is C{False}), then this method has no effect and
+        returns C{False}.
+
+        @param dirty: one of L{Item.VDIRTY <VDIRTY>},
+        L{Item.RDIRTY <RDIRTY>}, L{Item.CDIRTY <CDIRTY>},
+        L{Item.SDIRTY, <SDIRTY>} or a bitwise or'ed combination, defaults to
+        C{Item.VDIRTY}.
+        @type dirty: an integer
+        @param attribute: the name of the attribute that was changed,
+        optional, defaults to C{None} which means that no attribute was
+        changed
+        @type attribute: a string
+        @return: C{True} or C{False}
+        """
 
         if dirty is None:
-            dirty = Item.ADIRTY
+            dirty = Item.VDIRTY
 
         if dirty:
+            self._access = Item._countAccess()
             if self._status & Item.DIRTY == 0:
                 repository = self.getRepository()
                 if repository is not None and not repository.isLoading():
@@ -885,21 +1129,21 @@
 
         return False
 
-    def _setSaved(self, version):
-
-        self._version = version
-        self._status &= ~Item.NEW
-        self.setDirty(0)
-
     def delete(self, recursive=False):
-        """Delete this item and detach all its item references.
+        """
+        Delete this item.
 
-        If this item has children, they are recursively deleted first if
-        'recursive' is True.
-        If this item has references to other items and the references delete
-        policy is 'cascade' then these other items are deleted last.
-        A deleted item is no longer reachable through the repository or other
-        items. It is an error to access deleted items."""
+        If this item has references to other items and the C{deletePolicy}
+        aspect of the attributes containing them is C{cascade} then these
+        other items are deleted too.
+
+        It is an error to delete an item with children unless C{recursive}
+        is set to C{True}.
+
+        @param recursive: C{True} to recursively delete this item's children
+        too, C{False} otherwise (the default).
+        @type recursive: boolean
+        """
 
         if not self._status & (Item.DELETED | Item.DELETING):
 
@@ -942,21 +1186,30 @@
                     other.delete()
         
     def getItemName(self):
-        '''Return this item's name.
+        """
+        Return this item's name.
 
         The item name is used to lookup an item in its parent container and
         construct the item's path in the repository.
-        To rename an item use Item.rename().'''
+        To rename an item use L{rename}.
+
+        The name of an item must be unique among all its siblings.
+        """
 
         return self._name
 
     def getItemDisplayName(self):
-        """Return this item's display name.
+        """
+        Return this item's display name.
 
-        By definition, the display name is, in order of precedence, the
-        value of the 'displayName' attribute, the value of the attribute
-        named by the item's Kind 'displayAttribute' attribute or the item's
-        intrinsic name."""
+        By definition, the display name is, in order of precedence:
+            - the value of the C{displayName} attribute
+            - the value of the attribute named by the item's kind
+              C{displayAttribute} attribute
+            - or the item's intrinsic name (see L{getItemName}).
+
+        @return: a string
+        """
 
         if self.hasAttributeValue('displayName'):
             return self.displayName
@@ -970,12 +1223,18 @@
         return self._name
 
     def _refName(self, name):
-        'deprecated'
         
         return self._uuid
 
     def refCount(self):
-        'Return the total ref count for counted references on this item.'
+        """
+        Return the number of counted references to this item.
+
+        A reference is counted if the C{countPolicy} aspect of the attribute
+        containing it is C{count}.
+
+        @return: an integer
+        """
 
         count = 0
 
@@ -989,12 +1248,29 @@
         return count
 
     def getUUID(self):
-        'Return the Universally Unique ID for this item.'
+        """
+        Return the Universally Unique ID for this item.
+
+        The UUID for an item is generated when the item is first created and
+        never changes. This UUID is valid for the life of the item.
+
+        The UUID is a 128 bit number intended to be unique in the entire
+        universe and is implemented as specified in the IETF's U{UUID draft <www.ics.uci.edu/pub/ietf/webdav/uuid-guid/draft-leach-uuids-guids-01.txt>} spec.
+        """
         
         return self._uuid
 
     def getItemPath(self, path=None):
-        'Return the path to this item relative to its repository.'
+        """
+        Return the path to this item relative to its repository.
+
+        A path is a C{/} separated sequence of item names.
+
+        @param path: use this path instead of allocating a new Path object
+        for the result
+        @type path: a Path instance
+        @return: a Path instance
+        """
 
         if path is None:
             path = Path()
@@ -1005,9 +1281,15 @@
         return path
 
     def getRoot(self):
-        """Return this item's repository root.
+        """
+        Return this item's repository root.
 
-        All single-slash rooted paths are expressed relative to this root."""
+        A repository root is a direct child of the repository.
+        All single-slash rooted paths are expressed relative to this root
+        when used with this item.
+        
+        @return: an item
+        """
 
         if self._root.isStale():
             self._root = self.getRepository()[self._root._uuid]
@@ -1039,9 +1321,13 @@
                 child._setRoot(root)
 
     def getItemParent(self):
-        """Return this item's container parent.
+        """
+        Return this item's parent.
 
-        To change the parent, use Item.move()."""
+        To change the parent, use L{move}.
+
+        @return: an item
+        """
 
         if self._parent.isStale():
             self._parent = self.getRepository()[self._parent._uuid]
@@ -1060,9 +1346,12 @@
             self._references['kind'] = ref
 
     def getRepository(self):
-        """Return this item's repository.
+        """
+        Return this item's repository view.
 
-        The item's repository is defined as the item root's parent."""
+        The item's repository view is defined as the item's root's parent.
+        @return: a repository view
+        """
 
         if self._root is None:
             return None
@@ -1070,16 +1359,43 @@
             return self._root._parent
 
     def rename(self, name):
-        'Rename this item.'
+        """
+        Rename this item.
+
+        The name of an item needs to be unique among its siblings.
+        If C{name} is C{None}, the base64 representation of the item's
+        C{UUID} is used instead.
+
+        @param name: the new name for the item or C{None}
+        @type name: a string
+        """
 
         parent = self.getItemParent()
         link = parent._children._get(self._name)
         parent._removeItem(self)
-        self._name = name
+        self._name = name or self._uuid.str64()
         parent._addItem(self, link._previousKey, link._nextKey)
 
     def move(self, newParent, previous=None, next=None):
-        'Move this item under another container or make it a root.'
+        """
+        Move this item under another container.
+
+        The item's name needs to be unique among its siblings.
+        To make the item into a repository root, use the repository as
+        container.
+
+        Use C{previous} or {next} to place the item among its siblings.
+        By default, the item is added last into the sibling collection.
+        See also L{placeChild} to place the item without changing its
+        parent.
+
+        @param newParent: the container to move the item to
+        @type newParent: an item or the repository
+        @param previous: the optional item to place this item after
+        @type previous: an item
+        @param next: the optional item to place this item before
+        @type next: an item
+        """
 
         if newParent is None:
             raise ValueError, 'newParent cannot be None'
@@ -1087,7 +1403,7 @@
         parent = self.getItemParent()
         if parent is not newParent:
             parent._removeItem(self)
-            self._setParent(newParent)
+            self._setParent(newParent, previous, next)
 
     def _isRepository(self):
         return False
@@ -1139,7 +1455,19 @@
             self._notChildren = { name: name }
 
     def getItemChild(self, name, load=True):
-        'Return the child as named or None if not found.'
+        """
+        Return the named child or C{None} if not found.
+
+        The regular python C{[]} syntax may be used on the item to get
+        children from it except that when the child is not found,
+        C{KeyError} is raised.
+
+        @param name: the name of the child sought
+        @type name: a string
+        @param load: load the item if not currently loaded
+        @type load: boolean
+        @return: an item
+        """
 
         if self._status & Item.STALE:
             raise ValueError, "item is stale: %s" %(self)
@@ -1171,11 +1499,25 @@
         raise TypeError, key
 
     def isRemote(self):
-        'By default, an item is not remote.'
+        """
+        Tell whether this item is a remote item.
+
+        @return: C{False}
+        """
 
         return False
 
     def isItemOf(self, kind):
+        """
+        Tell whether this item is of a certain kind.
+
+        Like python's C{isinstance} function, this method tells whether an
+        item is of kind C{kind} or of a subkind thereof.
+
+        @param kind: a kind
+        @type kind: an item of kind C{Kind}
+        @return: boolean
+        """
 
         if self._kind is kind:
             return True
@@ -1186,13 +1528,31 @@
         return False
 
     def walk(self, path, callable, _index=0, **kwds):
-        """Walk a path and invoke a callable along the way.
+        """
+        Walk a path and invoke a callable along the way.
 
-        The callable's arguments should be (parent, childName, child, **kwds).
-        The callable's child argument is None if the path didn't
-        correspond to an existing item.
+        The callable's arguments need to be defined as C{parent},
+        C{childName}, C{child} and C{**kwds}.
+        The callable is passed C{None} for the C{child} argument if C{path}
+        doesn't correspond to an existing item.
         The callable's return value is used to recursively continue walking
-        unless this return values is None."""
+        when it is not C{None}.
+
+        For example: L{find} calls this method when passed a path with the
+        callable being the simple lambda body:
+
+            - C{lambda parent, name, child, **kwds: child}
+
+        A C{load} keyword can be used to prevent loading of items by setting
+        it to C{False}. Items are loaded as needed by default.
+
+        @param path: an item path
+        @type path: a C{Path} instance or a string representing a path
+        @param callable: a function, method, or lambda body
+        @type callable: a python callable
+        @param kwds: optional keywords passed to the callable
+        @return: the item the walk finished on or C{None}
+        """
 
         if _index == 0 and not isinstance(path, Path):
             path = Path(path)
@@ -1229,11 +1589,22 @@
         return None
 
     def find(self, spec, _index=0, load=True):
-        """Find an item as specified or return None if not found.
-        
-        Spec can be a Path, a UUID or a string in which case it gets coerced
-        into one of the former. If spec is a path, the search is done relative
-        to the item unless the path is absolute."""
+        """
+        Find an item.
+
+        An item can be found by a path determined by its name and container
+        or by a uuid generated for it at creation time. If C{spec} is a
+        relative path, it is evaluated relative to C{self}.
+
+        This method returns C{None} if the item is not found or if it is
+        found but not yet loaded and C{load} was set to C{False}.
+
+        @param spec: a path or UUID
+        @type spec: Path, UUID or a string representation thereof
+        @param load: load the item if it not yet loaded, C{True} by default
+        @type load: boolean
+        @return: an item or C{None} if not found
+        """
 
         if isinstance(spec, Path):
             return self.walk(spec, lambda parent, name, child, **kwds: child,
@@ -1254,7 +1625,14 @@
         return None
 
     def toXML(self):
+        """
+        Generate an XML representation of this item.
+
+        This method is not a general purpose serialization method for items
+        but it can be used for debugging.
 
+        @return: an XML string
+        """
         out = None
         
         try:
@@ -1271,14 +1649,71 @@
             if out is not None:
                 out.close()
 
-    def _saveItem(self, generator, version):
+    def _saveItem(self, generator, version, mergeWith=None):
+
+        withSchema = (self._status & Item.SCHEMA) != 0
+
+        if mergeWith is None:
+            self._xmlItem(generator, withSchema, version, 'save')
+            self._status |= Item.SAVED
 
-        self._xmlItem(generator,
-                      withSchema = (self._status & Item.SCHEMA) != 0,
-                      version = version, mode = 'save')
+        else:
+            oldDoc, oldDirty, newDirty = mergeWith
+            if oldDirty & newDirty:
+                raise ValueError, "merges overlap (%0.4x:%0.4x)" %(oldDirty,
+                                                                   newDirty)
+            def mergeNewOld(*attributes):
+                class merger(XMLOffFilter):
+                    def startElement(_self, tag, attrs):
+                        if tag == 'item':
+                            attrs['version'] = str(version)
+                        XMLOffFilter.startElement(_self, tag, attrs)
+                    def endElement(_self, tag):
+                        if tag == 'item':
+                            self._xmlAttrs(generator, withSchema,
+                                           version, 'save')
+                        XMLOffFilter.endElement(_self, tag)
+
+                merger(generator, *attributes).parse(oldDoc)
+                self._status |= Item.MERGED
+
+            def mergeOldNew(dirty, *attributes):
+                class merger(XMLThruFilter):
+                    def endElement(_self, tag):
+                        if tag == 'item':
+                            XMLOnFilter(generator, *attributes).parse(oldDoc)
+                        XMLThruFilter.endElement(_self, tag)
+
+                out = cStringIO.StringIO()
+                xml = XMLGenerator(out, 'utf-8')
+                xml.startDocument()
+                self._xmlItem(xml, withSchema, version, 'save',
+                              Item.DIRTY & ~dirty)
+                xml.endDocument()
+                newDoc = out.getvalue()
+                out.close()
+
+                merger(generator).parse(newDoc)
+                self._status |= Item.MERGED
+                
+            if newDirty == Item.VDIRTY:
+                mergeNewOld('attribute')
+            elif newDirty == Item.RDIRTY:
+                mergeNewOld('ref')
+            elif newDirty == Item.VRDIRTY:
+                mergeNewOld('attribute', 'ref')
+            elif oldDirty == Item.VDIRTY:
+                mergeOldNew(Item.VDIRTY, 'attribute')
+            elif oldDirty == Item.RDIRTY:
+                mergeOldNew(Item.RDIRTY, 'ref')
+            elif oldDirty == Item.VRDIRTY:
+                mergeOldNew(Item.VRDIRTY, 'attribute', 'ref')
+            else:
+                raise NotImplementedError, "merge %0.4x:%0.4x" %(oldDirty,
+                                                                 newDirty)
 
     def _xmlItem(self, generator, withSchema=False, version=None,
-                 mode='serialize'):
+                 mode='serialize', save=None):
 
         def xmlTag(tag, attrs, value, generator):
 
@@ -1287,6 +1722,8 @@
             generator.endElement(tag)
 
         isDeleted = self.isDeleted()
+        if save is None:
+            save = Item.DIRTY
 
         attrs = { 'uuid': self._uuid.str64() }
         if withSchema:
@@ -1330,8 +1767,10 @@
         xmlTag('container', attrs, parentID, generator)
 
         if not isDeleted:
-            self._xmlAttrs(generator, withSchema, version, mode)
-            self._xmlRefs(generator, withSchema, version, mode)
+            if save & Item.VDIRTY:
+                self._xmlAttrs(generator, withSchema, version, mode)
+            if save & Item.RDIRTY:
+                self._xmlRefs(generator, withSchema, version, mode)
 
         generator.endElement('item')
 
@@ -1354,8 +1793,10 @@
             if self.hasAttributeValue('kind'):
                 del self.kind
 
-            self._values._unload()
-            self._references._unload()
+            if self._values:
+                self._values._unload()
+            if self._references:
+                self._references._unload()
             repository._unregisterItem(self)
 
             self._parent._unloadChild(self._name)
@@ -1419,6 +1860,13 @@
         return self.getRepository().createRefDict(self, name,
                                                   otherName, persist)
 
+    def _countAccess(cls):
+
+        cls.__access__ += 1
+        return cls.__access__
+
+    _countAccess = classmethod(_countAccess)
+
     def __new__(cls, *args, **kwds):
 
         item = object.__new__(cls, *args, **kwds)
@@ -1431,18 +1879,26 @@
     class nil(object):
         def __nonzero__(self):
             return False
-    Nil       = nil()
+    Nil        = nil()
     
-    DELETED   = 0x0001
-    ADIRTY    = 0x0002
-    DELETING  = 0x0004
-    RAW       = 0x0008
-    ATTACHING = 0x0010
-    SCHEMA    = 0x0020
-    NEW       = 0x0040
-    STALE     = 0x0080
-    HDIRTY    = 0x0100
-    DIRTY     = ADIRTY | HDIRTY
+    DELETED    = 0x0001
+    VDIRTY     = 0x0002           # literal value(s) changed
+    DELETING   = 0x0004
+    RAW        = 0x0008
+    ATTACHING  = 0x0010
+    SCHEMA     = 0x0020
+    NEW        = 0x0040
+    STALE      = 0x0080
+    SDIRTY     = 0x0100           # name of sibling(s) changed
+    CDIRTY     = 0x0200           # parent or first/last child changed
+    RDIRTY     = 0x0400           # ref or ref collection value changed
+    MERGED     = 0x0800
+    SAVED      = 0x1000
+
+    VRDIRTY    = VDIRTY | RDIRTY
+    DIRTY      = VDIRTY | SDIRTY | CDIRTY | RDIRTY
+
+    __access__ = 0L
 
 
 class Children(LinkedMap):
@@ -1461,9 +1917,9 @@
     def linkChanged(self, link, key):
 
         if key is None:
-            self._item.setDirty(dirty=Item.HDIRTY)
+            self._item.setDirty(dirty=Item.CDIRTY)
         else:
-            link._value.setDirty(dirty=Item.HDIRTY)
+            link._value.setDirty(dirty=Item.SDIRTY)
     
     def __repr__(self):
 
@@ -1498,7 +1954,7 @@
 
 
 class ItemValue(object):
-    'A superclass for values that can be set on only one item attribute'
+    'A superclass for values that are owned by an item.'
     
     def __init__(self):
 
@@ -1530,4 +1986,5 @@
         if not self._dirty:
             self._dirty = True
             if self._item is not None:
-                self._item.setDirty()
+                self._item.setDirty(attribute=self._attribute,
+                                    dirty=Item.VDIRTY)

Index: osaf/chandler/Chandler/repository/util/ThreadLocal.py
diff -u osaf/chandler/Chandler/repository/util/ThreadLocal.py:1.1 osaf/chandler/Chandler/repository/util/ThreadLocal.py:1.2
--- osaf/chandler/Chandler/repository/util/ThreadLocal.py:1.1	Fri Oct 17 12:05:51 2003
+++ osaf/chandler/Chandler/repository/util/ThreadLocal.py	Mon Mar  8 14:56:57 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.1 $"
-__date__      = "$Date: 2003/10/17 19:05:51 $"
+__revision__  = "$Revision: 1.2 $"
+__date__      = "$Date: 2004/03/08 22:56:57 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -9,9 +9,11 @@
 
 
 class ThreadLocal(object):
-    """A class using thread local storage for its attributes.
+    """
+    A class using thread local storage for its attributes.
 
-    All attributes on this class have thread local values."""
+    All attributes on this class have thread local values.
+    """
     
     __slots__ = ()
     

Index: osaf/chandler/Chandler/repository/__hardhat__.py
diff -u osaf/chandler/Chandler/repository/__hardhat__.py:1.2 osaf/chandler/Chandler/repository/__hardhat__.py:1.3
--- osaf/chandler/Chandler/repository/__hardhat__.py:1.2	Wed Jan 28 16:53:39 2004
+++ osaf/chandler/Chandler/repository/__hardhat__.py	Mon Mar  8 14:56:51 2004
@@ -37,7 +37,7 @@
                           '-o api -v -n chandlerdb',
                           '--inheritance listed',
                           '--no-private',
-                          'item', 'schema')
+                          'item', 'schema', 'util')
 
 
 def clean(buildenv):

Index: osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py
diff -u osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py:1.29 osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py:1.30
--- osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py:1.29	Thu Feb  5 15:49:31 2004
+++ osaf/chandler/Chandler/repository/persistence/XMLRepositoryView.py	Mon Mar  8 14:56:53 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.29 $"
-__date__      = "$Date: 2004/02/05 23:49:31 $"
+__revision__  = "$Revision: 1.30 $"
+__date__      = "$Date: 2004/03/08 22:56:53 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -14,7 +14,7 @@
 from dbxml import XmlDocument, XmlValue
 
 from repository.item.Item import Item, ItemValue
-from repository.item.ItemRef import RefDict, TransientRefDict
+from repository.item.ItemRef import RefDict, TransientRefDict, Values
 from repository.persistence.Repository import Repository, RepositoryError
 from repository.persistence.Repository import VersionConflictError
 from repository.persistence.Repository import OnDemandRepositoryView
@@ -60,6 +60,8 @@
         del self._log[:]
         self._notRoots.clear()
 
+        self.prune(10000)
+
     def queryItems(self, query, load=True):
 
         store = self.repository.store
@@ -155,7 +157,8 @@
         env = repository._env
 
         self._notifications.clear()
-
+        histNotifications = None
+        
         before = datetime.now()
         count = len(self._log)
         size = 0L
@@ -165,6 +168,7 @@
         while True:
             try:
                 txnStarted = self._startTransaction()
+                unloads = {}
 
                 newVersion = versions.getVersion()
                 if count > 0:
@@ -188,7 +192,37 @@
                 
                     for item in self._log:
                         size += self._saveItem(item, newVersion,
-                                               data, versions, history)
+                                               data, versions, history, ood)
+
+                if newVersion > self.version:
+                    histNotifications = RepositoryNotifications(repository)
+                    
+                    def unload(uuid, version, docId, status, parent):
+
+                        if status & Item.DELETED:
+                            histNotifications.history(uuid, 'deleted',
+                                                      parent=parent)
+                        elif status & Item.NEW:
+                            histNotifications.history(uuid, 'added')
+                        else:
+                            histNotifications.history(uuid, 'changed')
+
+                        item = self._registry.get(uuid)
+                        if (item is not None and
+                            not item._status & Item.SAVED and
+                            item._version < newVersion):
+                            unloads[item._uuid] = item
+
+                    history.apply(unload, self.version, newVersion)
+            
+                if txnStarted:
+                    self._commitTransaction()
+
+                if lock:
+                    env.lock_put(lock)
+                    lock = None
+
+                break
 
             except DBLockDeadlockError:
                 self.logger.info('restarting commit aborted by deadlock')
@@ -210,93 +244,81 @@
 
                 raise
 
-            else:
-                if self._log:
-                    for item in self._log:
-                        item._setSaved(newVersion)
-                    del self._log[:]
-
-                self.logger.debug('refreshing view from version %d to %d',
-                                  self.version, newVersion)
-
-                if newVersion > self.version:
-                    try:
-                        oldVersion = self.version
-                        self.version = newVersion
-                        notifications = RepositoryNotifications(repository)
-                        
-                        def unload(uuid, version, docId, status, parent):
-
-                            if status & Item.DELETED:
-                                notifications.history(uuid, 'deleted',
-                                                      parent=parent)
-                            elif status & Item.NEW:
-                                notifications.history(uuid, 'added')
-                            else:
-                                notifications.history(uuid, 'changed')
-
-                            item = self._registry.get(uuid)
-                            if item is not None and item._version < newVersion:
-                                if self.isDebug():
-                                    self.logger.debug('unloading version %d of %s',
-                                                      item._version,
-                                                      item.getItemPath())
-                                item._unloadItem()
-                            
-                        history.apply(unload, oldVersion, newVersion)
-                        notifications.dispatchHistory()
-                        
-                    except:
-                        if txnStarted:
-                            self._abortTransaction()
-                        raise
-            
-                if txnStarted:
-                    self._commitTransaction()
-
-                if lock:
-                    env.lock_put(lock)
-                    lock = None
-
-                self._notRoots.clear()
-                self._notifications.dispatchChanges()
+        if self._log:
+            for item in self._log:
+                if not item._status & Item.MERGED:
+                    item._version = newVersion
+                item._status &= ~(Item.NEW | Item.DIRTY |
+                                  Item.MERGED | Item.SAVED)
+            del self._log[:]
+
+        if newVersion > self.version:
+            self.logger.debug('refreshing view from version %d to %d',
+                              self.version, newVersion)
+            self.version = newVersion
+            for item in unloads.itervalues():
+                self.logger.debug('unloading version %d of %s',
+                                  item._version, item)
+                item._unloadItem()
+                    
+        self._notRoots.clear()
 
-                if count > 0:
-                    self.logger.info('%s committed %d items (%ld bytes) in %s',
-                                     self, count, size,
-                                     datetime.now() - before)
-                
-                return
+        if histNotifications is not None:
+            histNotifications.dispatchHistory()
+        self._notifications.dispatchChanges()
+
+        if count > 0:
+            self.logger.info('%s committed %d items (%ld bytes) in %s',
+                             self, count, size,
+                             datetime.now() - before)
+        self.prune(10000)
 
-    def _saveItem(self, item, newVersion, data, versions, history):
+    def _saveItem(self, item, newVersion, data, versions, history, ood):
 
         uuid = item._uuid
         isNew = item.isNew()
         isDeleted = item.isDeleted()
+        isDebug = self.isDebug()
         
         if isDeleted:
             del self._deletedRegistry[uuid]
             if isNew:
                 return 0
 
-        if self.isDebug():
+        if isDebug:
             self.logger.debug('saving version %d of %s',
                               newVersion, item.getItemPath())
 
+        if uuid in ood:
+            docId, oldDirty, newDirty = ood[uuid]
+            mergeWith = (data.getDocument(docId).getContent(),
+                         oldDirty, newDirty)
+            if isDebug:
+                self.logger.debug('merging %s (%0.4x:%0.4x) with newest version',
+                                  item.getItemPath(), oldDirty, newDirty)
+        else:
+            mergeWith = None
+            
         out = cStringIO.StringIO()
         generator = XMLGenerator(out, 'utf-8')
         generator.startDocument()
-        item._saveItem(generator, newVersion)
+        item._saveItem(generator, newVersion, mergeWith)
         generator.endDocument()
         content = out.getvalue()
         out.close()
+
         size = len(content)
 
         doc = XmlDocument()
         doc.setContent(content)
         if isDeleted:
             doc.setMetaData('', '', 'deleted', XmlValue('True'))
-        docId = data.putDocument(doc)
+
+        try:
+            docId = data.putDocument(doc)
+        except:
+            self.logger.exception("putDocument failed, xml is: %s", content)
+            raise
 
         if isDeleted:
             parent=item.getItemParent().getUUID()
@@ -320,15 +342,19 @@
         def check(uuid, version, docId, status, parentId):
             item = items.get(uuid)
             if item is not None:
-                if item.getDirty() & status:
+                newDirty = item.getDirty()
+                oldDirty = status & item.DIRTY
+                if newDirty & oldDirty:
                     raise VersionConflictError, item
+                else:
+                    if (newDirty == item.VDIRTY or oldDirty == item.VDIRTY or
+                        newDirty == item.RDIRTY or oldDirty == item.RDIRTY or
+                        newDirty == item.VRDIRTY or oldDirty == item.VRDIRTY):
+                        items[uuid] = (docId, oldDirty, newDirty)
+                    else:
+                        raise NotImplementedError, 'Item %s may be mergeable but this particular merge (0x%x:0x%x) is not implemented yet' %(item.getItemPath(), newDirty, oldDirty)    
 
         history.apply(check, oldVersion, newVersion)
-
-        for item in items.itervalues():
-            self.logger.info('Item %s is out of date but is mergeable',
-                             item.getItemPath())
-        raise NotImplementedError, 'item merging not yet implemented'
 
 
 class XMLRepositoryClientView(XMLRepositoryView):

Index: osaf/chandler/Chandler/repository/util/LinkedMap.py
diff -u osaf/chandler/Chandler/repository/util/LinkedMap.py:1.11 osaf/chandler/Chandler/repository/util/LinkedMap.py:1.12
--- osaf/chandler/Chandler/repository/util/LinkedMap.py:1.11	Thu Jan  8 22:22:44 2004
+++ osaf/chandler/Chandler/repository/util/LinkedMap.py	Mon Mar  8 14:56:57 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.11 $"
-__date__      = "$Date: 2004/01/09 06:22:44 $"
+__revision__  = "$Revision: 1.12 $"
+__date__      = "$Date: 2004/03/08 22:56:57 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -132,6 +132,8 @@
 
         if self.has_key(key):
             current = self._get(key)
+            if current._previousKey == afterKey:
+                return
             if current._previousKey is not None:
                 previous = self._get(current._previousKey)
             else:
@@ -201,9 +203,10 @@
     def get(self, key, default=None, load=True):
 
         link = super(LinkedMap, self).get(key, default)
+
         if link is default and load and self._load(key):
             link = super(LinkedMap, self).get(key, default)
-            
+        
         if link is not default:
             return link._value
 

Index: osaf/chandler/Chandler/repository/persistence/XMLRepository.py
diff -u osaf/chandler/Chandler/repository/persistence/XMLRepository.py:1.63 osaf/chandler/Chandler/repository/persistence/XMLRepository.py:1.64
--- osaf/chandler/Chandler/repository/persistence/XMLRepository.py:1.63	Tue Feb  3 19:37:51 2004
+++ osaf/chandler/Chandler/repository/persistence/XMLRepository.py	Mon Mar  8 14:56:53 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.63 $"
-__date__      = "$Date: 2004/02/04 03:37:51 $"
+__revision__  = "$Revision: 1.64 $"
+__date__      = "$Date: 2004/03/08 22:56:53 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -182,7 +182,7 @@
 
             # None -> not found, 0 -> deleted
             if docId: 
-                return self._xml.getDocument(store.txn, docId, DB_DIRTY_READ)
+                return self.getDocument(docId)
 
         finally:
             if txnStarted:
@@ -313,6 +313,10 @@
     def deleteDocument(self, doc):
 
         self._xml.deleteDocument(self.store.txn, doc, self.store.updateCtx)
+
+    def getDocument(self, docId):
+
+        return self._xml.getDocument(self.store.txn, docId, DB_DIRTY_READ)
 
     def putDocument(self, doc):
 

Index: osaf/chandler/Chandler/repository/util/Streams.py
diff -u osaf/chandler/Chandler/repository/util/Streams.py:1.9 osaf/chandler/Chandler/repository/util/Streams.py:1.10
--- osaf/chandler/Chandler/repository/util/Streams.py:1.9	Thu Jan 22 18:29:20 2004
+++ osaf/chandler/Chandler/repository/util/Streams.py	Mon Mar  8 14:56:57 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.9 $"
-__date__      = "$Date: 2004/01/23 02:29:20 $"
+__revision__  = "$Revision: 1.10 $"
+__date__      = "$Date: 2004/03/08 22:56:57 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -188,6 +188,8 @@
                 return data
             else:
                 self._done.append(self._streams.pop(0))
+
+        return ''
 
     def close(self):
 

Index: osaf/chandler/Chandler/repository/util/UUID.py
diff -u osaf/chandler/Chandler/repository/util/UUID.py:1.10 osaf/chandler/Chandler/repository/util/UUID.py:1.11
--- osaf/chandler/Chandler/repository/util/UUID.py:1.10	Thu Jan 22 17:42:27 2004
+++ osaf/chandler/Chandler/repository/util/UUID.py	Mon Mar  8 14:56:57 2004
@@ -1,6 +1,6 @@
 
-__revision__  = "$Revision: 1.10 $"
-__date__      = "$Date: 2004/01/23 01:42:27 $"
+__revision__  = "$Revision: 1.11 $"
+__date__      = "$Date: 2004/03/08 22:56:57 $"
 __copyright__ = "Copyright (c) 2002 Open Source Applications Foundation"
 __license__   = "http://osafoundation.org/Chandler_0.1_license_terms.htm"
 
@@ -8,15 +8,27 @@
 
 
 class UUID(object):
-    """Implementation of UUID spec at http://www.ics.uci.edu/pub/ietf/webdav/uuid-guid/draft-leach-uuids-guids-01.txt.
+    """
+    Implementation of the IETF's U{UUID draft <www.ics.uci.edu/pub/ietf/webdav/uuid-guid/draft-leach-uuids-guids-01.txt>} spec.
 
-    A UUID is intended to be globally unique in the entire universe.
-    It is a 128 bit number.
-    UUID objects can be used as dictionary keys and are comparable."""
+    A UUID is intended to be a globally, universally unique 128 bit number.
+    UUID objects can be used as dictionary keys and are comparable.
+    """
 
     __slots__ = ("_uuid", "_hash")
     
     def __init__(self, uuid=None):
+        """
+        Construct a UUID.
+
+        Generate a new UUID when C{uuid} is C{None} or re-construct a UUID
+        instance from a string representation or from the 16 bytes
+        equivalent to the 128 bit number.
+
+        @param uuid: a 36 or 22 byte string representation of a UUID or 16
+        intrinsic bytes, C{None} by default.
+        @type uuid: a string
+        """
 
         super(UUID, self).__init__()
 
@@ -74,18 +86,26 @@
         return isinstance(other, UUID) and self._uuid != other._uuid
 
     def str16(self):
-        '''Return the standard hexadecimal string representation of this UUID.
+        """
+        Get a standard hexadecimal string representation of this UUID.
 
         This string is in the format abf5678c-9d49-11d7-e690-000393db837c and
-        is always 36 characters long.'''
+        is 36 characters long.
+
+        @return: a string
+        """
 
         return UUIDext.toString(self._uuid)
 
     def str64(self):
-        '''Return a shorter base64 encoded string representation of this UUID.
+        """
+        Get a shorter base64 encoded string representation of this UUID.
+
+        This string is 22 characters long and looks like this
+        aLRpUOtih7neqg00ejSUdY.
 
-        This string is always 22 characters long and looks like this
-        aLRpUOtih7neqg00ejSUdY.'''
+        @return: a string
+        """
 
         return UUIDext.to64String(self._uuid)
 



More information about the Commits mailing list