[Dev] Speeding up unit tests

[Ted Leung]

Hi,

I've just checked in some changes that will speed up unit tests in some 
cases.
This is being done by allowing the tests to read their data from a 
pre-populated
repository.

I've updated repository/tests/RepositoryTestCase.py to look for a 
preloaded
repository named __preloaded_repository__.  This repository should be 
in the
same directory as the test case that will use the data.   When you 
derive a
TestCase from RepositoryTestCase, you'll need to override the setUp 
method
like this (taken from the parcel loader test case):

     def setUp(self):
         super(ParcelLoaderTestCase,self)._setup(self)

         # location of tests and preloaded repository, if any
         self.testdir = os.path.join(self.rootdir, 'chandler', 
'repository',
          'parcel', 'tests')

         super(ParcelLoaderTestCase,self)._openRepository(self)
         Globals.repository = self.rep # to keep indexer happy


If you create a repository loaded with "the right data", then any test 
derived
from repository.tests.RepositoryTestCase will read this preloaded 
repository.
You can use the program repository/tests/PreloadedRepositoryUtils.py to 
create
a preloaded repository in the current directory.  Here's the usage info 
for
PreloadeRepositoryUtils:

usage: PreloadedRepositoryUtils: [--create [parcels]] [--test]
   --create [parcels]  create a preloaded repository containing core 
schema
                       if the word 'parcels' is supplied as an argument, 
the
                       preloaded repository contains all parcels as well
   --test              run a simple test comparing preloaded vs 
non-preloaded performance

After you have created preloaded repositories where you want to use 
them, you
just run the unit tests as normal.  If you are interested in using 
this, you
should probably preload data for the layers under your test, but not 
the data
for your test.  So using the parcel loader as an example, the parcel 
loader tests
should use a preloaded repository that contains the core schema, but 
not containing
any parcels.

The unit tests previously used the BerkeleyDB RAM database mode to 
achieve some
speedups.  This mode is incompatible with pre-loading.  It's important 
to
understand when preloading will actually speed things up.  If your test 
needs to
load the core schemas (and possibly parcels), and does not do a lot of 
I/O after
that, then preloading will definitely speed things up.  If your test 
needs to do
a lot of I/O beyond loading up the core schemas and parcels, then 
preloading probably
will not help you because the improvement in schema/parcel load time 
will be dominated
by the slowdown in general I/O times (due to not using the RAM database)

If you have suggestions for making the process more user friendly, 
please let me know.

----
Ted Leung                 Open Source Applications Foundation (OSAF)
PGP Fingerprint: 1003 7870 251F FA71 A59A  CEE3 BEBA 2B87 F5FC 4B42