[Dev] CHANDLERHOME and directory layout (was Re: Unit test runner)

[Phillip J. Eby]

At 04:30 PM 2/4/05 -0800, Ted Leung wrote:
>Ok,
>
>So I'm late with this commentary (to the point that the test runner, etc 
>are already checked in).  So far I like what I see.
>
>My only other comment is that I agree that we should be adding test 
>methods in preference to test modules.

Also, I noticed today that most of the tests that use the CHANDLERHOME 
environment variable don't actually need to.  Most uses of CHANDLERHOME are 
just trying to find the repository/packs/ directory or repository/tests, 
but these paths can be obtained
via the '__file__' attribute of the modules in question, e.g:

     import os, repository
     packdir = os.path.join(os.path.dirname(repository.__file__),'packs')

In other words, if you take the dirname() of a module's __file__ attribute, 
you get the directory where that module lives.  If all you need to do is 
find the a module's directory (or subdirectory thereof) in order to load 
some resources (like packs), this is all that's needed, so use of 
CHANDLERHOME can be dropped in these places.

This leaves Globals.chandlerDirectory and the location of 'chandler.log' 
for tests, but the majority of uses of chandlerDirectory are also to locate 
a module's directory, with the remainder mainly being to determine either 
the profile directory or the parcel directory.

It looks to me like it might be useful to add a utility function 
'fileNearModule' (similar to the one in PEAK), such that:

     import repository
     fileNearModule(repository, 'packs/schema.pack')

returns the filename of 'repository/packs/schema.pack'.  It could then be 
used for all the various things that are now looking for paths near modules.

We could then banish CHANDLERHOME, and simply have tests dump 
'chandler.log' in the current directory by default.

Parcels are a little harder, because the 'parcels' directory isn't a 
package or module, so there's no way to get its __file__.  However, if I 
understand correctly, the parcel directory is only needed in order to scan 
it for parcels to be loaded.  So, instead of having a single fixed location 
for parcels, the parcel manager could have a list of them, generated by 
scanning sys.path.  This has a couple of advantages for both development 
and deployment, because we could then support installing parcels in a 
profile directory as well as off of the Chandler home directory, 
distinguishing between Chandler-supplied and user-installed parcels.

Actually, perhaps in 0.6 it would be simplest to just put Chandler-supplied 
parcels directly into the Chandler home directory, and have user-installed 
parcels live just off of the profile directory.

But anyway, I'm getting ahead of myself here.  For right now, I'm proposing 
to add a 'fileNearModule' function, and to use it to replace all of the 
'CHANDLERHOME' and 'chandlerDirectory' uses that are looking for a filename 
near a module.  For any remaining uses of CHANDLERHOME, I propose making it 
default to '.', which would make it default to the current directory.

The end result would be that only a few parcel-related uses of 
Globals.chandlerDirectory would remain, and 'chandlerDirectory' itself 
would eventually be phased out, replaced by looking for Chandler-supplied 
parcels using fileNearModule, and for user-supplied parcels using 
'profileDir/parcels' (which Chandler's startup code would add to 'sys.path'.

And when that latter step is complete, Chandler would no longer need 
CHANDLERHOME or PYTHONPATH setup in order to run; running it from the 
Chandler installation location would suffice, allowing us to drop at least 
the RunPython script, and maybe the RunChandler script as well.

Anybody have ideas for where 'fileNearModule' should go?  In 'tools', 
perhaps?  It needs to be used by both the repository and by application 
code, so ideally it should be in a place that's already a dependency for both.