[PEAK] dumb question about easy_install

[Phillip J. Eby]

At 02:41 PM 6/28/2006 -0400, Michael Bayer wrote:
>This is in contrast to PYTHONPATH, which had a very simple and
>obvious operation (and also consistent with every other language
>environment's behavior).  PYTHONPATH's behavior now diverges from
>whats documented in all the python books and websites when anything
>setuptools-related has been installed, and the alternative is now
>significantly more tedious and complicated (and also will totally hit
>newbies by surprise).

I don't really understand the problem here.  If you are developing a 
setuptools-based package, "setup.py develop" or easy_install are the 
documented ways to install it -- *regardless of destination 
directory*.  There are also documented ways of uninstalling and 
deactivating setuptools-based packages.  If you are using setuptools, 
PYTHONPATH is only a way of adding or removing *installation directories* 
from sys.path, not of overriding individual packages.

If you are *not* developing a setuptools-based package, how did the 
problematic package get installed, or conversely, why do you need to 
override it with something else, without uninstalling or at least 
deactivating it?


>Why cant we retain "classic" behavior for at least paths that are
>explicitly in PYTHONPATH ?

We do.  The specific problem you're having is that you used setuptools in 
site-packages, but not on PYTHONPATH.  This is the *reverse* of the normal 
setup, where you do *not* use setuptools on site-packages, but *do* on 
PYTHONPATH, or use it in both places, or neither.

Eggs always override non-eggs, because eggs can be controlled in ways that 
non-egg paths can't.  The order is (roughly):

eggs on PYTHONPATH
eggs in site-packages
non-eggs on PYTHONPATH
non-eggs in site-packages

"eggs" here includes packages installed via "setup.py develop", but not 
ones installed via "setup.py install --single-version-externally-managed", 
even though they too are technically eggs.


>   or at least have some other variable like
>PEAKPATH or something ?   this new behavior seems to be not in the
>spirit of simple and transparent behavior for an installer tool

The big problem is that eggs *have* to override non-eggs in order to ensure 
that system-installed (or otherwise directly-installed) packages don't 
block access to them.  This is a critical feature for users who don't have 
write access to site-packages.


>import sys; sys.__plen = len(sys.path)
>.. list of modules ...
>import os; import sys; new=sys.path[sys.__plen:]; del sys.path 
>[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new;
>sys.__egginsert = p+len(new); [sys.path.insert(0, x) for x in
>os.environ.get('PYTHONPATH', '').split(os.pathsep)];

Note that if you create a .pth file that is alphabetically before 
'easy-install', or place one on PYTHONPATH, you can have it bump 
sys.__egginsert up to your desired insertion point.  I don't really 
recommend this, because I still don't understand what benefit you get from 
manipulating PYTHONPATH in this way (other than familiarity, of course).

 From my POV, manipulating PYTHONPATH to include or exclude individual 
packages is a pain, as it's so much easier to either create single-program 
environments (i.e. a single-directory PYTHONPATH with all needed eggs 
installed) or multi-version environments (i.e., a single directory with 
lots of different eggs, automatically found by the programs that need them.)

These approaches seem clearly superior to library-at-a-time PYTHONPATH 
munging, which people often complain about, especially in Java where the 
paths *are* in terms of jars rather than in terms of "environments" where 
jars can be discovered.