[PEAK] Re: setuptools test command generates exception after interpreter shutdown

[P.J. Eby]

At 11:58 AM 5/4/2010 -0400, Nikolaus Rath wrote:
>On 05/04/2010 11:23 AM, P.J. Eby wrote:
> > Now, to narrow it down further, we'll have to remove stuff from the
> > setuptools code execution path to find out what part of setuptools is
> > relevant.  I would start with removing code from the test command's
> > 'run()' method, especially the running of other commands such as
> > egg_info, and the saving and restoring of sys.path and sys.modules.  My
> > guess at this point is that it may be the sys.modules restoration that
> > triggers it.
> >
> > Actually, there's a quick way to test that: add 'import multiprocessing,
> > logging' to your setup.py (not the test code).  If that makes it work,
> > then the sys.modules restoration is probably the culprit.  If it still
> > doesn't work, then it's almost certainly a multiprocessing/logging bug.
>
>Good guess, adding the import to setup.py indeed fixes the problem. Are
>you able to do something about this? Or is that an intrinsic problem
>with the restoration?

Well, the original intention of the addition was to prevent problems 
with importing older (installed) versions of a project, and only 
secondarily to prevent the imported contents of the package from 
lingering in sys.modules.  I suppose it could be more selective in 
clearing only the modules that were imported from the project 
directory.  However, even those could be doing atexit stuff or having 
other methods run.  Really, it looks like it's not safe to drop 
references to module objects in the general case.

So, I think what's going to have to happen is that the sys.modules 
save-and-restore will have to be skipped.  I'm worried that it'll 
introduce other problems, but I don't really see a way around it.

(FWIW, I think that the multiprocessing+logging combination causing 
an atexit function to be registered is a hideous abomination, but 
then I'm not fond of the logging module to begin with.  Modifying 
global state during module import is just plain evil.)