Ignore changes in the amount of whitespace
Differences between version dated 2006-01-30 01:11:51 and 2008-11-15 17:51:33
(spanning 3 versions)
Deletions are marked like this.
Additions are marked like this.
The Quick Guide to Python Eggs
==============================
**NOTE: If all you want to do is install a project distributed as an .egg file**, head straight to the `Easy Install page <EasyInstall>`_. EasyInstall makes installing Python code as easy as typing ``easy_install SomeProjectName``. You don't need to read this page unless you want to know more about how eggs themselves work.
.. contents:: **Table of Contents**
Overview
--------
"Eggs are to Pythons as Jars are to Java..."
Python Eggs are zipfiles using the ``.egg`` extension, that support including data and C extensions as well as Python code. They can be used with Python 2.3 and up, and can be built using the `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_ package (see the `Python Subversion sandbox <http://svn.python.org/projects/sandbox/trunk/setuptools/>`_ for source code, or the `EasyInstall page <EasyInstall#installing-easy-install>`_ for current installation instructions). Once the implementation is far enough along, we plan to propose it for inclusion with the standard library beginning in Python 2.5.
Python eggs are a way of bundling additional information with a Python project, that allows the project's dependencies to be checked and satisfied at runtime, as well as allowing projects to provide plugins for other
projects. There are several binary formats that embody eggs, but the most common is '.egg' zipfile
format, because it's a convenient one for *distributing* projects. All of the formats support including package-specific data, project-wide metadata, C extensions, and Python code.
The easiest way to install and use Python eggs is to use the `"Easy Install" <EasyInstall>`_ Python package manager, which will find, download, build, and install eggs for you; all you do is tell it the name (and optionally, version) of the Python project(s) you want to use.
Python eggs can be used with Python 2.3 and up, and can be built using the `setuptools <http://peak.telecommunity.com/DevCenter/setuptools>`_ package (see the `Python Subversion sandbox <http://svn.python.org/projects/sandbox/trunk/setuptools/>`_ for source code, or the `EasyInstall page <EasyInstall#installing-easy-install>`_ for current installation instructions).
The primary benefits of Python Eggs are:
* They enable tools like the `"Easy Install" <EasyInstall>`_ Python package manager (**NEW!**)
* They enable tools like the `"Easy Install" <EasyInstall>`_ Python package manager
* They are a "zero installation" format for a Python package; no build or install step is required, just put them on ``PYTHONPATH`` or ``sys.path`` and use them
* .egg files are a "zero installation" format for a Python package; no build or install step is required, just put them on ``PYTHONPATH`` or ``sys.path`` and use them (may require the runtime installed if C extensions or data files are used)
* They can include package metadata, such as the other eggs they depend on
Using Eggs
----------
If you have a pure-Python egg that doesn't use any in-package data files, and you don't mind manually placing it on ``sys.path`` or ``PYTHONPATH``, you can use the egg without installing ``setuptools``. For eggs containing C extensions, however, or those that need access to non-Python data files contained in the egg, you'll need the ``pkg_resources`` module from ``setuptools`` installed. For installation instructions, see the `EasyInstall page`_.
If you have a pure-Python .egg file that doesn't use any in-package data files, and you don't mind manually placing it on ``sys.path`` or ``PYTHONPATH``, you can use the egg without installing ``setuptools``. For eggs containing C extensions, however, or those that need access to non-Python data files contained in the egg, you'll need the ``pkg_resources`` module from ``setuptools`` installed. For installation instructions, see the `EasyInstall page`_.
In addition to providing runtime support for using eggs containing C extensions or data files, the ``pkg_resources`` module also provides an API for automatically locating eggs and their dependencies and adding them to ``sys.path`` at runtime. (See the `API documentation <http://peak.telecommunity.com/DevCenter/PkgResources>`_ and `setuptools documentation <http://peak.telecommunity.com/DevCenter/setuptools>`_ for details.)
Implementation Status
---------------------
Egg Builder (``bdist_egg`` command)
* (DONE) format suitable for use with zipimport if no resource access (including extensions) is needed
* (DONE) add required ``EGG-INFO`` metadata directory for storing info about the egg itself
* (DONE) include ``PKG-INFO`` metadata file in the ``EGG-INFO``
* (DONE) Hand-made EGG-INFO files can be placed in an ``PackageName.egg-info`` directory of the distribution
* (DONE) By default, all .so/.dll/.pyd files are "eager" (listed in ``EGG-INFO/native_libs.txt``)
* (DONE) build process generates .py/.pyc/.pyo stubs to load extensions via ``pkg_resources.resource_filename()`` calls
* (DONE) Supply most metadata via ``setup()`` arguments instead of files, to make syntax checking easier
* (DONE) Allow ``eager_resources`` to be specified via a ``setup()`` keyword instead of a file.
* (DONE) needs option to not include source code in .egg (default is to include source, for IDE's, debugging, etc.)
Resource Manager Open Issues
* (DONE) Egg-info can include list of "eager resources" which must be extracted all-at-once if any of them are extracted (listed in ``EGG-INFO/eager_resources.txt``)
* (DONE) Default cache location should be settable via ``PYTHON_EGGS_CACHE`` environment variable, with fallback to either ``~/.python-eggs`` or a "Python-Eggs" subdirectory of the Windows "application data" directory.
* (DONE) The extraction process needs to be race-condition safe.
* (DONE) Fix home directory location on Windows
* (DONE) Allow listing or globbing of resources, and unpack entire resource directories if their ``resource_filename()`` is requested from a zipped distribution
The runtime implementation has been stable for some time now, and the EasyInstall package manager is now close to beta quality. The runtime still has a couple of minor issues, which should probably be in the official documentation:
* The extract process treats the file's timestamp in the zipfile as "local" time with "unknown" DST. It's theoretically possible that a DST change could cause the system to think that the file timestamp no longer matches the zip timestamp. Also, the resulting Unix-style timestamp for the extracted file may differ between systems with different timezones. This is an unfortunate side effect of the fact that the zip file format does not include timezone information or a UTC timestamp.
* Cleanup on Windows doesn't work, because the .pyd's remain in use as long as some Python process using them is still running. An application that really wants to clean up on exit can presumably spawn another process to do something about it, but that kind of sucks, and doesn't account for the fact that another process might still be using the file anyway. (Michael Dubner suggested using ``HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\Session Manager\\FileRenameOperations`` to fix the Windows problem, but unfortunately didn't give any details or sample code.) Sample code at http://aspn.activestate.com/ASPN/docs/ActivePython/2.2/PyWin32/win32api__MoveFileEx_meth.html, but in practice we really can't do this without requiring the PyWin32 extensions, which isn't such a great idea. So, the simple solution is to just avoid doing cleanups, and instead stick with a persistent cache directory.
Namespace Packages
* (DONE) Need to implement hook in ``Distribution.install_on()`` to fix up existing NS packages, and to flag namespace packages identified from distribution metadata (e.g. ``EGG-INFO/namespace_packages.txt``).
* (DONE) Need ``namespace_package()`` API in ``pkg_resources``
* (DONE) Implement thread-safety by holding the import lock while manipulating namespace packages
Dependency/Discovery
* (DONE) Given a sys.path entry, iterate over available distributions, either .egg directories, .egg files, or distributions identified by .egg-info directories.
* (DONE) Version spec syntax (needs to support specifying "extras" in order to implement optional dependencies)
* (DONE) ``require()`` API call to automatically resolve dependencies and add them to sys.path
* (DONE) ``EGG-INFO.in/depends.txt`` (feature->requirements mapping specifying distribution+version)
Questions
---------
* (DONE) Add support for extending the distribution finder for non-filesystem sys.path entries (by adapting PEP 302 importer objects)
What's the difference between Python Eggs and Zero Install (http://0install.net)?
* (DONE) Add a zip distribution finder to support "baskets" -- i.e., multiple distributions inside a single .egg file.
A: Zero Install is a Unix only software package installer that does not work on Windows.
