The PEAK Developers' Center   PythonEggs UserPreferences
 
HelpContents Search Diffs Info Edit Subscribe XML Print View
 The PEAK Developers' Center   FrontPage   RecentChanges   TitleIndex   WordIndex   SiteNavigation   HelpContents 
The following 720 words could not be found in the dictionary of 50 words (including 50 LocalSpellingWords) and are highlighted below:
0a3   0install   2a1   2p1   2rc5   9b1   Accessing   Active   All   Also   An   Any   Automatic   Bar   Bicking   Building   But   By   Center   Check   Cleanup   Contents   Control   Current   Declaring   Dependencies   Dependency   Dev   Develoment   Developing   Discovery   Distribution   Dubner   Easy   Eclipse   Edit   Eggs   Ex   Extensions   Fast   File   For   Found   Guide   Here   However   Ian   If   Implementation   Importer   In   Install   Instead   Is   It   Jars   Java   June   Luckily   Manager   Many   Michael   Mode   Move   Name   Normally   Not   Note   Now   Once   One   Operations   Optional   Or   Other   Overview   Package   Packages   Pkg   Plus   Project   Protocol   Pure   Python   Pythons   Questions   Quick   Rename   Requirement   Resources   Running   Sample   See   Session   Set   Setuptools   So   Some   Source   Status   Subversion   Table   That   The   Then   There   These   They   Thingy   This   To   Trac   Twisted   Unix   Using   Usually   Version   Web   What   When   Win32   Windows   With   You   Zero   about   absolutely   access   account   activestate   actually   add   added   adding   addition   additional   advertised   after   all   allow   allowing   allows   along   already   also   alternative   always   an   and   another   answer   any   anyway   anywhere   application   applications   approach   appropriate   are   as   ascending   ask   aspn   assume   at   attempt   attempting   attributes   author   automatically   available   avoid   back   basically   bdist   be   because   been   before   being   below   benefits   beta   better   between   binary   both   brackets   build   building   built   bundled   bundling   but   by   cache   call   calling   calls   can   case   cause   change   check   checked   clarity   clean   cleanups   close   code   com   combined   come   comma   command   commmand   common   comparison   compatible   complete   complex   components   concept   condition   conditions   conf   config   consist   consult   contain   contained   containing   contains   contents   control   convenient   copied   costly   could   couldn   couple   course   covered   create   creates   current   data   declare   declaring   depend   dependencies   dependency   depends   deposited   desired   details   develop   developing   did   didn   differ   difference   different   dig   direct   directly   directories   directory   dirname   dist   distributed   distributing   distribution   distributions   do   docs   documentation   does   doing   don   download   easiest   easy   edgewall   edit   effect   egg   eggs   either   elsewhere   embody   enable   end   enough   entry   environment   error   especially   even   every   exactly   example   examples   existing   exit   expect   expects   extensible   extension   extensions   extract   extracted   extras   fact   fail   falling   falls   far   features   few   file   filename   filenames   files   find   fix   foobar   for   format   formats   found   framework   frameworks   from   function   generated   get   give   great   hand   has   have   haven   having   head   here   higher   how   however   huge   idea   identifying   if   ignore   implementation   import   in   include   included   including   indepenent   information   insensitive   inside   install   installation   installed   installer   installing   instead   instructions   interfacing   into   is   isn   issues   it   its   jar   join   just   keep   kind   know   knows   left   lets   libraries   library   like   link   list   listed   lists   ll   local   locate   located   locates   locating   long   longer   look   machine   mail   main   make   makes   manage   management   manager   managing   manipulate   manipulating   manipulation   manually   manuals   matches   matching   meaningful   met   metadata   meth   might   mind   minor   modern   module   modules   moment   monolithic   more   most   multiple   must   name   named   names   namespace   need   needed   needs   net   newer   newest   next   no   non   normal   not   note   nothing   now   numbering   object   of   offer   offers   official   older   on   one   only   open   operation   operators   opposed   option   optional   optionally   options   or   order   org   os   other   our   out   overviews   overwritten   own   package   packaged   packages   packed   page   pair   parent   pass   patch   path   peak   pep   peps   perfectly   persistent   pipermail   pkg   place   placed   placing   plain   platform   plugins   point   points   possible   practice   presumably   primary   principle   probably   problem   process   program   project   projects   protocol   provide   provides   providing   pure   put   putting   pyd   python   quality   question   quick   raise   raised   ranges   rarely   re   read   ready   really   reason   reasons   rebuild   recommend   reference   release   remain   replacd   replace   replaces   requested   require   required   requirement   requirements   requiring   resource   resources   resulting   returning   rewritten   right   run   running   runtime   sake   same   sample   sandbox   satisfied   says   scale   scanned   schemes   script   scripts   search   section   security   see   selected   separate   separated   separator   setup   setuptools   several   should   side   sig   similar   simple   simply   single   site   slashes   snippet   so   software   solution   some   something   sorted   source   spawn   special   specific   specify   spelling   split   square   stable   standardized   step   stick   still   straight   stream   string   strings   style   subdirectories   subdirectory   subversion   such   sucks   suggested   supply   support   suppose   sure   svn   sys   system   systems   target   techniques   telecommunity   tell   temporary   than   that   the   their   them   themselves   then   theoretically   there   these   they   think   this   those   though   time   timestamp   timezone   timezones   tips   to   tools   trac   track   trap   treats   trunk   try   twisted   typical   typically   typing   under   understands   unfortunate   unfortunately   unknown   unless   unlike   unpack   until   up   updating   usable   use   used   uses   using   valid   variable   ve   version   versions   via   want   wants   way   we   web   well   what   when   where   which   whose   wide   will   win32api   wish   with   within   without   wondering   work   working   would   wrappers   you   your   zero   zip   zipfile   zope  

Clear message

The Quick Guide to Python Eggs

NOTE: If all you want to do is install a project distributed as an .egg file, head straight to the Easy Install page. EasyInstall makes installing Python code as easy as typing easy_install SomeProjectName. You don't need to read this page unless you want to know more about how eggs themselves work.

Table of Contents

Overview

"Eggs are to Pythons as Jars are to Java..."

Python eggs are a way of bundling additional information with a Python project, that allows the project's dependencies to be checked and satisfied at runtime, as well as allowing projects to provide plugins for other projects. There are several binary formats that embody eggs, but the most common is '.egg' zipfile format, because it's a convenient one for distributing projects. All of the formats support including package-specific data, project-wide metadata, C extensions, and Python code.

The easiest way to install and use Python eggs is to use the "Easy Install" Python package manager, which will find, download, build, and install eggs for you; all you do is tell it the name (and optionally, version) of the Python project(s) you want to use.

Python eggs can be used with Python 2.3 and up, and can be built using the setuptools package (see the Python Subversion sandbox for source code, or the EasyInstall page for current installation instructions).

The primary benefits of Python Eggs are:

There are also other benefits that may come from having a standardized format, similar to the benefits of Java's "jar" format.

Using Eggs

If you have a pure-Python .egg file that doesn't use any in-package data files, and you don't mind manually placing it on sys.path or PYTHONPATH, you can use the egg without installing setuptools. For eggs containing C extensions, however, or those that need access to non-Python data files contained in the egg, you'll need the pkg_resources module from setuptools installed. For installation instructions, see the EasyInstall page.

In addition to providing runtime support for using eggs containing C extensions or data files, the pkg_resources module also provides an API for automatically locating eggs and their dependencies and adding them to sys.path at runtime. (See the API documentation and setuptools documentation for details.)

With this support, you can install and keep multiple versions of the same package on your system, with the right version automatically being selected at runtime. Plus, if an egg has a dependency that can't be met, the runtime will raise a DistributionNotFound error that says what package and version is needed.

By the way, in case you're wondering how you can tell a "pure" (all-Python) egg from one with C extensions, the difference is that eggs containing C extensions will have their target platform's name at the end of the filename, just before the .egg. "Pure" eggs are (in principle) platform-indepenent, and have no platform name. If you're using the pkg_resources runtime to find eggs for you, it will ignore any eggs that it can tell are not usable on your platform or Python version. If you're not using the runtime, you'll have to make sure that you use only compatible eggs.

Once you have the runtime installed, you need to get your desired egg(s) on to sys.path. You can do this manually, by placing them in the PYTHONPATH environment variable, or you can add them directly to sys.path in code. This approach doesn't scale well, however, because as you need additional eggs, you'll be managing a longer and longer PYTHONPATH or sys.path by hand. Not only that, but you'll have to manually keep track of all the eggs needed by the eggs you're using! Luckily, there is a better way to do it.

Automatic Discovery

The better way to manage your eggs is to place them in a directory that's already on sys.path, such as site-packages, or the directory that your application's main script is in, or a directory that you'll be adding to PYTHONPATH or sys.path. Then, before attempting to import from any eggs, use a snippet of code like this:

from pkg_resources import require
require("FooBar>=1.2")

This will search all sys.path directories for an egg named "FooBar" whose release version is 1.2 or higher, and it will automatically add the newest matching version to sys.path for you, along with any eggs that the FooBar egg needs. (A note about versions: the egg runtime system understands typical version numbering schemes, so it knows that versions like "1.2a1" and "1.2rc5" are actually older than the plain version "1.2", but it also knows that versions like "1.2p1" or "1.2-1" are newer than "1.2".)

You can specify more than one requirement when calling require(), and you can also specify more complex version requirements, like require("FooBar>=1.2", "Thingy>1.0,!=1.5,<2.0a3,==2.1,>=2.3"). Requirement strings basically consist of a distribution name, an optional list of "options" (more on this in a moment), and a comma-separated list of zero or more version conditions. Version conditions basically specify ranges of valid versions, using comparison operators. The version conditions you supply are sorted into ascending version order, and then scanned left to right until the package's version falls between a pair of > or >= and < or <= conditions, or exactly matches a == or != condition.

Note, by the way, that it's perfectly valid to have no version conditions; if you can use any version of "FooBar", for example, you can just require("FooBar"). Distribution names are also case-insensitive, so require("foobar") would also work, but for clarity's sake we recommend using the same spelling as the package's author.

Some eggs may also offer "extras" - optional features that, if used, will need other eggs to be located and added to sys.path. You can specify zero or more options that you wish to use, by placing a comma-separated list in square brackets just after the requested distribution name. For example, the "FooBarWeb" web framework might offer optional FastCGI support. When you require("FooBarWeb[FastCGI]>=1.0"), the additional eggs needed to support the FastCGI option will also be added to sys.path. (Or, if one of them isn't found, a pkg_resources.DistributionNotFound error will be raised, identifying what dependency couldn't be satisfied.)

To find out what options an egg offers, you should consult its documentation, or unpack and read its EGG-INFO/depends.txt file, which lists an egg's required and optional dependencies.

(Note: the pkg_resources module does not automatically look for eggs on PyPI or download them from anywhere; any needed eggs must already be available in a directory on sys.path, or require() will raise a DependencyNotFound error. You can of course trap this error in your code and attempt to find the needed eggs on PyPI or elsewhere. If you want to automatically install dependencies for a project you're working on, you should probably build it using setuptools, which lets you declare dependencies where they can be found by tools like EasyInstall. Setuptools is also needed in order to build eggs.)

Building Eggs

To build an egg from a package's setup.py, you'll need to have setuptools installed. If you haven't already installed it in order to use the pkg_resources runtime, just check it out of Python's Subversion sandbox and run setup.py install to install it, or see the EasyInstall page's installation instructions (). Now you're ready to build eggs.

Edit the target package's setup.py and add from setuptools import setup such that it replaces the existing import of the setup function. Then run setup.py bdist_egg.

That's it. A .egg file will be deposited in the dist directory, ready for use. If you want to add any special metadata files, you can do so in the SomePackage.egg-info directory that bdist_egg creates. ("SomePackage" will of course be replacd by the name of the package you're building.) Any files placed in this directory are copied to an EGG-INFO directory within the egg file, for use at runtime. Other metadata files are automatically generated for you, so don't edit them, as the next time you run a setup command they may be overwritten with the automatically generated versions.

Note: packages that expect to find data files in their package directories, but which do not use either the PEP 302 API or the pkg_resources API to find them will not work when packaged as .egg files. One way you can check for this is if the .egg file contains data files, and the package is using __file__ to find them. You'll need to then patch the package so it uses pkg_resources.resource_filename() or one of the other resource_* APIs instead of __file__. See the section on Accessing Package Resources, below, for more information about updating packages to use the resource management API instead of __file__ manipulation.

Declaring Dependencies

Some eggs need other eggs to function. However, there isn't always a meaningful place for a library to call require(), and in any case a library's source code is rarely the place to declare its version dependencies. So setuptools allows you to declare dependencies in your project's setup script, so that they will be bundled inside the egg's metadata directory, and both the runtime and EasyInstall can then automatically find the additional eggs needed, adding them to sys.path when your project is installed or requested at runtime via require(). (Note: the EasyInstall program will find and download dependencies from the internet automatically, but for security reasons simply using require() in Python code does not do this. require() only locates eggs that are in directories on the local machine that are listed in sys.path)

For more information on declaring your project's dependencies, see the setuptools documentation.

Developing with Eggs

Here are a few quick tips and techniques about developing software using eggs' features. These are just overviews, though, and you should dig into the complete setuptools documentation and API documentation manuals if there's not enough information here.

Running Eggs from Source

So far, we've only covered how to use eggs that have actually been installed, by building them with the distutils and then putting them in a directory on sys.path. (Note: EasyInstall can download source distributions, automatically build eggs from them, and install the eggs for you, with just a single command -- even if the package's author did nothing special to support Python Eggs. Check it out.)

But what if you are developing a package and working from source code? You don't want to have to rebuild the egg every time you make a change to the source code. But, you have code in your script or application that calls require() and expects the egg you're developing to be available. For example, see this question from Ian Bicking about working with packages checked out from subversion, but not built as eggs.

If you're using setuptools, the answer is simple: run "setup.py develop" to create a "source egg" - a special link to your project's source directory, combined with wrappers for your source scripts. See the setuptools documentation under "Develoment Mode" and also the "develop" commmand reference for more details.

Accessing Package Resources

Many modern Python packages depend on "resources" (data files) that are included with the package, typically placed within the package's subdirectory in a normal installation. Usually, such packages manipulate their modules' __file__ or __path__ attributes in order to locate and read these resources. For example, suppose that a module needs to access a "foo.config" file that's in its package directory. It might do something like:

foo_config = open(os.path.join(os.path.dirname(__file__),'foo.conf').read()

However, when code like this is packed inside a zipfile, it can no longer assume that __file__ or __path__ contain filenames or directory names, and so it will fail.

Packages that access resource files, and want to be usable inside a zipfile (such as a .egg file), then, must use the PEP 302 get_data() extension (see under "Optional Extensions to the Importer Protocol") before falling back to direct __file__ access.

Using this protocol can be complex, however, so the egg runtime system offers a convenient resource management API as an alternative. Here's our "foo_config" example, rewritten to use the pkg_resources API:

from pkg_resources import resource_string
foo_config = resource_string(__name__, 'foo.conf')

Instead of manipulating __file__, you simply pass a module name or package name to resource_string, resource_stream, or resource_filename, along with the name of the resource. Normally, you should try to use resource_string or resource_stream, unless you are interfacing with code you don't control (especially C code) that absolutely must have a filename. The reason is that if you ask for a filename, and your package is packed into a zipfile, then the resource must be extracted to a temporary directory, which is a more costly operation than just returning a string or file-like object.

Note, by the way, that if your resources include subdirectories of their own, you must specify resource names using '/' as a path separator. The resource API will replace slashes with a platform-appropriate filename, if in fact filenames are being used (as opposed to e.g. zipfile contents). For more examples and information, see the setuptools documentation and the API documentation for pkg_resources.

Implementation Status

The runtime implementation has been stable for some time now, and the EasyInstall package manager is now close to beta quality. The runtime still has a couple of minor issues, which should probably be in the official documentation:

  • The extract process treats the file's timestamp in the zipfile as "local" time with "unknown" DST. It's theoretically possible that a DST change could cause the system to think that the file timestamp no longer matches the zip timestamp. Also, the resulting Unix-style timestamp for the extracted file may differ between systems with different timezones. This is an unfortunate side effect of the fact that the zip file format does not include timezone information or a UTC timestamp.
  • Cleanup on Windows doesn't work, because the .pyd's remain in use as long as some Python process using them is still running. An application that really wants to clean up on exit can presumably spawn another process to do something about it, but that kind of sucks, and doesn't account for the fact that another process might still be using the file anyway. (Michael Dubner suggested using HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\Session Manager\\FileRenameOperations to fix the Windows problem, but unfortunately didn't give any details or sample code.) Sample code at http://aspn.activestate.com/ASPN/docs/ActivePython/2.2/PyWin32/win32api__MoveFileEx_meth.html, but in practice we really can't do this without requiring the PyWin32 extensions, which isn't such a great idea. So, the simple solution is to just avoid doing cleanups, and instead stick with a persistent cache directory.

Questions

What's the difference between Python Eggs and Zero Install (http://0install.net)?

A: Zero Install is a Unix only software package installer that does not work on Windows.

PythonPowered
EditText of this page (last modified 2008-11-15 17:51:33)
FindPage by browsing, title search , text search or an index
Or try one of these actions: AttachFile, DeletePage, LikePages, LocalSiteMap, SpellCheck