[PEAK] Breaking up PEAK

[Phillip J. Eby]

Now that Python Eggs are much closer to hatching, it's probably time to 
start thinking about breaking up PEAK into separately-installable packages, 
and unbundling some of its dependencies.

Here are some preliminary thoughts on how the breakup might go.  This is 
all extremely vague at the moment, so I'm hoping this will spur some 
discussion.  By the way, if you haven't looked at all of the following 
pages in the last 24 hours or so, you might want to glance at them to catch 
up on what this is about:

     http://dirtsimple.org/2005/05/easyinstall-new-era-in-python-package.html
     http://peak.telecommunity.com/DevCenter/EasyInstall
     http://peak.telecommunity.com/DevCenter/PythonEggs


Primary Frameworks
------------------

PyProtocols - dispatch, protocols

PEAK-Components - All of the core, including binding, naming, config, 
model, and util (dependencies=PyProtocols)

PEAK-Running - running (dependencies = PEAK-Components
PEAK-Events - events (dependencies = PEAK-Running, ZConfig (for EventDriven))
PEAK-Security - security (dependencies = PEAK-Components)
PEAK-Storage - storage (dependencies = PEAK-Components)
PEAK-Web - (dependencies = PEAK-Security, PEAK-Events(?), WSGIRef, optional 
FCGIApp)


Tool Bundles
------------
PEAK-N2
PEAK-Supervisor  (dependencies = PEAK-Events, ZConfig, optional FCGIApp)
PEAK-Help
PEAK-LocalServer  (dependencies = PEAK-Running, WSGIRef)
PEAK-DDT


Other
-----
WSGIRef
FCGIApp
PEAK-Unstable - metamodels, net, query


My initial feeling about this list is that it's both too coarse and too 
fine-grained at the same time.  It's too coarse because for example both 
peak.util and peak.running are umbrella packages containing many things 
that could quite reasonably be split out individually.  But it's also too 
fine-grained, because it doesn't seem like some of these things deserve 
packages of their own.  Should LocalServer maybe be bundled with 
PEAK-Web?  But what if you just want to use it as a PEAK WSGI server?

I have the nagging suspicion that attempting to split PEAK up into chunks 
is going to force some re-evaluation of where certain things live, because 
a use-case driven view shows that the groupings people need aren't really 
related to the groupings that have been embodied in the software.  I'm also 
uneasy because keeping track of versioning and release information for 
*fifteen* packages (versus two now) seems a little overwhelming.

On the other hand, the potential advantages to the split are pretty 
substantial too.  First, although all of these things need documentation 
and release management, they are all substantially smaller packages.  So, 
looking at something like PEAK-Security, considering the documentation that 
already exists, turning it into a nice stable standalone package doesn't 
sound too difficult.  Indeed, with small enough packages, it's possible 
that volunteers could more easily write add-on documentation.  And, people 
encountering these small packages don't run into that "trying to learn 
PEAK" (as in *all* of it) barrier.

Further, it will be more obvious to people just how much functionality is 
available in the PEAK "family of products" -- frankly I myself am amazed 
whenever I realize that I don't even know myself how much stuff is in 
there.  There's more than I can keep track of consciously any more!  So, as 
a branding exercise, there will definitely be more heft to the apparent 
functionality, and less "what the heck is PEAK anyway?"-type 
questions.  (Of course, those will become "what's this 
binding/naming/config stuff?" questions, but that's quite okay.)

Eggs aren't actually ready for this unbundling just yet, 
unfortunately.  There are two main pieces missing, besides the fact that 
the runtime isn't very robust yet.  First, there's no equivalent for 
os.listdir() for resources in a zipfile (which peak.web needs in order to 
allow site components to be packaged as eggs).  Second, there's no build 
support in setuptools yet for breaking a distribution into multiple eggs, 
each with its own metadata (e.g. version, dependencies, description, 
etc.).  Without these pieces, and a more robust runtime, PEAK isn't quite 
ready to break up yet.  But we're definitely getting a lot closer.

Your thoughts?