[PEAK] Breaking up PEAK

[Phillip J. Eby]

At 09:30 AM 6/6/2005 -0400, Erik Rose wrote:
>I'd worry less about trying to normalize the structure of PEAK as if it
>were a DB and more about the interaction with users' brains. I think
>the breakdown you suggested—3 clumps of about 5 packages each—is good;
>it fits ESR's definition of compactness
>(http://www.catb.org/~esr/writings/taoup/html/ ch04s02.html#compactness), 
>and each clump even fits in the
>5-plus-or-minus-2 size of human working memory. If you break things
>down further, I think new users will again descend into panic, saying
>"I have no idea what this thing does; there are 500 pieces!"

That's just it, though; if things can stand on their own, they can stand on 
their own.  For example, there is an assortment of testing utilities in 
peak.util: mockdb, mockets, and unittrace.  The odds that anybody will do 
much of anything with them in peak.util is pretty much nil, but if I 
bundled them as a mini-project (perhaps called "MockTesting"), it'd 
probably get some interest from TDD-oriented folks.

Similarly, nobody's going to install PEAK just to get peak.util.uuid and 
friends, but as a "PyUUID" package that provides a cross-platform API for 
the draft UUID/GUID spec, and that's another thing people would download.

Neither MockTesting nor PyUUID would have the same audience as the rest of 
PEAK, and there's absolutely no connotation that you have to learn these 
packages in order to "learn PEAK".

There are many other use-case oriented groupings of PEAK's contents that 
could be spun off to live as semi-independent packages.  The purpose here 
isn't to "normalize" PEAK, but rather to make its functionality accessible 
to a wider audience (in the overall sense) by having individual packages be 
available for narrower audiences.

An interesting side effect, by the way, is that it's going to drive some 
innovations in the PEAK core machinery.  For example, if PEAK splits up, 
I'll really want to have a way for 'peak.ini' to be split across egg 
boundaries, and ways for eggs to easily trigger registrations of certain 
functionality when another package is present.

For example, peak.security as it stands today could depend solely on 
today's PyProtocols, if it didn't register certain functions with 
peak.binding to support attribute metadata.  But if I made 
peak.util.imports into an "ImportTools" package, then I could release an 
"ACLRules" package, depending on ImportTools and RuleDispatch ('dispatch' 
from PyProtocols), by using the 'whenImported()' functionality in 
ImportTools to only register those functions if and when peak.binding.api 
is imported.

On the other hand, the idea of having such intricate inter-package 
dependencies also sounds pretty scary, considering that PEAK could easily 
break into a *lot* of separately distributed packages.  On the other hand, 
almost everything I've named above besides RuleDispatch is pretty darn 
stable already, so it's not like you'll notice most of the time.

However, people creating third-party distributions of PEAK are probably 
going to go through some initial pain as we make the switch.  (OTOH, there 
are probably few people doing that with PEAK as a whole, but many more such 
people who are interested in doing it for exactly one of PEAK's many packages.)


>Btw, I'm sure you've considered this, but if you're going to switch
>version control systems, now would be a natural time. Maybe one or two
>of them has some tricks to ease cross-package versioning.

Yes, going to Subversion would probably help a lot with the reshuffling, as 
far as retaining directory histories once stuff starts moving around.  The 
main drawback I see is that Subversion doesn't do repository symlinks: when 
you copy stuff it's effectively a branch.  On the other hand, that will 
certainly encourage making stuff cleanly separable.

I've been using Subversion at OSAF for a few weeks now, but it has some 
quirks that I find seriously annoying.  For one thing, the whole "mime 
types and EOL type get set on the client" thing *really* ticks me 
off.  That should be something you configure on the repository, for 
heaven's sake.  Its HTML notice emails don't play well with pipermail, 
either, and it's a serious pain to build a server for it.

On the plus side, however, there's a really nice Wiki/case tracking system 
(Trac) that's written in Python and integrates with Subversion, and I could 
probably go for replacing our current Wiki software with it at some point.

Of course, finding hours in the day to do any of this is always a 
problem.  Currently, my spare time is still being filled by working on 
setuptools and EasyInstall themselves, in order to enable this whole 
deal.  I think the coding tasks could probably be completed in a few more 
weekends, but documentation, specifications, support, and advocacy for the 
"eggs" concept and implementation are likely to be ongoing for some time.  :(