[TransWarp] Namespaces stabilizing; new stuff planned

[Phillip J. Eby]

At 03:25 PM 6/10/02 -0500, Phillip J. Eby wrote:
>
>This will leave us with the following top-level subpackages:
>
>peak.api
>peak.binding
>peak.model
>peak.metamodels
>peak.naming
>peak.util
>

And we've now settled on "peak.running" for what was previously going to be
peak.deployment:

peak.running.logs
peak.running.processes
peak.running.config

I18N will probably be spread throughout the system, rather than be a
specialized package.  We may end up with peak.running.locales or
peak.running.config.locales or some such for various needs as they come up.
 The miscellaneous Unicode related functions we have right now will go in
peak.util, and get called from other packages as part of their normal
operation.  They may also get put in a peak.api.i18n module for convenience
access, especially if they're frequently used gettext-ish functions.

The peak.storage package plan is shaping up as well, so far there will be
something like:

peak.storage.connections
peak.storage.racks
peak.storage.records
peak.storage.sql
peak.storage.ldap

We're refactoring the ManagedConnections from TW to a more general "managed
object" concept.  ManagedConnection won't be used as a base class, instead,
you'll create ManagedConnections like this:

storage.ManagedConnection(someConnectionManager(args))

Actually, that's not quite correct, normally you'll just use a bindTo()
operation and the naming services will get you the ManagedConnection.

Anyway, ManagedConnection will be a proxy that uses methods provided by the
factory to wrap the "real" connection object, which will be DBAPI or LDAP
connection objects.

One of the tricky bits in dealing with cross-backend Python DBAPI is that
each driver defines its own exceptions, so your error handling code is
bound to the module that the driver comes from!  (To name just one issue.)
Anyway, connectionManagers will be "drivers" that supply information to
make the underlying DB objects conform to standardized interfaces.  The
ManagedConnection will supply wrapping services like the general structure
of retries, closing/reopening connections, and general exception handling.
 The idea is that ManagedConnection should be suitable for doing this kind
of wrapping around any "connection-like" object, not just databases.
(Although that sort of calls into question its placement in peak.storage,
so the core parts may move into util if they become generic enough.)

I'm going to do a bit more reading on JAC before I implement this; there
are some definite similarities between how they do method wrapping and what
I want to do.  But at the same time I don't want to drift too far into AOP
theory rather than getting the connection stuff working soon.  :)

Another thing that the wrapping of managed connections will be for is to
make SQL and LDAP record types conform to one another.  The Python LDAP
module returns results in the form of (stringDN, attributeDict) tuples,
with lists for attribute values.  DBAPI providers return tuples.
peak.storage objects will always return records based on the new "struct"
type, which allows an object to "smell like" a tuple and a dictionary at
the same time.  The DN of an LDAP record will become a field along with the
rest, and field values will be made into tuples instead of lists, making
the resulting data structures completely immutable.  This is the best way
to guarantee that there will be no inadvertent state coupling via shared
data.  And it makes them guaranteed thread-safe, too.

There's a *lot* of stuff in this space still to be finalized, however.  I
haven't yet addressed issues of standardizing parameter API's, cursor
management, etc.

Once there's a decent LDAP connection object with proper records, we'll
want to implement some type of LDAP context object for the naming package.
There are other context providers we want to do too, mostly related to
configuration stuff, that may come first.

In fact, the next issue up for Ty and I is finalizing the plan for the
configuration stack, which will by default include the ability to set
configuration items via files, the environment, and variables in __main__
(i.e. defined in your startup script).  We just haven't finalized the
precise precedence order and semantics as yet.