[TransWarp] PEAK Tutorial

[Phillip J. Eby]

At 11:20 PM 11/13/02 +0100, Ulrich Eck wrote:
>Hi Phillip,
>
>>Understood.  I'd like to have some useful example applications, but until
>>we finish the rest of the 0.5 release plans, there aren't many generally
>>useful programs we can write that show off the framework.  The binding,
>>config, and naming packages are extremely useful as *part* of an
>>application, and could be used to develop sysadmin-ish scripts and tools,
>>but "real" apps want a domain model and storage capabilities.
>
>at least config and naming are really new to us (didn't have such a thing
>with Transwarp) and some (complete) snippets would help alot to
>get into it without having to dig through all the code guessing how
>it could work.

The main usage for the naming system is to have a way to decouple "what" 
you want, from how it must be accessed.  Thus, one may reference a name 
like "myApplicationDB", and have that translated to an actual database 
connection.  Logfiles, lockfiles, and various other kinds of services are 
useful to look up by name as well.  In theory, if you have a distributed 
object naming service, you could implement a peak.naming "context" class 
which could be used to retrieve objects from the naming service.

At a more detailed level, the naming package also provides support for 
"addresses", which are generally more specific than names.  Instead of 
"myApplicationDB", an address might be something like 
"pgsql://user:pass@server/dbname", to specify a particular PostgreSQL 
database.  Addresses are also handled by naming contexts, but for many 
kinds of addresses, it's sufficient to use the "GenericURLContext" and 
simply register a URL scheme class that parses the address and provides a 
default implementation for retrieval.

So far, the naming package only has two contexts existing: AbstractContext 
and GenericURLContext.  Before 0.5, I'd like to have at least one "flat" 
namespace context (e.g. a context that looks up configuration properties) 
and one "hierarchical" namespace context (e.g. a filesystem context that 
gives you a namespace into a file system).  Currently, the two contexts Ty 
and I have built outside of PEAK proper are simple flat contexts that run 
off of configuration files that are of interest or use only in our 
employer's runtime environment, that allow us to refer to existing abstract 
names (e.g. "app1_db") for database connections in that environment.

A big advantage of the peak.naming framework is simply that it defines a 
standardized interface for dealing with, well, namespaces.  An application 
written to use the peak.naming interfaces doesn't need to be changed much 
to deal with different back ends or changing low-level details.  One gives 
the system names, and gets back objects, or gives the system an object and 
tells it what name to store it under.  If you take a look at the JNDI 
tutorials for Java, you'll get an idea of the motivation behind 
peak.naming, but of course peak.naming is *much* simpler to deal with both 
as a user and as a framework extender.  (Although there are still some 
minor bits of Java cruftiness remaining in peak.naming that I haven't yet 
figured out how to purge.)

The configuration package is designed to do two things: 1) provide a 
framework for "placeful" and "lazily immutable" lookups of data by 
interface or property name, and 2) establish specific "root" components 
outside of one's explicit component hierarchy which can supply a place for 
"placeless" lookups.

Both utilities (keyed by Interface objects) and properties (keyed by 
PropertyName strings) are looked up using objects' _getConfigData() 
methods, if present.  If something isn't found, it's looked for in the 
object's parent, and so on until the "top" of the explicit hierarchy is 
reached.  At that point, the system continues the search at the 
"LocalConfig" which corresponds to that root component, and then finally to 
the system-wide "GlobalConfig" object.

This is the basic infrastructure of peak.config.  In addition, peak.config 
offers a variety of components and functions to make the actual 
configuration process easier.  For example, it can parse configuration 
files with a fairly sophisticated, Python-and-PEAK-oriented format, that 
supports inclusions of data in arbitrary user-defined formats.  (See the 
"[peak.config.loaders]" section of the peak.ini file, and the 
config.ISettingLoader interface.)

peak.binding, peak.config, and peak.naming all interconnect with each other 
in fairly intimate ways.  Binding components define _getConfigData() 
methods, for example, and attribute bindings like bindToUtilities and 
bindToProperty call back into the configuration system to look up 
values.  The naming system relies on configuration properties and utilities 
in order to find handlers for different name and address schemes, and to 
control its operation in other ways.  Attribute bindings of course can also 
call into the naming system to look up other components, and the naming 
system components are implemented in terms of binding components.

Where things get really interesting, and it can make your head spin when 
you think about it, is that since the naming system is controlled by the 
configuration system, which in turn is completely contextual, it means that 
you could in principle have two components with a completely different idea 
of what say, "http:" means.  Which is a very good thing, because in the 
same application you might have different components that *need* different 
things from an HTTP connection.  All you need to do is supply appropriate 
configuration "near" the affected objects.

And you can do this either at runtime via the config API's, or you can do 
it using things like binding.Constant() at code-writing time.  Or you can 
put it in a config file...  Or...

Well, guess I should save something for the tutorial, huh?


>We have experience in using the binding package and i can probably
>guess how the storage parts work.

The easiest way to jump into a PEAK package, at least if it's relatively 
mature, is to look at its "interfaces" module.  For example, in the naming 
package you'll see IBasicContext for naming lookups and address spaces, 
IReadContext and IWriteContext for namespaces that can iterate over their 
contents or store things for you, etc.

You needn't even use the source for this, as the online API docs render the 
interfaces almost as thoroughly (and more pleasantly formatted) as the 
source code.


>you have a note about a small script from ty .. would it be possible
>to give this as an example (all needed files) on how config and naming
>works ??

Afraid not, since as I mentioned before they're being used for proprietary 
things and wouldn't be of much use or interest outside anyway.  But the 
"peak.ini" file is a pretty good example of some of what the config system 
can do, and is also a good entry point to seeing how parts of the naming 
system work.  Interestingly, it was Ty's first reading of the "peak.ini" 
file that prompted him to say "This is something I could almost *use*!"


>that would save us a lot of time ..
>and like i said before, we're willing to help writing howto's
>and examples like i did it for Transwarp 0.2
>
>what do you think ??

I'd love to have some help.  But I don't really have any non-proprietary 
uses of PEAK to pass out to anybody right now, and am unlikely to until 
after the model and storage packages are done for 0.5.  So if there's 
anything that you can do to document, based on the code and docs that are 
out there right now, then great.

I tell you what, if you have a naming service that you use right now for 
distributed object stuff, how about you create a naming context based on 
it?  I'll "walk you through" the process of creating one via e-mail.  And 
then it can either get included in PEAK or supplied on the side as an 
example of a naming context.

The first steps for creating a context are:

1. Decide if it will be a Basic, Read, or Write context.  A "basic" context 
can really only look things up, or check if the name exists -- effectively 
__getitem__ and has_key() operations.  A "read" context can also iterate 
(think keys(), values(), items(), etc.), and a "write" context can have 
objects bound or unbound in it (think __setitem__, __delitem__, and rename).

2. Subclass AbstractContext (found in peak.naming.contexts).  Define a 
"_get()" method, based on the instructions found in the source of the base 
class.  This will get you a "basic" context.  Add an __iter__ method if you 
want a read context, and so on.

If you'll take a look at these steps and bring me your questions, I'll help 
you through it.