The PEAK Developers' Center   HelloWorld UserPreferences
 
HelpContents Search Diffs Info Edit Subscribe XML Print View
Version as of 2003-12-09 16:51:12

Clear message


By R. David Murray (rdmurray at bitdance.com) (w/edits by PJE)

Preface

There's a long tradition of teaching programmers how to use programming systems via "Hello, World!" examples. Why should PEAK be left out? I'm going to try to develop a series of stripped-to-the-essentials examples here that provide complete working PEAK programs that demonstrate some of the basic services provided by PEAK. Don't expect to find deep knowledge here; this article is intended to help you get over the first hump of using PEAK, while giving you a little bit of its "flavor".

We'll be using a Unixish command shell for our examples. The percent sign (%) will be used to indicate a command prompt.

(Note to Windows users: if you're not using a Unix-like shell (e.g. Cygwin/Bash), you'll need to change the command lines used here. For example, peak foo bar will become something like c:\Python22\Python.exe c:\Python22\peak foo bar. None of the #! lines will work for you, either, and to set environment variables like PYTHONPATH, you'll need to use SET instead of export, and so on. However, there's nothing Unix-specific about the framework or examples, and your applications will still run on Windows. It's just that the standard Windows shell doesn't offer as many conveniences in these areas. If you have trouble running the examples on Windows, write up your question on the PEAK mailing list, and we'll try to get these examples updated with more Windows-specific command lines.)

Table of Contents

  1. Basic Configuration
    1. Introduction
    2. Running helloworld under PEAK
    3. ".ini" Files and the "runIni" Command
    4. Executable Configuration Files
    5. Making the Application Object Configurable
    6. Binding attributes to configuration
    7. Points to Remember
  2. Command Arguments and Simple Database Access
    1. Command Arguments
    2. Domain Models
    3. Data Managers
      1. Loading data from the flat file
      2. Building the Data Manager subclass
    4. Putting it all Together
    5. Points to Remember
  3. Multiple Commands and Writable Databases
    1. AbstractInterpreter and Bootstrap
    2. peak.running.shortcuts
    3. Storing a New Message: the "for" Subcommand
    4. Storing a New Message: Modifying the Data Manager
    5. Oh no, a bug!
  4. SQL, N2, and Naming
    1. N2
    2. peak.ini, and Naming
    3. Greater Configurability
    4. Reading from the SQL Database
    5. Doing it Right
  5. Exploring the rest of PEAK

1 Basic Configuration

1.1 Introduction

At base, a PEAK program is a Python program. PEAK is a toolkit and framework, which means it provides stuff you can use in your Python programs to get things done more easily and more quickly, and usually more elegantly.

So, the simplest PEAK "Hello, world!" program is also, in a trivial sense, the Python "Hello, world!" program:

    1 print "Hello, World!"
This, however, is not very interesting.

One of the things PEAK provides is a framework for giving application programs easy access to their associated configuration data. The goal of this first chapter will be to show how to turn the output of the simple "Hello, world!" program into a configured string. I'm going to get us there step by step, keeping the program in a runnable state at each step (except the first!)

1.2 Running helloworld under PEAK

Let's start by introducing the peak command. For convenience in starting PEAK applications, the peak.running package provides a peak command-line script, that is installed alongside Python when you install PEAK. We'll use the peak script to invoke our application. First, just type:

peak 
at your shell prompt, and see what comes up. You should receive a usage message, something like:
 
Usage: peak NAME_OR_URL arguments... 
 
The 'peak' script bootstraps and runs a specified command object or command 
class.  The NAME_OR_URL argument may be a shortcut name defined in the 
'peak.running.shortcuts' property namespace, or a URL of a type 
supported by 'peak.naming'.  For example, if you have a class 'MyAppClass' 
defined in 'MyPackage', you can use: 
 
    peak import:MyPackage.MyAppClass 
 
to invoke it.  Arguments to the found object are shifted left one position, 
so in the example above it will see 'import:MyPackage.MyAppClass' as its 
'argv[0]'. 
 
The named object must implement one of the 'peak.running' command interfaces, 
or be callable.  See the 'Bootstrap' class in 'peak.running.commands' for 
more details on creating command objects for use with 'peak'.  For the 
list of available shortcut names, see '/usr/lib/python/site-packages/peak/peak.ini'. 
 

(The last line will list the actual path to peak.ini on your system.)

Okay, that looks helpful, although the stuff about command interfaces and shortcuts is perhaps a little intimidating. For now, we'll stick to the easy part, where it said we could use a callable! Let's make a helloworld module, with a HelloWorld function, in helloworld.py:

    1 def HelloWorld():
    2     print "Hello, world!"

Can we run it with the peak script now? Let's try it:

 
% peak import:helloworld.HelloWorld 
Traceback (most recent call last): 
  File "C:\Python22\Scripts\peak", line 7, in ? 
    sys.exit( 
  File "C:\cygwin\home\pje\PEAK\src\peak\running\commands.py", line 303, in __call__ 
    return cmd.interpret(cmd.argv[1]) 
  File "C:\cygwin\home\pje\PEAK\src\peak\running\commands.py", line 624, in interpret 
    raise InvocationError("Name not found: %s" % name) 
peak.running.commands.InvocationError: Name not found: import:helloworld:HelloWorld 
 

Oops. What's wrong? Oh, wait, since helloworld is a module, not a script, it needs to be on the PYTHONPATH. Let's fix that:

 
% export PYTHONPATH=. 
% peak import:helloworld.HelloWorld 
Hello, world! 
 

Nice! But we said we were going to make the message configurable. To do that, we're going to use the peak runIni command.

1.3 ".ini" Files and the "runIni" Command

runIni is one of the built-in "shortcut" commands that were alluded to in the peak script's usage message. It takes the argument following runIni, and loads it as a configuration file. Take a moment now to type peak runIni, and look at the usage message:

 
Usage: peak runIni CONFIG_FILE arguments... 
 
CONFIG_FILE should be a file in the format used by 'peak.ini'.  (Note that 
it does not have to be named with an '.ini' extension.)  The file should 
define a 'running.IExecutable' for the value of its 'peak.running.app' 
property.  The specified 'IExecutable' will then be run with the remaining 
command-line arguments. 
 

Okay, there's that business about executable command interfaces again, but we won't let that put us off because we know we can get away with using a function for now.

So, now we know we need an .ini-format file, with a peak.running.app property. But it doesn't have to actually be named with .ini, so let's just call our configuration file helloworld:

 
[peak.running] 
app = importString("helloworld.HelloWorld") 
 

(Note to experienced Pythonistas: PEAK does not use the Python ConfigParser module to parse .ini files, so don't assume that you can use ConfigParser syntax or semantics here. To meet PEAK's configuration requirements, both the syntax and semantics needed to be different from those supplied by ConfigParser.)

How did we know to format the file this way? Well, we looked at the peak.ini file, of course, and learned by example. Here we've defined a configuration rule for the property named peak.running.app. The rule says (in effect), "when someone asks for the peak.running.app property, call the PEAK importString function on the string "helloworld.HelloWorld", and return the value.

The importString() function acts like a Python import statement, such that importString("x.y.z") is equivalent to from x.y import z. Thus, the importString() line of the config file shown above is roughly equivalent to:

 
    from helloworld import HelloWorld as app 
 

The reason we say "roughly" is that PEAK configuration files aren't executed the way Python code is. Properties in the configuration file are not Python assignment statements: they're rules that supply a Python expression whose value is computed only when the property is looked up. When the configuration file is loaded, the expressions are read, but not executed. Indeed, if the property is never looked up, the expression is never evaluated at all.

This can be a big advantage over using a Python script for configuration, since you can create a shared .ini file that describes properties used by several applications. Even if some of those properties might involve a time-consuming operation such as opening a database connection, only the applications that actually use the property will pay the cost of computing it.

Indeed, this allows you to use a single site-wide configuration file to provide configuration for all of your PEAK-based applications, using the PEAK_CONFIG environment variable to point to the file. Of course, the applications can't use property names that conflict with other applications' property names. But, there are many configuration properties used by PEAK that you may wish to share across multiple applications, such as logging rules, naming services, and command shortcuts, to name just a few. Property definitions supplied by application-specific configuration files will of course override any definitions in the site-wide configuration defaults.

Anyway, we now have a configuration file for our application, trivial as it is so far. Let's try it out:

 
% peak runIni helloworld 
Hello, world! 
 

Looks like it still works.

By the way, just as a side note, if an application object like our HelloWorld function returns a value, that value has significance. If the value is an integer (or None), it will be used as the return code from the peak script, and of our command line as a whole. If it is not an integer, it will be echoed to the terminal, and 1 will be used as the return code from the peak script. If you want, try this out now by adding various return statements to your helloworld.py and checking the exit code from the peak script in your shell.

1.4 Executable Configuration Files

The configuration file for a complex application is often central to that application. Under Unix-like command shells, PEAK allows configuration files to be the "executable" file of the application, the file whose name is entered at the console to invoke the program. This is done using the "magic cookie" of the unix shell:

 
#!/usr/bin/env peak runIni 
 
[peak.running] 
app = importString('helloworld:HellowWorld') 
 
This uses the Unix utility env to find the peak command in the current PATH, and calls it. peak then parses the file just like it does when called via peak runIni [file] from the command line. (Note the configuration file must also be made executable with chmod +x helloworld if you're using a Unix-like OS, as opposed to just a Unix-like shell.)

Python still needs imported modules to be on the PYTHONPATH, though, so you have to call this helloworld file like this:

 
% PYTHONPATH=. ./helloworld 
 
(Or else set the PYTHONPATH separately, as we did earlier. In that case, you can just run ./helloworld. Try it now.)

In a real application, you could of course install your modules in a directory on the system PYTHONPATH (such as in the Python site-packages directory), thus enabling your command to simply be called by name, without setting any pesky environment variables.

If you can't arrange for the package to be on the system PYTHONPATH, or you don't want it to be there, you have a couple of different options. For example, you might change the "magic cookie" line to read:

#!/usr/bin/env PYTHONPATH=/where/my/modules/are peak runIni 

But this doesn't work on operating systems that treat the entire #! line as a single command argument. For those operating systems, you might write a short shell script instead. In this case you might call the config file helloworld.ini, since the shell script will be called helloworld. If helloworld.ini and helloworld.py are in the same directory as the helloworld script, the script might look like this:

 
#!/bin/sh 
here=`dirname $0` 
PYTHONPATH=$here peak runIni $here/helloworld.ini 
 
Of course, in a real app you'd probably put the script in your /usr/local/bin directory or equivalent and the module files in some special directory, and replace $here with the actual path to those module files.

We're going to assume from here on out that you've got your system configured to run the helloworld example when you type ./helloworld. You can do that by making helloworld an executable in one of the ways we've described, or you can just cheat and type peak runIni helloworld everywhere we say to type ./helloworld.

1.5 Making the Application Object Configurable

Okay, enough messing around with application deployment issues, and on to configuration. Let's move the "Hello world" message into the configuration file.

The configration file follows .ini format, which means you have "sections" in square brackets followed by lines that set values for various names within the section. PEAK doesn't place many restrictions on what sections or property names can appear in an .ini file, so we can pretty much choose whatever section name and property names we want, as long as they're valid Python identifiers.

We'll be very unoriginal and call our configuration section helloworld. (It's generally a good idea to have an application's properties grouped under names that begin with the application name, so they won't conflict with those of other applications.) The text of our message will be placed in the message configuration property:

 
#!/usr/bin/env peak runIni 
 
[peak.running] 
app = importString('helloworld:HellowWorld') 
 
[helloworld] 
message = "Hello, world!" 
 
At this point, you can run helloworld again, and nothing will have changed. PEAK will have loaded the new configuration information we provided, but our program doesn't yet actually use that information.

To use the configuration information we need to obtain it via the PEAK configuration framework. At this point we are actually starting to use PEAK facilities directly in our Python code, so we'll need to import them from the PEAK package. PEAK provides a simple way to access the vast majority of the PEAK facilities through its api module:

 
from peak.api import * 
 
This will load into the local namespace a minimal set of names through which all of the PEAK frameworks can be accessed. It uses a lazy import mechanism, so ultimately your program will only end up loading into memory those PEAK facilities it actually uses. (As you might be noticing, "lazy loading" and "lazy evaluation" are recurring themes in PEAK. We'll be running into them over and over again, in this and subsequent chapters.)

In addition, we can no longer use a simple, dumb function object as our application. In order to access the configuration information, the application object must participate in the appropriate PEAK frameworks.

In this case, the framework for a simple command line command such as our helloworld is provided by the AbstractCommand class of the peak.running.commands module (lazily imported for us by peak.api):

 
class HelloWorld(commands.AbstractCommand): 
 
An "abstract" class is a base class that cannot be used by itself. To be used, it must be subclassed and certain abstract methods overridden with "real" code in the concrete subclass.

In the case of AbstractCommand, there is only one method we need to override: the _run method, which will do the actual work of our application (and optionally return an exit code, just like our function could). So, without getting into the configurability yet, our complete helloworld.py file would now look like this:

    1 from peak.api import *
    2 
    3 class HelloWorld(commands.AbstractCommand):
    4 
    5     def _run(self):
    6         print >>self.stdout, "Hello, world!"
Hmm, what's that output redirection? Doesn't print normally write to stdout?

Indeed, it does. And stdout is normally the shell command's stdout, and certainly is in our little application here. But an AbstractCommand can be run in many environments, not just from a shell script the way we are doing here. So to future-proof our code, we use the facilities AbstractCommand provides for communicating with its environment, whatever that may turn out to be.

(In addition to self.stdout, AbstractCommand also provides self.stdin, self.stderr, self.environ, and self.argv.)

So, again, we can run our program at this point and get the same results as before. But we still haven't used the configuration information. To do that, we need the binding package, to "bind" an attribute of our application to a configuration variable.

1.6 Binding attributes to configuration

Specifically, we will use the Obtain class from the peak.binding package. Obtain, coupled with the PropertyName class, allows us to pull values out of the configuration file:

 
    message = binding.Obtain(PropertyName('helloworld.message')) 
 

Note that this code does not mean, "set message to the configuration property helloworld.message. Instead it means, "when the message attribute of an instance of this class is accessed, obtain the helloworld.message property, and use that as the attribute's value from then on." (See? We told you lazy loading was a recurring theme in PEAK!)

Notice that this means bindings like Obtain are Python "descriptors", similar to the Python built-in type property. So, in order for a binding to be used, it must be placed in a class, and then used from an instance of the class.

Obtain can find things using mechanisms other than PropertyName, but PropertyName is the one we are interested in here. PropertyName is a string subclass, and its constructor takes as its argument the fully qualified property name. By "fully qualified", I mean the name of the section the property is found in, followed by a dot, followed by the rest of the property name. So the message property from our helloworld section becomes the PropertyName helloworld.message.

The PropertyName class ensures that its instances are of valid syntax for use in a configuration file, but that's not the only reason we need to use it. If we gave Obtain an ordinary string, it would not look it up in our configuration file. Instead, it would try to use the string as a URL or a component path -- two ideas we don't need or want to get into right now.

Anyway, when we access our HelloWorld object's message attribute in the _run() method, the binding.Obtain descriptor will look up the configuration property for us. It does this by searching the context in which the instance is located. The peak.running.commands framework will provide us with with such a context automatically, when it creates an instance of our HelloWorld class. (We won't deal with the specifics of this mechanism here; suffice to say that some code we inherit from AbstractCommand will take care of the details for us.)

Let's look at the revised program:

    1 from peak.api import *
    2 
    3 class HelloWorld(commands.AbstractCommand):
    4 
    5     message = binding.Obtain(PropertyName('helloworld.message'))
    6 
    7     def _run(self):
    8         print >>self.stdout, self.message
With this version of helloworld.py we are finally using the configuration information provided in the helloworld file. To prove this, first run the helloworld command:
 
% ./helloworld 
Hello, world! 
 
Now copy helloworld to hellonewworld and edit it to look like this:
 
#!/usr/bin/env peak runIni 
 
[peak.running] 
app = importString('helloworld.HelloWorld') 
 
[helloworld] 
message = "Hello, new world!" 
 
If you now run this new command:
 
% ./hellonewworld 
 
you'll be greeted by our new message:
 
Hello, new world! 
 
Although this is a very simple example, it should already be clear that PEAK provides an easy to use and powerful configuration mechanism. Simple use of PropertyName and values provided in config files barely even scratch the surface of what PEAK can do, but even if you used nothing else PEAK would still be useful just for making it easy to write configurable python shell scripts!

1.7 Points to Remember

Before we move on to the next chapter, let's recap the concepts we've covered so far:

2 Command Arguments and Simple Database Access

Now its time to go a tiny step beyond the triviality of the "Hello, world!" example. In this chapter I'll expand the example to handle saying hello to an arbitrary variety of things. Since we want to be flexible. we'll get the greeting message for each thing from a database table. Well, really just a flat file, but that's so we don't get distracted with SQL connections and all that just yet.

In a "real" application, we'd be using SQL or some other robust storage as our "back end". But here we want to focus on the issues of creating an "object-relational mapping", without getting bogged down in the relational part. Unfortunately, this makes the tutorial a bit lopsided, because we'll be building a sophisticated object-relational mapping over data that would've been trivial to use in its original form!

So, try to ignore that part, and focus on the ideas, which scale up to vastly larger applications than what we're showing here.

2.1 Command Arguments

To greet more than one thing, we'll need to be able to tell our command what to greet. Let's rename our command to hello, and give what we want to say hello to as the command argument, e.g.:

 
% hello world 
% hello fred 
% hello Klause 
 

If all we want to do is say hello in the same boring way every time, we could revise our helloworld.py file as follows:

    1 from peak.api import *
    2 
    3 class HelloWorld(commands.AbstractCommand):
    4 
    5     message = binding.Obtain(PropertyName('helloworld.message'))
    6 
    7     def _run(self):
    8         print >>self.stdout,  self.message % self.argv[1]
Note the use of self.argv to get access to the command arguments.

The corresponding hello file would look like this:

 
#!/usr/bin/env peak runIni 
 
[peak.running] 
app = importString('helloworld.HelloWorld') 
 
[helloworld] 
message = "Hello, %s.  How are you today?" 
 
Now we'll get something like this:
 
% export PYTHONPATH=. 
% ./hello world 
Hello, world.  How are you today? 
% ./hello fred 
Hello, fred.  How are you today? 
% ./hello Klause 
Hello, Klause.  How are you today? 
 

2.2 Domain Models

When working with application data, PEAK uses the concept of a "domain model" for that data. A "domain model" describes the kinds of "problem domain" (i.e. "real-world") objects you're working with, and their relationships to other kinds of objects. In PEAK-speak, the objects are called "Elements", and the relationships are called "Features". Features are implemented as object attributes, using custom descriptors.

To facilitate grabbing data from our database, we'll define a simple Element class, in a hello_model.py file:

    1 from peak.api import *
    2 
    3 class Message(model.Element):
    4 
    5     class forname(model.Attribute):
    6         referencedType = model.String
    7 
    8     class text(model.Attribute):
    9         referencedType = model.String
Here Message is our Element, the thing we are going to load from our database file. forname and text are attributes our Message objects will have once loaded. I think their purposes should be pretty obvious.

You probably noticed immediatly that we are defining classes inside classes here. The nested classes actually create attribute descriptors, similar to the Python built-in property type. However, instead of having to define functions and then wrap them into property object, we can simply subclass a predefined "Feature" type such as model.Attribute, and provide parameters such as referencedType, or define methods to control the feature's behavior.

(Note, by the way, that referencedType does not necessarily refer to the class of the objects or values that will be stored in the attribute. It can also reference an object like model.String, that simply provides metadata describing what values are acceptable for the attribute. For more extensive examples of using Model types, see the bulletins example in the examples directory of the PEAK CVS tree.)

2.3 Data Managers

The domain model by itself is simply a schema, perhaps with some behavior. (For example, we might add a hello() method to our Message class, so that an instance of Message could actually deliver its message directly.)

But, a domain model by itself doesn't know anything about storage. (This is so that we can reuse the domain model with different kinds of storages.) To store and retrieve instances of our domain model classes, we need a Data Manager. Data Managers are responsible for loading and storing the data described by the domain model classes.

For the present example we're only interested in loading data from a "database table". So we'll subclass QueryDM, a peak.storage class that provides a read-only interface to a datastore:

 
from peak.api import * 
 
class MessageDM(storage.QueryDM): 
 
This class is another abstract class that has to be specialized for our intended use. Specifically, we have to add the code that does the actual reading and writing of data from the model attributes, to and from the external datastore.

To keep our example simple, we'll use a flat file as our external data store. In keeping with PEAK design principles, we won't hardcode the filename into our Data Manager, but will instead make it configurable:

 
    filename = binding.Obtain(PropertyName('helloworld.messagefile')) 
 
Obviously, we'll now need a different line in our hello configuration file::
 
[helloworld] 
messagefile = config.fileNearModule('helloworld', 'hello.list') 
 
Here we've used another PEAK function: config.fileNearModule() will construct an appropriately qualified filename for the second argument based on the assumption that it is in the same directory as the module named by the first argument. So, messagefile will be the path to the hello.list file, located in the same directory as our helloworld.py file.

Since we're only using a QueryDM in this example, we only have to worry about reading data from the datastore, not writing it (That's a later example). To specify how to do this, we override the _load method of QueryDM. Our _load method needs to return a dictionary of names and values, which will get used through a __setstate__ call to load the data into our Message instances.

2.3.1 Loading data from the flat file

Now we need to whip up a data format for our messagefile. Let's have the thing we are saying hello to be first, and the actual message second, separated by a "|" character.

So we'll create a hello.list file like this:

 
world  | Hello, world! 
Fred   | Greetings, good sir. 
Klause | Guten Aben, Herr Klause. 
 
(Forgive my feeble attempts at Deutsche.)

Because this is going to be a read-only file, we're going to cheat and load the file only once, the first time it's used. We'll use another peak.binding tool to accomplish this:

    1     def data(self):
    2         data = {}
    3         file = open(self.filename)
    4         for line in file:
    5             fields = [field.strip() for field in line.split('|',1)]
    6             forname, text = fields
    7             data[forname] = {'forname': forname, 'text': text}
    8         file.close()
    9         return data
   10 
   11     data = binding.Make(data)

binding.Make is similar to binding.Obtain, in that it's used inside a class body to create a property-like descriptor for the class' instances. It's different, in that it takes a function as its argument, rather than a configuration key. The function should take at least one parameter (self), and return the value to be used for an attribute. In this way, it's very similar to the property built-in, but with a key difference: a property's fget function is called every time it is used, but the result of a binding.Make function is cached and reused for subsequent accesses of the attribute.

So, here's what will happen. The first time an instance of our QueryDM subclass accesses its data attribute, the function above will be called, and the result stored in the instance's data attribute. It will then be immediately available for use, and won't be computed again for that instance unless the attribute is deleted.

(Of course, in the case of our current hello program, we'll only ever make one query on the database. If we were going to make a longer-running program, or allow the database to be modified, using this sort of caching might be a bad idea. However, this design decision affects only our data manager's implementation, and not the rest of the application. Our main, command-line application will not be affected, and neither will our Message class, if we decide to change how or where the messages are stored.)

2.3.2 Building the Data Manager subclass

Here's the complete contents of the last new file we need for our expanded hello application, the hello_storage.py file. This also adds the _load method to the QueryDM:

    1 from peak.api import *
    2 from hello_model import Message
    3 
    4 class MessageDM(storage.QueryDM):
    5 
    6     defaultClass = Message
    7     filename = binding.Obtain(PropertyName('helloworld.messagefile'))
    8 
    9     def data(self):
   10         data = {}
   11         file = open(self.filename)
   12         for line in file:
   13             fields = [field.strip() for field in line.split('|',1)]
   14             forname, text = fields
   15             data[forname] = {'forname': forname, 'text': text}
   16         file.close()
   17         return data
   18 
   19     data = binding.Make(data)
   20 
   21     def _load(self, oid, ob):
   22         return self.data[oid]
defaultClass specifies the class that will be used to instantiate objects retreieved from this Data Manager. In our case, that's Message from our model class. binding.Obtain you've met before, so its purpose here should be obvious.

A data manager is like a container for application objects. It's keyed by the notion of an oid: an "object ID" for objects of this kind. So, when we use a MessageDM instance, we'll retrieve objects from it like this:

 
        aMessage = myMessageDM[someName] 
 

When we do this, the MessageDM will return what's called a "ghost". It will be an instance of Message that contains no data, but knows its object ID, and knows that it's not yet loaded. As soon as we try to use the Message (by accessing any attributes or methods), it will "phone home" to the data manager it was retrieved from, asking for its data to be loaded.

At this point, the MessageDM._load() method is going to get called. It'll be given the object ID that was used to access the object originally (the oid parameter), and the applicable "ghost" object (ob). The data that _load() returns will be used to fill in the ghost's instance dictionary so that it will become a "real" object, and the attribute access that triggered the _load call can finally be satisfied.

(Notice, by the way, that if we were using a relational database, the _load method is probably where we'd put an SQL query to retrieve the data for the given object ID.)

2.4 Putting it all Together

All that remains, then, is to use our new data manager from our main application program:

    1 from peak.api import *
    2 from hello_model import Message
    3 
    4 class HelloWorld(commands.AbstractCommand):
    5 
    6     Messages = binding.Make(
    7         'hello_storage.MessageDM',
    8         offerAs=[storage.DMFor(Message)]
    9     )
   10 
   11     def _run(self):
   12         storage.beginTransaction(self)
   13         print >>self.stdout, self.Messages[self.argv[1]].text
   14         storage.commitTransaction(self)
As you can see, our main program has stayed fairly simple, despite the additional complexity of using a database. (And in case you think "using a database" is an inflated way of refering to a flat file, observe that we can replace the simple flat file with access to something like an SQL database, simply by changing the _load method in our data manager.)

In our revised HelloWorld class, we see another use of binding.Make, this time taking an import string that specifies a class that should be instantiated. Previously, we used binding.Make with a function, but it also accepts classes, or strings that say where to import classes or functions from. (Indeed, it takes anything that implements or adapts to the binding.IRecipe interface, but that's more than you need to know right now.)

When used with a class (or an import string that names a class), binding.Make will call it once, the first time the named attribute is used. In this case, that means that it will automatically create a new MessageDM as the Messages attribute of the HelloWorld instance it's contained in. So, in effect, we are declaring that each HelloWorld instance should have its own MessagesDM instance, stored in its Messages attribute.

Here we also use an additional keyword argument (offerAs) to binding.Make. offerAs is a list of "configuration keys" under which the created component will be "offered" to child components (via the configuration system). In this case, we're saying that any child components of a HelloWorld instance should use its Messages attribute, if they are looking for a data manager that provides storage services for Message instances.

The PEAK configuration system offers many kinds of "configuration keys" under which components or configuration properties can be found. We've previously worked with PropertyName, which is one kind of configuration key. And here we work with another, storage.DMFor(), that creates a configuration key denoting a data manager for a particular element type. PEAK does not limit you to using its predefined kinds of configuration keys, however. You can also create your own key types for specialized purposes, by implementing the config.IConfigKey interface.

Once you "offer" an attribute as the source of a configuration key, it can then be referenced by other uses of binding.Obtain by child components. For example, if we had another class that needed to use a data manager for Message instances, we could add something like this to that class:

 
    Messages = binding.Obtain(storage.DMFor(Message)) 
 

This would use the configuration system to find an appropriate data manager. And, if an instance of this other class were contained within an instance of HelloWorld, it would use the HelloWorld object's Messages attribute to fill its own Messages attribute. (Note that the similarity in names has nothing to do with how it works; we could have called one of the attributes "foobar" and it would make no difference.)

The big advantage of this is that it allows us to create "loosely coupled" reusable components. A component that needs some service can simply Obtain it via a suitable key, and a higher-level application component can "offer" an appropriate service implementation. What's more, even binding.Obtain can use the offerAs argument, thereby specifying that an obtained component will be available to the containing component's children, perhaps under another configuration key as well as the key that was used to obtain the service.

Anyway, once we've got the data manager to be used for Message instances, we can look up any Message instance in it by using the instance's object id (in this case, its forname) as a key. In our case, that key is the string passed in as an argument to our command (self.argv[1], supplied to us by the peak script). As previously discussed, this gives us a Message instance, which we can then display by printing its text attribute.

Our last addition to HelloWorld is the use of beginTransaction and commitTransaction to enclose our data manipulation in a transaction. For our simple read-only application here, a transaction is of relatively little importance, but transaction management will matter a great deal when you use read/write databases.

Anyway, our program now works like this:

 
% export PYTHONPATH=. 
% ./hello world 
Hello, world! 
% ./hello Fred 
Greetings, good sir. 
% ./hello Klause 
Guten Aben, Herr Klause. 
 

2.5 Points to Remember

Let's recap some of the key topics we've covered in this chapter:

Whew! That's a lot of things, but we're still only scratching the surface. Each of the major topic areas listed above could be expanded into entire tutorials of their own. But, for now, you may want to simply experiment a bit with what you've seen so far, before delving deeper into the PEAK API documentation and source code.

3 Multiple Commands and Writable Databases

(Note: This chapter has not yet been edited by PJE)

We're stretching pretty hard here to find ways to use "Hello, world!" to demonstrate PEAK concepts. Just keep supressing your impulse to laugh at the triviality of the example, and keep focused on the concepts it's demonstrating.

At this point our hello program can greet anything whose name is recorded in the table. Suppose we have a new thing we want to greet? What we need is a way to update the database table to record a new greeting. Our hello program will now have two different functions we need it to perform: greeting, and recording new greetings. (Of course, we could also implement these as two separate commands, but then we wouldn't have an excuse to talk talk about AbstractInterpreter and demonstrate Bootstrap).

So in this chapter we'll expand the hello command to have subcommands:

 
    % ./hello for Jeff: Hi, guy! 
    % ./hello to Jeff 
    Hi, guy! 
 
This will require revising our storage implementation to allow writing to our database. We'll stick to using a file for now, to keep the distraction of SQL at bay for a little while longer.

3.1 AbstractInterpreter and Bootstrap

Another abstract class provided by PEAK is AbstractInterpreter. This class represents something PEAK can run that will execute subcommands based on the first argument word. PEAK also provides an subclass of AbstractInterpreter called Bootstrap, that looks up a URL or command shortcut and runs it.

If you think that sounds a lot like what the peak script does, you're quite correct. If you take a look at the actual peak script, it looks something like:

    1 #!/usr/bin/env python2.2
    2 
    3 from peak.running import commands
    4 commands.runMain( commands.Bootstrap )

Which means that we've actually been using Bootstrap all along, to run our programs. Since we'd like to be able to use commands like hello to and hello for in the same way that we can use peak help or peak runIni, we'll make our new main program a subclass of commands.Bootstrap:

    1 from peak.api import *
    2 from hello_model import Message
    3 
    4 
    5 class HelloWorld(commands.Bootstrap):
    6 
    7     usage = """
    8 Usage: hello command arguments
    9 
   10 Available commands:
   11 
   12     for -- sets a greeting
   13     to  -- displays a greeting
   14 """
   15 
   16     Messages = bindings.Make(
   17         'hello_storage.MessageDM',  offerAs=[storage.DMFor(Message)]
   18     )
   19 
   20 
   21 class toCmd(commands.AbstractCommand):
   22 
   23     usage = """
   24 Usage: hello to <name>
   25 
   26 Displays the greeting for "name".
   27 """
   28 
   29     Messages = binding.Obtain(storage.DMFor(Message))
   30 
   31     def _run(self):
   32         storage.beginTransaction(self)
   33         print >>self.stdout, self.Messages[self.argv[1]].text
   34         storage.commitTransaction(self)
So we've got a new HelloWorld main program, this time a subclass of commands.Bootstrap. HelloWorld is still the holder for the MessageDM binding. The Bootstrap class will automatically make the subcommands into children of our HelloWorld component, so the subcommands will be able to Obtain the DM from their context.

The only other thing it's got is a usage class variable. To see how this is used, try typing ./hello at your command prompt:

 
% ./hello 
 
Usage: hello command arguments 
 
Available commands: 
 
    for -- sets a greeting 
    to  -- displays a greeting 
 
 
./hello: missing argument(s) 
 
As you can see, PEAK is taking care of a lot of the routine tasks associated with writing a script!

Our original AbstractCommand is still there, but now we've named it toCmd. And, since it a different class from HelloWorld where we defined the binding to our MessageDM, it needs to Obtain that binding. Remember, it is thereby getting access to the same MessageDM instance as the one in the associated HelloWorld instance. So, by using Make(...,offerAs=[something]) in a parent component, and Obtain(something) in child components, a parent component can share one instance of a service with any child component that needs it.

3.2 peak.running.shortcuts

Next, we need to hook up the toCmd class so it can be invoked as a subcommand. How can we do that? Remember when we looked at the help for the peak script? The first paragraph said:

  
The 'peak' script bootstraps and runs a specified command object or command  
class.  The NAME_OR_URL argument may be a shortcut name defined in the  
'peak.running.shortcuts' property namespace, or a URL of a type  
supported by 'peak.naming'. 
 

Hm. If we look at the peak.ini file, we see a section called [peak.running.shortcuts], containing a bunch of properties called runIni, help, and many other commands.

Does this mean that if we add a similar section to our hello file, we can create subcommands of our own? Let's try adding this new section to hello:

 
[peak.running.shortcuts] 
to = importString('helloworld.toCmd') 
 

and now try it:

 
    % ./hello to Fred 
    Greetings, good sir. 
 

Excellent! By the way, you may have noticed that when we turned our command into a subcommand, we did not need to change our argv index number. The argument array stored in self.argv of the subcommand has the subcommand name in argv[0], and the rest of the arguments starting in argv[1]. That's because AbstractInterpreter classes like Bootstrap automatically shift the arguments over for us when they create the subcommand object.

Also by the way, we should mention that it wasn't strictly necessary to edit the configuration file to do what we just did. We also could have defined a binding in our HelloWorld class to "offer" the right configuration value, like this:

 
    __toCmd = binding.Obtain( 
        'import:helloworld.toCmd',  
        offerAs=['peak.running.shortcuts.to'] 
    ) 
 

But now that you've seen how, you can also see why we didn't do it. It's rather ugly to do this sort of configuration in code, compared to using an .ini file. But it's nice to know you can do it if you need to.

Of course, the configuration file is also more flexible: notice, for example, that we could make multiple configuration files for the same code, each file specifying a different set of subcommands, perhaps for different users of the app. You could almost say that PEAK's motto is, "code reuse, through flexibility".

Now for something completely different. Let's try this:

 
    % ./hello to 
 
Given how ./hello magically generated a usage string, you might think this would do so as well. After all, we provided one in the code above, right? Well, an AbstractCommand doesn't automacially display a usage when no arguments are supplied because, after all, no arguments might be required. It will automatically display the usage if we raise a commands.InvocationError, in our _run method, though:
    def _run(self): 
        if len(self.argv)<2: 
            raise commands.InvocationError("Missing name") 
        storage.beginTransaction(self) 
        print >>self.stdout, self.Messages[self.argv[1]].text 
        storage.commitTransaction(self) 

Now we'll get:

 
%s ./hello to 
 
Usage: hello to <name> 
 
Displays the greeting for "name". 
 
to: Missing name 
 

There's just one problem left with the hello command. Try running hello runIni, and see what we get:

 
% ./hello runIni 
 
Usage: peak runIni CONFIG_FILE arguments... 
 
CONFIG_FILE should be a file in the format used by 'peak.ini'.  (Note that 
it does not have to be named with an '.ini' extension.)  The file should 
define a 'running.IExecutable' for the value of its 'peak.running.app' 
property.  The specified 'IExecutable' will then be run with the remaining 
command-line arguments. 
 
 
runIni: missing argument(s) 
 

Whoops! Just because our configuration file contains its own [peak.running.commands] section, doesn't mean that the settings in peak.ini don't apply. We need to do something about this, so that hello doesn't reuse all the peak subcommands.

Looking at peak.ini, we sometimes see that properties sometimes end with a *. What happens if we define a * rule in the shortcuts section,?

 
[peak.running.shortcuts] 
*  = commands.NoSuchSubcommand 
to = importString('helloworld.toCmd') 
 

Let's try it now:

 
% ./hello runIni 
 
Usage: hello command arguments 
Available commands: 
 
    for -- sets a greeting 
    to  -- displays a greeting 
 
 
runIni: No such subcommand 'runIni' 
 

Good. To recap, we used commands.NoSuchSubcommand, which raises an InvocationError for us, and we used a * rule to define a default value for properties whose names are within a particular "property namespace". That is, any name we look up in peak.running.shortcuts from our configuration file, that isn't explicitly defined there or in our app, will return the commands.NoSuchSubcommand class. That's just what we want for now.

Actually... there is still one more problem. commands.Bootstrap also accepts URLs on the command line by default. We can turn that off by adding:

 
    acceptURLs = False 
 
to our HelloWorld class. With these changes, our Bootstrap derivative will now do the right thing. Let's move on to the for command now.

3.3 Storing a New Message: the "for" Subcommand

Now, we know we're going to have to rewrite our hello_storage.py to allow us to write to the databse, but let's start this part of the task by writing the subcommand first. As you'll quickly see, any consideration of how we implement the saving of the data is completely independent of how we go about initiating the save in the application program.

So, we need another rule in our hello configuration file:

 
[peak.running.shortcuts] 
*  = commands.NoSuchSubcommand 
to = importString('helloworld.toCmd') 
for = importString('helloworld.forCmd') 
 
and another AbstractCommand subclass in helloworld.py

 
class forCmd(commands.AbstractCommand): 
 
    usage = """ 
Usage: hello for <name>: <greeting> 
 
Stores "greeting" as the greeting message for "name". 
""" 
 
    Messages = binding.Obtain(storage.DMFor(Message)) 
 
    def _run(self): 
        if len(self.argv)<2:  
            raise commands.InvocationError("Missing arguments") 
        parts = ' '.join(self.argv[1:]).split(':',1) 
        if len(parts)!=2: 
            raise commands.InvocationError("Bad argument format") 
        forname, message = parts 
        storage.beginTransaction(self) 
        newmsg = self.Messages.newItem() 
        newmsg.forname = forname.strip() 
        newmsg.text = message.strip() 
        storage.commitTransaction(self) 
 
To put a new object in our database, we ask the Data Manager for a new "blank" object. (Actually, it can have a preloaded default state, but we'll ignore that for now). Then we modify it just like we would any other writable object we got from the Data Manager, and the transaction machinery takes care of getting the data written to the backing store at transaction commit time.

At this point the for subcommand of our hello command is runable:

 
% ./hello for 
 
Usage: hello for <name>: <greeting> 
 
Stores "greeting" as the greeting message for "name". 
 
for: Missing arguments 
 
% ./hello for foobar 
 
Usage: hello for <name>: <greeting> 
 
Stores "greeting" as the greeting message for "name". 
 
for: Bad argument format 
 
% ./hello for Jeff: Hi, guy! 
Traceback (most recent call last): 
  File "/usr/local/bin/peak", line 4, in ? 
    commands.runMain( commands.Bootstrap ) 
  File "/usr/local/lib/python2.3/site-packages/peak/running/commands.py", line 70, in runMain 
    result = factory().run() 
  File "/usr/local/lib/python2.3/site-packages/peak/running/commands.py", line 211, in run 
    return self._run() or 0 
  File "/var/home/rdmurray/proj/peak/helloworld/07writabledb/helloworld.py", line 53, in _run 
    newmsg = self.Messages.newItem() 
AttributeError: 'MessageDM' object has no attribute 'newItem' 
 
Ah, yes. As you'll recall, we used a read-only Data Manager base class when we developed our database. So we can't store anything until we fix that.

3.4 Storing a New Message: Modifying the Data Manager

OK, it's time to do some serious surgery on our Data Manager. As I said before, I'm still going to keep us away from SQL for the moment. But if you'll recall, I mentioned how important transactions are when writing to a database. Writing to normal file system files is not transaction oriented, so what can we do?

PEAK provides a useful utility class that partially solves this problem: EditableFile. EditableFile is an object that represents a file and participates in the PEAK transaction machinery.

Here's what the EditableFile docstring has to say:

    """File whose text can be manipulated, transactionally 
 
    Example:: 
 
        myfile = EditableFile(self, filename="something") 
        print myfile.text   # prints current contents of file 
 
        # Edit the file 
        storage.beginTransaction(self) 
        myfile.text = myfile.text.replace('foo','bar') 
        storage.commitTransaction(self) 
 
    Values assigned to 'text' will be converted to strings.  Setting 'text' 
    to an empty string truncates the file; deleting 'text' (i.e. 
    'del myfile.text') deletes the file.  'text' will be 'None' whenever the 
    file is nonexistent, but do not set it to 'None' unless you want to replace 
    the file's contents with the string '"None"'! 
 
    By default, files are read and written in "text" mode; be sure to supply 
    a 'fileType="b"' keyword argument if you are editing a binary file.  Note 
    that under Python 2.3 you can also specify 'fileType="U"' to use "universal 
    newline" mode. 
 
    'EditableFile' subclasses 'TxnFile', but does not use 'autocommit' mode, 
    because it wants to support "safe" alterations to existing files. 
This looks pretty straightfoward to use, especially since we can assume, since we are writing in a Data Manager, that we will be inside a transaction and don't have to worry about that aspect here.

OK, now that we've got a way to transactionally update a file, we need to exchange our QueryDM base class to a base class that supports updating the database. That would be EntityDM.

EntityDM requires two additional methods to be defined by the concrete class: _new, and _save. _new is called when a new object is added to the DM, and needs to store the data for that object in the external database. _save is called when an object's state is changed, and a transaction boundry has been reached where that state needs to be synchronized with the external database.

At this point our original scheme of reading and parsing the file and storing the results the data attribute is going to break down. With our current user interface it would actually work just fine. The file would get reparsed on each command invocation. But suppose we later rewrite the interface so that multiple commands can be issued in the same run of the application? If we keep our pre-parsed data attribute, a message updated by a for command wouldn't be seen by subsequent to commands.

So, we're going to have to use the EditableFile instance as our reference for each lookup from the database. Our new hello_storage.py will now look like this:

    1 from peak.api import *
    2 from peak.storage.files import EditableFile
    3 from hello_model import Message
    4 import re
    5 
    6 class MessageDM(storage.EntityDM):
    7 
    8     defaultClass = Message
    9     fn = binding.Obtain(PropertyName('helloworld.messagefile'))
   10 
   11     def file(self):
   12         return EditableFile(filename=self.fn)
   13 
   14     file = binding.Make(file)
   15 
   16     def __makeRe(self, oid):
   17         return re.compile(r"^%s[|](.*)$" % oid, re.M)
   18 
   19     def __findMatch(self, oid, text):
   20         return self.__makeRe(oid).search(text)
   21 
   22     def _load(self, oid, ob):
   23         m = self.__findMatch(oid, self.file.text)
   24         return {'forname': oid, 'text': m.group(1)}
   25 
   26     def _new(self, ob):
   27         self.file.text += "%s|%s\n" % (ob.forname, ob.text)
   28 
   29     def _save(self, ob):
   30         self.file.text = self.__makeRe(ob.forname).sub(
   31             "%s|%s" % (ob.forname, ob.text), self.file.text)
Well, that actually got a little simpler, didn't it? But I notice that in my attempts to (temporarily) stay away from the complexities of SQL, I seem to have wandered into the complexities of regular expressions.

Let's get that out of the way first. The not-too-gnarly little regular expression I use is designed to do two things: it matches any line (that's the re.M bit) that starts with the oid followed by a vertical bar, and when used for a search it returns a match object that has a group containing everything after the vertical bar to the next newline.

This regular expression is only going to work if there's no space between the forname and the vertical bar, but since we're now updating the database with our program as well as reading it, we can insure that. If you are following along with running code, though, you might want to edit the old hello.list to remove the spaces.

There are a few things to notice about our revised DM.

We're still getting the filename from that same configuration variable. Now, however, we are turning that into an EditableFile. Again we use binding.Make to create a descriptor that will return a real value (and cache it) when the class attribute is actually accessed.

Note that without binding.Make here we'd be stuck, since if we tried to do something like

 
file = EditableFile(filename=fn) 
 
our application would blow up when we tried to access the EditableFile, because it would have hold of the descriptor instance instead of being able to access the filename through the descriptor. (To PJE: I suspect I'm explaining this badly, because I only half understand it; hopefully you can clarify it)

In the _load method the match group in the regex means the parsing out of the message is already done for us, and all we have to do is return the data.

The _new method simply appends an appropriate record to the file.

The _save method uses re's substitution method to generate a new version of the string representing the file with the body of the line matching the forname replaced by the new value.

With this code in place our for method is working:

 
% ./hello for Jeff: Hi, guy! 
% ./hello to Jeff 
Hi, guy! 
 

3.5 Oh no, a bug!

At this point certain readers are getting antsy because there's a bug in the for method we created up above. Assuming you've got all four of the entries in your database we've been using for examples, what happens if you do this:

 
% ./hello for Jeff: Long time, no see 
 
Obviously, the intent is to replace the current message for Jeff with a new one. However, our for code assumed the forname passed to it was a new name. Currently, the code will write a new entry to the database, which because of our storage implementation will append a new "Jeff" record to the end of the file. So techncially we've got a bug in our database implementation, too.

Let's fix that problem first.

If a regex doesn't match, re will return None. So in our _new method we can check to see if the new forname is already in the database or not, and raise an error if it is:

    1     def _new(self, ob):
    2         if self.__findMatch(ob.forname, self.file.text):
    3             raise KeyError, "%s is already in the database" % ob.forname
    4         self.file.text += "%s|%s\n" % (ob.forname, ob.text)

Now at least we'll get an error if we use our buggy for subcommand:

 
% ./hello for Jeff: Hello, dude. 
  [...first part of traceback elided...] 
  File "/usr/local/lib/python2.3/site-packages/peak/storage/data_managers.py", line 432, in flush 
    oid = ob._p_oid = self._new(ob) 
  File "/var/home/rdmurray/proj/peak/helloworld/07writabledb/hello_storage.py", line 27, in _new 
    raise KeyError, "%s is already in the database" % ob.forname 
KeyError: 'Jeff is already in the database' 
 

Fixing the for command feels a little less "natural". In fact, this is a limitation of the current DM framework that is slated to be fixed in alpha 4.

For now one way to handle this is to implement a __contains__ for the DM;

    1     def __contains__(self, oid):
    2         return self.__findMatch(oid, self.file.text) is not None
Armed with this, we can rewrite our for method as follows:
    1     def _run(self):
    2         if len(self.argv)<2: raise commands.InvocationError("Missing arguments")
    3         parts = ' '.join(self.argv[1:]).split(':')
    4         if len(parts)!=2: raise commands.InvocationError("Bad argument format")
    5         forname = parts[0].strip(); message = parts[1].strip()
    6         storage.beginTransaction(self)
    7         if forname in self.Messages:
    8             msg = self.Messages[forname]
    9         else:
   10             msg = self.Messages.newItem()
   11             msg.forname = forname
   12         msg.text = message
   13         storage.commitTransaction(self)

With this change, updating the database works:

 
% ./hello for Jeff: Hey, Dude! 
% ./hello to Jeff 
Hey, Dude! 
 

4 SQL, N2, and Naming

(Note: This chapter has not yet been edited by PJE)

OK, the big moment has come (hold your chuckles!). Management is impressed with our demo program, and they want us to take it into production, for use in greeting customers. There's a wrinkle, though: they aren't interested in having a custom greeting for every customer. Instead, they want us to access the corporate PostgreSQL database, check the group to which each customer has been assigned, and use a greeting associated with the customer's group to greet the customer.

So in this chapter we're going to learn how to use PEAK facilities to work with an SQL database, how to combine data from more than one database but keep them all in sync, and we'll also get to see just how flexible a PEAK program can be in the face of change. Along the way we'll introduce PEAK's "n2" utility, and give you a small glimpse of how powerful PEAK's naming system is. Oh, yeah, and we'll exercise the configuration system a bit more, too.

Then we'll finish it all off by throwing away our version 1 demo program and writing version 2 to match the new reality.

4.1 N2

"N2" stands for "Namespace Navigator", and it is a general purpose tool for browsing and manipulating any namespace for which someone has written a navigator adapter. It is available as a subcommand of the peak command:

%peak n2 
[n2] help 
Available commands: 
 
abort  commit l      ls     mksub  py     rd     rmsub  unset          
bind   dir    ll     md     mv     python rm     set                   
cd     help   ln     mkdir  pwd    quit   rmdir  show                  
Ty originally wanted to call it "into", so peak n2 is also a pun on "peek into".

Why is n2 of interest to us now? One of the n2 adapters avaiable is one that will allow us to talk to an SQL database. Not that that's the way our program will talk to the corporate database, mind. Rather, we want this tool to make it easy for us to set up a test SQL database we can test our modidified program against before we try it on the corporate database.

To do our testing, we'll use an SQLite database. (If you don't have SQLite on your system, you can get it from [WWW]http://www.hwaci.com/sw/sqlite/. You'll also need the PySQLite python interface library, available from [WWW]http://sourceforge.net/projects/pysqlite/. Or use your operating system's package facility if it has one (for example, on FreeBSD you could do "cd /usr/ports/database/py-PySQLite; make install").

Once these programs are installed, you can use n2 to work with a test SQLite database:

% peak n2 sqlite:test.db 
1> 
You'll note that the prompt is different this time.
1> help 
Available commands: 
 
abort        \abort       \buf-load    \echo        \import      \reset 
commit       \buf-append  \buf-save    \exit        \python      \rollback 
go           \buf-copy    \buf-show    \export      \quit        \set 
help         \buf-edit    \commit      \go          \reconnect   \sleep 
rollback     \buf-get     \describe    \help        \redraw      \source 
This n2 command interpreter is allowing you to enter SQL commands as the normal thing. So most n2 commands are preceeded by a "\". We can enter SQL commands, terminated by semi-colons, and they will be applied against the "test.db" SQLite databse. (Don't worry right now about how sqlite:test.db got turned into a connection to an SQLite database, we'll come back to that in the next section).

What we need is a test table that maps customer names to customer groups, since that's what we'll be accessing in the corporate database. The schema for that database lists the table as custgroups, and the columns of interest as NAME and GRP. Both columns are strings, VARCHAR(40) and (20), respectively. That's all we really need to know. In production our application will be using the strings from the corporate database, so we can use any old strings we like for our tests.

What we need to do is create a test table with the two critical columns, and some test data stored in it:

1> create table custgroups (NAME VARCHAR(40), GRP VARCHAR(20)); 
 
 
(0 rows) 
1> insert into custgroups (NAME, GRP) VALUES ('Jeff', 'vip'); 
 
 
(0 rows) 
1> insert into custgroups (NAME, GRP) VALUES ('Joe', 'peon'); 
 
 
(0 rows) 
1> insert into custgroups (NAME, GRP) VALUES ('Klause', 'vip'); 
 
 
(0 rows) 
1> insert into custgroups (NAME, GRP) VALUES ('Jackie', 'vvip'); 
 
 
(0 rows) 
1> insert into custgroups (NAME, GRP) VALUES ('Fred', 'oridinary'); 
 
 
(0 rows) 
select * from custgruops; 
NAME                 GRP                  
-------------------- -------------------- 
Jeff                 vip                  
Joe                  peon                 
Klause               vip                  
Jackie               vvip                 
Fred                 ordinary             
(5 rows) 
1> commit 
1> \quit 
Well, that was easy. Now we just have to tie it into our program.

Note that the "commit" step, which is an n2 command and not an sql command, is critical. N2 starts a transaction when it connects to the database. The commit command causes n2 to tell the database to commit the current transaction. If you don't do that, none of the changes made to create the table and insert the data will actually take effect.

4.2 peak.ini, and Naming

Remember back at the beginning when we took a look at the usage information for the peak command? The first argument to the peak command is described as "NAME_OR_URL", where a name is something defined in the peak.running.shortcuts section of an ini file, and URL is "of a type supported by peak.naming".

Well, by now you should understand how to define something in 'peak.running.shortcuts' and what that accomplishes, since we used it in the last chapter. It will thus probably come as no surprise to you to find that the 'peak' command has a ini file, and in that ini file there's a peak.running.shortcuts section. It makes interesting reading when you've got few moments. You'll find our friend runIni in there, as well as the n2 we just learned about.

Right now, though, we're interested in that second bit, the one about the URL. PEAK provides a general way to refer to "resources" via "names". Since "URL" stands for "Uniform Resource Locator", it's a natural syntax for PEAK to use. A URL is divided into multiple parts; we're only worried about the first two for now. The first part is what comes before the first ":", and the second part is what comes after. The first part is called the "scheme", and the second is a path that, when interpreted in the context of the scheme by the naming system, will yeild an object.

The peak command further requires that the referenced object support one of the `peak.running' interfaces or be callable. That's important: the naming system will look up anything it understands, but something that uses the naming system to look things up may have additional constraints on what referenced objects will actually work when specified in such a URL.

PEAK of course makes schemes and production rules completely configurable. The peak.naming.schemes section of the peak.ini file looks like this:

[peak.naming.schemes] 
 
# This section defines naming context factories or URL.Base subclasses to 
# be used for various URL schemes.  The entry name is the URL scheme, and 
# the value is either the object or an import string for loading it. 
 
import  = "peak.naming.factories.peak_imports:importContext" 
smtp    = "peak.naming.factories.smtp:smtpURL" 
uuid    = "peak.naming.factories.uuid:uuidURL" 
nis     = "peak.naming.factories.nisns:nisURLContext" 
config  = "peak.naming.factories.config_ctx:PropertyContext" 
 
ldap    = "peak.storage.LDAP:ldapURL" 
sybase  = "peak.storage.SQL:GenericSQL_URL" 
pgsql   = "peak.storage.SQL:GenericSQL_URL" 
psycopg = "peak.storage.SQL:GenericSQL_URL" 
mockdb  = "peak.storage.SQL:GenericSQL_URL" 
gadfly  = "peak.storage.SQL:GadflyURL" 
sqlite  = "peak.storage.SQL:SqliteURL" 
cxoracle = "peak.storage.SQL:OracleURL" 
dcoracle2 = "peak.storage.SQL:OracleURL" 
 
logfile = "peak.running.logs:logfileURL" 
logging.logger = "peak.running.logs:peakLoggerContext" 
 
lockfile     = "peak.running.lockfiles:lockfileURL" 
nulllockfile = "peak.running.lockfiles:lockfileURL" 
shlockfile   = "peak.running.lockfiles:lockfileURL" 
flockfile    = "peak.running.lockfiles:lockfileURL" 
winflockfile = "peak.running.lockfiles:lockfileURL" 
win32.dde    = "peak.storage.DDE:ddeURL" 
 
http    = "peak.naming.factories.openable:OpenableURL" 
ftp     = "peak.naming.factories.openable:OpenableURL" 
https   = "peak.naming.factories.openable:OpenableURL" 
file    = "peak.naming.factories.openable:FileURL" 
pkgfile = "peak.naming.factories.openable:PkgFileURL" 
icb     = "peak.net.icb:ICB_URL" 
 
 
 
tcp       = "peak.net.sockets.tcpudpURL" 
udp       = "peak.net.sockets.tcpudpURL" 
unix      = "peak.net.sockets.unixURL" 
unix.dg   = "peak.net.sockets.unixURL" 
fd.socket = "peak.net.sockets.fdURL" 
 
zconfig.schema = "peak.config.load_zconfig:ZConfigSchemaContext" 
shellcmd       = "peak.naming.factories.shellcmd:ShellCommandCtx" 
That's a lot of schemes. Take a look at that last one, `shellcmd'. The peak command usage says we can use any scheme recognized by peak.naming to provide the thing the peak command is going to run. Presumably that means that we could tell it to run, say, the "ls -l" command. Let's try it:
% peak shellcmd:"ls -l" 
total 15 
-rwxr-xr-x  1 rdmurray  wheel   272 Dec  7 22:04 hello 
-rw-r--r--  1 rdmurray  wheel   159 Dec  7 22:04 hello.list 
-rw-r--r--  1 rdmurray  wheel   201 Dec  7 22:04 hello_model.py 
-rw-r--r--  1 rdmurray  wheel  1013 Dec  7 22:04 hello_model.pyc 
-rw-r--r--  1 rdmurray  wheel  1097 Dec  7 22:04 hello_storage.py 
-rw-r--r--  1 rdmurray  wheel  3308 Dec  7 22:04 hello_storage.pyc 
-rw-r--r--  1 rdmurray  wheel  1519 Dec  7 22:04 helloworld.py 
-rw-r--r--  1 rdmurray  wheel  2878 Dec  7 22:04 helloworld.pyc 
Well, how about that. Not, perhaps, a stunningly useful way to run the "ls" command, but it certainly demonstrates the flexibility of the PEAK naming system.

The n2 subcommand we used in the last section also takes any argument supported by peak.naming. The referenced object must support a certain interface, but we'll ignore that for now.

When we used n2 to open a connection to our SQLite "test.db" database, we were using the "sqlite" scheme listed above. You'll note that there are several other database schemes supported by PEAK. That's important, because as you'll recall the corporate database our program really needs to connect to is Postgres. Fortunately, psycopg is a scheme for accessing a Postgres database, so we're all set.

In fact, if we actually had that corporate database to connect to, we could do something like:

% peak n2 psycopg://ouruserid:ourpassword@some.corporate.server.com/customerdb 
and run some queries against that table and find out what kind of data they are really expecting us to interface with.

Now, this is all well and good, but so far it hasn't gotten us any nearer making our program get group data from an SQL database. But PEAK wouldn't have a naming system if it didn't use it everywhere, of course, so you probably won't find it too surprising to that the naming.Obtain function also accepts anything handled by peak.naming, as long as it supports the IComponentKey interface or can be adapted to it.

Hmm. "...or can be adapted to it." Now explaining that would require quite a long digression. A very important digression if you really want to understand the essence of PEAK, but one that is far too long to get in to here. You can read [WWW]PEP 246 and/or the [WWW]PyProtocols [WWW]documentation if you want to delve into it now.

But all you really need to know for our purposes here is that within the PEAK system we have a way of adding behavior to objects in a very flexible way. So if we have an object that represents a connection to a database, and Obtain wants it to support IConfigKey, a little piece of code will make it do so in a way transparent to us as users of that particular service. What this means in practice is that in general when something uses the naming system to look things up, and it seems reasonable to use an object represented by a given URL, but you don't know if that object supports whatever interface the component doing the lookup requires, try it. There may be an adpater in the system that will take care of it. And if it doesn't work, but it still seems reasonable that it should, you can write an adapter so it will work. But that, as they say, is a topic for another day. (Or if I can keep at this long enough, another chapter.)

What this all means right now is that we can put something in our HelloWorld class like this:

db = Obtain("sqlite:test.db") 
and like magic our program will have a connection to our SQLite database.

Of course, that's just the beginning....

4.3 Greater Configurability

Before we start accessing that database, let's think a little about the fact that this is a test database we're working with. We know we're going to have to switch which database we access eventually. And that's something that could change in the future; after all, they might move the database to some new machine someday.

So really we should put the pointer to the database into the configuration file instead of burying it in the code. The naming system provides a little utility for facilitating this called LinkRef:

[helloworld] 
 
messagefile = config.fileNearModule('helloworld', 'hello.list') 
db = naming.LinkRef('sqlite:test.db') 
So now we've got a "link" to a named resource stored in our helloworld.customerdb configuration variable.

Hmm. But that isn't really quite what we want, either. After all, our project group is working on other projects using PEAK, and some of those will need to access to this same corporate database. It would be a shame to have to go to the config file for each app and adjust it if the database gets moved.

PEAK of course provides a way to deal with that. We can create a special section in our ini file named Load Settings From, and in that define a file variable pointing to another ini file. (Footnote: actually, you might want to use ZConfig here rather than an ini file, since ZConfig is probably better for "end user" configuration tasks, but that's another topic I'm not ready to delve in to.)

So we'll create a new file, global.ini:

[corporate] 
 
customerdb = naming.LinkRef('sqlite:test.db') 
And at the top of our hello file:
[Load Settings From] 
file = config.fileNearModule('helloworld', 'global.ini') 
In our deployed application we'll actually want to use either a hardcoded path name, or reference to a module related to the shared parts of our corporate PEAK applications.

This leads to another interesting point, though. If this global.ini file were already deployed, and we needed some of the settings in it, but we wanted to use our test database rather than the real customerdb, what could we do? Naturally enough, all it takes is to add a [corporate] section to our hello config file, and define in there the config variables we want to override.

4.4 Reading from the SQL Database

We're finally ready to use the database.

Before we do this the right way, though, lets conduct a little exercise. PEAK is designed to allow you to develop applications that are flexible and easy to modify. Suppose we were trying to do this database change on an emergency basis, and we wanted to touch as little of our existing system as possible? Let's see what we can do.

Our domain model pairs names with greetings. As far as the user of the to subcommand goes, that's all that matters. So lets see if we can maintain that pairing while using the data from our new database.

First, we'll need a reference to the database as a class variable in our DM.

    1 customerdb = binding.Obtain(PropertyName('corporate.customerdb'))

Next, we'll need to change our _load method to get the right message, using auxiliary info from the customer data base:

    1     def _load(self, oid, ob):
    2         row = ~self.customerdb("select GRP from custgroups where NAME='%s'" %
    3             oid)
    4         m = self.__findMatch(row.GRP, self.file.text)
    5         msg = m.group(1) % oid
    6         return {'forname': oid, 'text': msg}
Here you see that we can pass our SQL database object an SQL query, and for a select get back a cursor into the database. Since we're sure we're only going to get one record back, we use the unary operator "oneOf" ("~") to pick it out. The "row" object we get back has our select columns as attributes. So we get the data from the GRP column for this user, and use that to look up our message. You'll note that I'm back to doing "%" substitution on the resulting message, so that we are in fact personalizing our message for each customer. If corporate doesn't like that, it's easy enough to back out.

Nothing else in our application has to change. Although our usage message for the for subcommand is now wrong (it sets the message for the group, not the thing being greeted), it works just fine for storing messages into our database under the group names:

% ./hello for vip:"I'm so pleased to see you, %s" 
% ./hello to Jeff 
I'm so pleased to see you, Jeff 
Another nice feature worth pointing out, even though it isn't useful in our simple example, is that the SQL database object participates in the transaction. (In fact, there has to be an active transaction for the sql database function to work.) So if our subcommands were in fact updating the SQL database, both the changes to it and the changes to our EditableFile would be kept consistent across PEAK transaction boundaries. If our PEAK transaction aborts, all database changes, to whichever database, are rolled back (assuming the database supports transaction abort, of course).

And to go production, all we need to do is change that customerdb declaration in the configuration file. The PEAK database object hides all the details of the different databases from us. (Well, as long as we can stick to standard SQL it does, anyway.)

If you think about what you just learned about how PEAK can be used to tie together disparate, possibly legacy databases in the corporate setting, and maintain enterprise-level control over the integrity of the data, you can see why PEAK is an "Enterprise Application Toolkit".

4.5 Doing it Right

All right, that incorrect usage message is bothering me. Let's fix this application the right way.

Our domain model has actually changed: instead of the domain consisting of "Message" objects that are associated with particular things to greet, we now have "Customers" that belong to "Groups", and it is the groups that have messages associated with them.

So, if we are going to bring our app in tune with the new reality, we'll have to start by revising our domain model:

    1 from peak.api import *
    2 
    3 
    4 class Group(model.Element):
    5 
    6     class name(model.Attribute):
    7         referencedType = model.String
    8 
    9     class text(model.Attribute):
   10         referencedType = model.String
   11 
   12 
   13 class Customer(model.Element):
   14 
   15     class name(model.Attribute):
   16         referenceType = model.String
   17 
   18     class group(model.Attribute):
   19         referenceType = Group

The most interesting thing to notice about this new model is that the reference type for the group attribute of our Customer is actually the Group element we define. When we use this model, we're going to actually store references to Group objects in the group attribute of the Customer objects. But we'll do it the PEAK way.

As long as we're rewriting anyway, we might as well get rid of that clunky file and move to a real database for our messages. Perhaps someday they'll move into a table on the corporate database, but for now we'll stick with our SQLite database, since has proven its worth so far.

We'll replace the messagefile configuration item in our hello file with one for the new database:

messagedb = naming.LinkRef('sqlite:messages.db') 

Our hello_storage.py file will see the greatest amount of change. Let's start with the simpler class, the Data Manager for the Customers. This data is read only, so we can use QueryDM here:

    1 from peak.api import *
    2 from hello_model import Customer, Group
    3 
    4 class CustomerDM(storage.QueryDM):
    5 
    6     defaultClass = Customer
    7     customerdb = binding.Obtain(PropertyName('corporate.customerdb'))
    8     GroupDM = binding.Obtain(storage.DMFor(Group))
    9 
   10     def _load(self, oid, ob):
   11         row = ~self.customerdb("select GRP from custgroups where NAME='%s'" %
   12             oid)
   13         group = self.GroupDM[row.GRP]
   14         return {'name': oid, 'group': group}
As you can see, there's some interesting stuff happening here.

First, we're using the binding system to pick up a reference to the Data Manager for the group. Even though we don't define that until later in the file, we don't have a reference problem, because the dereference doesn't happen until runtime.

Second, we're returning an instance retreived from the GroupDM as the value of one of our attributes. This kind of arrangement is why Data Managers return ghosts: when we access a Customer object, even though it contains a reference to a Group object, no system activity (ie: reading from databases) happens because of this until we actually try to reference an attribute of the Group. Thus in PEAK inter-object links are not expensive, even when the other objects are backed by potentially slow external databases and/or occupy large amounts of memory when active.

OK, here's the GroupDM, which is not any less complicated than our old MessageDM from before we added the customers, but which shows how to use SQL to do the same stuff:

    1 class GroupDM(storage.EntityDM):
    2 
    3     defaultClass = Group
    4     messagedb = binding.Obtain(PropertyName('helloworld.messagedb'))
    5 
    6     def _load(self, oid, ob):
    7         row = ~self.messagedb("select text from messages where name='%s'" %
    8             oid)
    9         return {'name': oid, 'text': row.text}
   10 
   11     def _new(self, ob):
   12         if ob.name in self:
   13             raise KeyError, "%s is already in the database" % ob.name
   14         self._save(ob)
   15 
   16     def _save(self, ob):
   17         self.messagedb(("insert or replace into messages (name, text) values "
   18             "('%s', '%s')") % (ob.name, ob.text))
   19 
   20     def __contains__(self, oid):
   21         #using where would be more efficient here, but I don't
   22         #know how to test the cursor to see if I got any results
   23         for row in self.messagedb('select name from messages'):
   24             if row.name==oid: return 1
   25         return 0

Now, of course, our application needs to use the new domain objects:

    1 from peak.api import *
    2 from hello_model import Customer, Group
    3 
    4 class HelloWorld(commands.Bootstrap):
    5 
    6     usage = """
    7 Usage: hello command arguments
    8 
    9 Available commands:
   10 
   11     for -- sets a greeting
   12     to  -- displays a greeting
   13 """
   14 
   15     CustomerDM = binding.Make('hello_storage:CustomerDM',
   16         offerAs=[storage.DMFor(Customer)])
   17     GroupDM = binding.Make('hello_storage:GroupDM',
   18         offerAs=[storage.DMFor(Group)])
   19     #Does this actually do anything useful?:
   20     CustomerDB = binding.Obtain(PropertyName('corporate.customerdb'),
   21         offerAs=[PropertyName('corporate.customerdb')])
   22     MessageDB = binding.Obtain(PropertyName('helloworld.messagedb'),
   23         offerAs=[PropertyName('helloworld.messagedb')])
   24 
   25 
   26 class toCmd(commands.AbstractCommand):
   27 
   28     usage = """
   29 Usage: hello to <name>
   30 
   31 Displays the greeting for "name".
   32 """
   33 
   34     Customers = binding.Obtain(storage.DMFor(Customer))
   35 
   36     def _run(self):
   37         if len(self.argv)<2: raise commands.InvocationError("Missing name")
   38         storage.beginTransaction(self)
   39         print >>self.stdout, self.Customers[self.argv[1]].group.text
   40         storage.commitTransaction(self)
   41 
   42 
   43 class forCmd(commands.AbstractCommand):
   44 
   45     usage = """
   46 Usage: hello for <group>: <greeting>
   47 
   48 Stores "greeting" as the greeting message for group "group".
   49 """
   50 
   51     Groups = binding.Obtain(storage.DMFor(Group))
   52 
   53     def _run(self):
   54         if len(self.argv)<2: raise commands.InvocationError("Missing arguments")
   55         parts = ' '.join(self.argv[1:]).split(':')
   56         if len(parts)!=2: raise commands.InvocationError("Bad argument format")
   57         groupname = parts[0].strip(); message = parts[1].strip()
   58         storage.beginTransaction(self)
   59         if groupname in self.Groups:
   60             group = self.Groups[groupname]
   61         else:
   62             group = self.Groups.newItem()
   63             group.name = groupname
   64         group.text = message
   65         storage.commitTransaction(self)

The changes here are relatively small, almost trivial. The biggest change is that we are getting the message from the text attribute of the group attribute of a Customer object. The changes in for are mostly to the usage string and some variable name changes, just to be consistent.

If we test this program, though, we'll quickly find a design error:

% ./hello for vip:"I am so happy to see you, %s" 
% ./hello to Jeff 
I am so happy to see you, %s 
Woops, we forgot about our tailored messages.

How to solve this? Well, clearly a tailored message is more a property of the Customer proper. So we can add something to our domain model: a greeting function. And we can take advantage of this to deal with the case where our bosses are serious about not wanting tailored messages. We'll allow for messages that don't have a substitution point:

%!python 
class Customer(model.Element): 
 
    class name(model.Attribute): 
        referenceType = model.String 
 
    class group(model.Attribute): 
        referenceType = Group 
 
    def greeting(self): 
        if '%' in self.group.text: 
            return self.group.text % self.name 
        else: return self.group.text 

Now our print line in helloworld.py becomes:

print >>self.stdout, self.Customers[self.argv[1]].greeting() 
Which is more Demeter-proof anyway.

And now everyone is happy:

% ./hello to Jeff 
I am so happy to see you, Jeff 
% ./hello for vvip:"Greetings, Your Excelency!" 
% ./hello to Jackie 
Greetings, Your Excelency! 

5 Exploring the rest of PEAK

One important topic we haven't covered here is the "component hierarchy", although we've alluded to it on occasion by talking about "parent" and "child" components. Generally speaking, a component contained within another component is a "child" component of the container, and PEAK provides mechanisms for components to access their containers to "obtain" configuration or other components. It's also possible to treat the component hierarchy like a kind of file system, using paths to navigate from one component to another.

As you begin to build more sophisticated applications, you'll probably want to have more understanding of how components are attached to their parent components, and how they can receive notification that they've been attached. And, if you want to create applications that run without using the peak script, you'll also want to know about "root components" and how to create them with config.makeRoot.

In the meantime, to get you started on your journey, you may wish to explore the [WWW]current API documentation. There, you'll find a list of some of the major features provided by other PEAK frameworks, and quick-reference documentation for them. Also, if you want to look up an API feature like PropertyName or binding.Obtain, you can use the peak help command, e.g.:

% peak help binding.Obtain 
 
Help on class Obtain in module peak.binding.components: 
 
class Obtain(peak.binding.once.Attribute) 
 |  'Obtain(componentKey,[default=value])' - finds/caches a needed component 
 | 
 |  Usage examples:: 
 | 
 |      class someClass(binding.Component): 
 | 
 |          thingINeed = binding.Obtain("path/to/service") 
 |          otherThing = binding.Obtain(IOtherThing) 
 |          aProperty  = binding.Obtain(PropertyName('some.prop'), default=42) 
 | 
 |  'someClass' instances can then refer to their attributes, such as 
 |  'self.thingINeed', instead of repeatedly calling 
 |  'self.lookupComponent(someKey)'. 
 | 
 |  The initial argument to the 'Obtain' constructor must be adaptable to 
 |  'binding.IComponentKey'.  If a 'default' keyword argument is supplied, 
 |  it will be used as the default in case the specified component key is not 
 |  found. 
 | 
 ... etc. 

So have fun with PEAK, and I'll see you at the top!


PythonPowered
EditText of this page (last modified 2003-12-09 16:51:12)
FindPage by browsing, title search , text search or an index
Or try one of these actions: AttachFile, DeletePage, LikePages, LocalSiteMap, SpellCheck