The PEAK Developers' Center   Diff for "IntroToPeak/LessonFour" UserPreferences
 
HelpContents Search Diffs Info Edit Subscribe XML Print View Up
Ignore changes in the amount of whitespace

Differences between version dated 2003-12-20 02:29:26 and 2005-02-03 13:16:14 (spanning 25 versions)

Deletions are marked like this.
Additions are marked like this.

small glimpse of how powerful PEAK's naming system is. Oh, yeah,
and we'll exercise the configuration system a bit more, too.
 
Then we'll finish it all off by throwing away our version 1
demo program and writing version 2 to match the new reality.
Then we'll finish it all off by throwing away our version 1 model
and storage implementation program and writing version 2 to match
the new reality.
 
To do our testing, we'll use an SQLite database. If you don't
have SQLite on your system, you can get it from

python interface library, available from
[http://sourceforge.net/projects/pysqlite/]. Or use your operating
system's package facility if it has one (for example, on FreeBSD
you could do "cd /usr/ports/database/py-PySQLite; make install".
you could do "cd /usr/ports/database/py-PySQLite; make install" and on Gentoo Linux you can "emerge pysqlite").
 
'''Contents'''
 

and URL is "of a type supported by peak.naming".
 
You should already be familiar with 'peak.running.shortcuts',
including how to define your own subcommands using it, from chapter
including how to define your own subcommands using it, from Lesson
3. Now we're interested in that second bit, the one about the URL.
 
PEAK provides a general way to refer to "resources" via "names".

%peak sqlite:test.db
<peak.storage.SQL.SQLCursor object at 0x82f160c>
}}} Well, that doesn't seem very useful. But it did ''something''.
It opened at `test.db` database and returned an object, as
It opened a `test.db` database and returned an object, as
promised. In this case, an object representing a "database
cursor" pointing to the database.
 

=== Basics ===
 
"N2" stands for "Namespace Navigator", and it is a general purpose
tool for browsing and manipulating any namespace for which someone has
written a navigator adapter. It is available as a subcommand of the
tool for browsing and manipulating namespaces, and it will also
allow you to interact with any resource for which someone has written
a navigator adapter. It is available as a subcommand of the
`peak` command: {{{
%peak help n2
class N2(peak.running.commands.AbstractCommand)

set up our test SQL database so we can test our modidified program
against it before we try it on the corporate database.
 
=== Namespace Navigation ===
=== Database Navigation ===
 
The first thing we need to do is figure out the tables, columns.
and data types from the corporate database with which we'll need
to interact, so we can set up our test database to emulate them.
So let's use the namespace navigation features of n2 to take
So let's use the SQL interaction features of n2 to take
a peak into the corporate database: {{{
%peak n2 psycopg:
psycopg://ouruserid:ourpassword@bigserver.example.com/customerdb

}}}
You'll note that the prompt is different this time. This is
because we are talking to a different "Interactor". The
default interactor is the one for exploring python namespaces.
default interactor is the one for exploring PEAK namespaces.
But because we referenced an object that supports an SQL
Interface, n2 picked up the sql interactor and is using it
Interface, n2 picked up the sql Interactor and is using it
instead.
 
{{{

We can enter SQL commands, terminated by semi-colons, and they will
be applied against the databse.
 
We're looking for the tables and columns relevant to our
application. Since this is a namespace explorer, there ought
to be a way to list the namespace. In the python interactor
there was that attractive looking `ls` command, but we don't
have that here.
We're looking for the tables and columns relevant to our application.
There ought to be a way to list things in the database. In the
python interactor there was that attractive looking `ls` command,
but we don't have that here.
 
`describe` sounds like it might be interesting. Let's try
that:{{{
1> help \describe
help: no such command: \describe
}}} Well, we know there is one. Perhaps that `\` is just
syntactic sugar? {{{
1> help describe
\describe [-d delim] [-m style] [-h] [-f] [-v] [name] -- describe objects in database, or named object
 
-d delim use specified delimiter

-h suppress header
-f suppress footer
-v verbose; give more information
}}} That's better.
 
OK, so we can use this to get information about named objects.
}}} OK, so we can use this to get information about named objects.
But name is optional, in which case it describes the objects
in the database. Sounds like what we want. {{{
1> \describe

}}} Ah, good. Looks like "customers" is probably the table
we want. Let's see if we can find out more about it. {{{
1> \describe customers
Feature not implemented yet.
1>
}}} That's not as helpful. Perhaps what we need is that "verbose"
option on the general \describe command. {{{

 
 
(0 rows)
1> select * from custgruops;
1> select * from customers;
NAME GRP
-------------------- --------------------
Jeff vip

}}}
Ok, so we didn't get an error, and we got back a cursor.
Let's see if it is iterable: {{{
>>> for i in c('select * from custgroups'):
>>> for i in c('select * from customers'):
... print i
...
('Jeff', 'vip')

('Fred', 'ordinary')
}}} Well, that's pretty cool. Here is another
thing you might expect to have work, that does: {{{
>>> for i in c('select * from custgroups'):
>>> for i in c('select * from customers'):
... print "Name: %-10s Group: %-10s" % (i.NAME, i.GRP)
...
Name: Jeff Group: vip

Suppose you have something, say a connection to an SQL database.
But what you (in this case, "you" is the n2 subcommand) really want
is an object that has a bunch of command hooks on it for interacting
with an object that provides a namespace. That set of "hooks"
(method calls and attributes) constitutes a "protocol" for talking
to the object in a well defined way and getting well defined results.
The SQL connection object, by virtue of providing an interface
(another protocol) for talking with an SQL database, clearly provides
a namespace such as n2 is looking for. But it doesn't have the
specific hooks that would make it really easy for n2 to talk to it.
with the object. That set of "hooks" (method calls and attributes)
constitutes a "protocol" for talking to the object in a well defined
way and getting well defined results. The SQL connection object,
by virtue of providing an interface (another protocol) for talking
with an SQL database, clearly provides the ability to interact in
the way n2 is looking for. But it doesn't have the specific hooks
that would make it really easy for n2 to talk to it.
 
So what n2 does is it hands the `PyProtocols` subsystem the SQL
So what n2 does is to hand the `PyProtocols` subsystem the SQL
connection object it has, and asks for an object that supports the
protocol it wants (`IN2Interactor`) in return. `PyProtocols` looks
in the table that has been built through the declarations made in
various modules, and find a pointer to a piece of code that can do
various modules, and finds a pointer to a piece of code that can do
just what n2 wants: wrap up an SQL connection object in another
object that can act as an intermediary (an "adapter") between the
protocol n2 wants and the one the SQL connection object actually

n2 code asks for it.
 
The fantastic thing about this arrangement is that someone can come
along later and write an adpater that takes, say, an `IIMAPConnection`
along later and write an adapter that takes, say, an `IIMAPConnection`
and provides an `IN2Interactor` interface. They can declare this
adapter in some module completely apart from either n2 or the `IMAP`
module. And then, supposing there is already a naming scheme for

we'd be able to say {{{
% peak n2 imap:ourname:ourpass@imaphost.example.com
>
}}} and be off and exploring our IMAP namespace.
}}} and be off and exploring our IMAP folders.
 
 
== Greater Configurability ==

Yes, this is another example of PEAK using adaptation, so you can
see why I took some time to cover it. In this case, the `Obtain`
method, when handed an argument, simply tries to adapt it to the
`IComponentKey` protocol. Many things are adaptable to IComponentKey
`IConfigKey` protocol. Many things are adaptable to `IConfigKey`
including, in this case, strings such as `sqlite:test.db`.
 
But before we start accessing the test database from our code, let's

messagefile = config.fileNearModule('helloworld', 'hello.list')
db = naming.LinkRef('sqlite:test.db')
}}} So now we've got a "link" to a named resource stored in
our `helloworld.customerdb` configuration variable.
our `helloworld.db` configuration variable.
 
Hmm. But that isn't really quite what we want, either. After all,
our project group is working on other projects using PEAK, and some

our new database.
 
First, we'll need a reference to the database as a class
variable in our DM. {{{
variable in our Data Manager. {{{
#!python
customerdb = binding.Obtain(PropertyName('corporate.customerdb'))
}}}

Next, we'll need to change our `_load` method to get the right
message, using auxiliary info from the customer data base. Remember,
our database connection object is callable; if we pass it an SQL
command, we'll get back an interable cursor containing the results:
command, we'll get back an iterable cursor containing the results:
{{{
#!python
    def _load(self, oid, ob):
        row = ~self.customerdb("select GRP from custgroups where NAME='%s'" %
        row = ~self.customerdb("select GRP from customers where NAME='%s'" %
            oid)
        m = self.data[row.GRP] % oid
        return {'forname': oid, 'text': msg}
        m = self.data[row.GRP]['text'] % oid
        return {'forname': oid, 'text': m}
}}} What's that funny looking `~` doing in front of our database
call? Since we're sure we're only going to get one record back,
we use the unary operator "oneOf" ("~") to pick it out. Using this
operator also has the advantage that if we ''don't'' get one and
only one row back, it will throw an error.
call? The `ICursor` interface, which is an interface supported by
the type of object returned by a DM (Data Manager, not to be confused with Domain Model),
defines the python unary
negation operator to be the function `oneOf`. `oneOf` will raise
an error if there is anything other than one row accessable from
the cursor. If there is only one row, it returns it.
 
Since we're sure we're only going to get one record back, we use
`oneOf` ("~") to pick it out. Using this operator also has the
advantage that if we ''don't'' get one and only one row back, it
will throw an error. This follows the PEAK philosphy of failing
as soon as possible when something that shouldn't happen, does.
 
As we saw above, the "row" object we get back has our select columns
as attributes. So we get the data from the `GRP` column for this

% ./hello for vip:"I'm so pleased to see you, %s"
% ./hello to Jeff
I'm so pleased to see you, Jeff
}}}
Another nice feature worth pointing out, even though it isn't useful
in our simple example, is that the SQL database object participates
in the transaction. (In fact, there has to be an active transaction
for the sql database function to work.) So if our subcommands
were in fact updating the SQL database, both the changes to it and
the changes to our `EditableFile` would be kept consistent across
PEAK transaction boundaries. If our PEAK transaction aborts,
''all'' database changes, to whichever database, are rolled back
(assuming the database supports transaction abort, of course).
}}} Now, the fact that this simple change actually works is
due to a quirk in our implementation. When we set a new message
for a group, our `get` function finds that this `forname` doesn't
exist, so our `toCmd` creates a new object, thereby avoiding
the error we'd otherwise get when it tried to load the non-existent
`forname`. This is logical, but the quirk that makes the app
still work is that our storage implementation for the message
database will ''update'' the message even though we did a
``newItem`` to get the object. This is not something that a
good app design should depend on, so we'll fix it right in a
moment.
 
The moral of this story is that while PEAK's excellent
separation of concerns often makes it ''possible'' to finess
a fix, one should not, in general, try to bend an incorrect
domain model to a purpose at variance with its design. Instead,
the domain model should be fixed.
 
Before we do that, though, let me point out another nice feature
of PEAK, even though it isn't useful in our simple example. Observe
that the SQL database object participates in the transaction. (In
fact, there has to be an active transaction for the sql database
function to work.) If our subcommands were in fact updating
the SQL database, both the changes to it and the changes to our
`EditableFile` would be kept consistent across PEAK transaction
boundaries. If our PEAK transaction aborts, ''all'' database
changes, to whichever database, are rolled back (assuming the
database supports transaction abort, of course).
 
And to go production, all we need to do is change that customerdb
Also, to go production all we need to do is change that customerdb
declaration in the configuration file. The PEAK database object
hides all the details of the different databases from us. (Well,
as long as we can stick to standard SQL it does, anyway.)

 
== Doing it Right ==
 
All right, that incorrect usage message is bothering me. Let's fix
this application the right way.
All right, enough of taking shortcuts. Let's fix this application
the right way.
 
Our domain model has actually changed: instead of the domain
consisting of "Message" objects that are associated with particular

    class name(model.Attribute):
        referencedType = model.String
 
    class text(model.Attribute):
    class greetingtemplate(model.Attribute):
        referencedType = model.String
 
 
class Customer(model.Element):
 
    class name(model.Attribute):
        referenceType = model.String
        referencedType = model.String
 
    class group(model.Attribute):
        referenceType = Group
        referencedType = Group
}}}
 
The most interesting thing to notice about this new model is that

As long as we're rewriting anyway, we might as well get rid of that
clunky file and move to a real database for our messages. Perhaps
someday they'll move into a table on the corporate database, but
for now we'll stick with our SQLite database, since has proven its
for now we'll stick with our SQLite database, since it has proven its
worth so far.
 
We'll replace the `messagefile` configuration item in our `hello` file

messagedb = naming.LinkRef('sqlite:messages.db')
}}}
 
Our `hello_storage.py` file will see the greatest amount of change.
We also need to create the database and the `messages` table.
As when we created the `customers` database, `n2` is handy here: {{{
% peak n2 sqlite:messages.db
1> create table messages (name varchar(20), text varchar(80));
1> commit
}}}
 
Our `storage.py` file will see the greatest amount of change.
Let's start with the simpler class, the Data Manager for the
Customers. This data is read only, so we can use `QueryDM` here: {{{
#!python
from peak.api import *
from hello_model import Customer, Group
from model import Customer, Group
 
 
class CustomerDM(storage.QueryDM):
 

    GroupDM = binding.Obtain(storage.DMFor(Group))
 
    def _load(self, oid, ob):
        row = ~self.customerdb("select GRP from custgroups where NAME='%s'" %
        row = ~self.customerdb("select GRP from customers where NAME='%s'" %
            oid)
        group = self.GroupDM[row.GRP]
        return {'name': oid, 'group': group}

that until later in the file, we don't have a reference problem,
because the dereference doesn't happen until runtime.
 
Second, we're returning an instance retreived from the `GroupDM`
Second, we're returning an instance retrieved from the `GroupDM`
as the value of one of our attributes. This kind of arrangement
is why Data Managers return ghosts: when we access a `Customer`
object, even though it contains a reference to a Group object,

external databases and/or occupy large amounts of memory when
active.
 
OK, here's the `GroupDM`, which is not any less complicated
OK, here's the `GroupDM`, which is not much less complicated
than our old `MessageDM` from before we added the customers,
but which shows how to use SQL to do the same stuff: {{{
#!python

    defaultClass = Group
    messagedb = binding.Obtain(PropertyName('helloworld.messagedb'))
 
    def _getstate(self, oid):
 
        try:
            row = ~self.messagedb(
                        "select text from messages where name='%s'" % oid)
        except exceptions.TooFewResults:
            return None
 
        return {'name': oid, 'greetingtemplate': row.text}
 
    def _load(self, oid, ob):
        row = ~self.messagedb("select text from messages where name='%s'" %
            oid)
        return {'name': oid, 'text': row.text}
        state = self._getstate(oid)
        if not state: raise KeyError, "%s not in database" % oid
        return state
 
    def _new(self, ob):
        if ob.name in self:
        if self.get(ob.name):
            raise KeyError, "%s is already in the database" % ob.name
        self._save(ob)
        self.messagedb(("insert into messages (name, text) values "
            "('%s', '%s')") % (ob.name, ob.greetingtemplate))
        return ob.name
 
    def _save(self, ob):
        self.messagedb(("insert or replace into messages (name, text) values "
            "('%s', '%s')") % (ob.name, ob.text))
        self.messagedb("update messages set text='%s' where name='%s'" %
            (ob.greetingtemplate, ob.name))
 
    def __contains__(self, oid):
        #using where would be more efficient here, but I don't
        #know how to test the cursor to see if I got any results
        for row in self.messagedb('select name from messages'):
            if row.name==oid: return 1
        return 0
}}}
    def get(self, oid, default=None):
 
Now, of course, our application needs to use the new domain
        if oid in self.cache:
            return self.cache[oid]
 
        state = self._getstate(oid)
        if state: return self.preloadState(oid, state)
 
        return default
}}} We no longer need the `Flush` or `abortTransaction` methods,
because we no longer have any local data structures we need to keep
synced. The SQL connection will take care of flushing and aborting
the database.
 
We do, however, now actually need different methods for the `_new`
versus `_save` case, because the SQL commands to update a record
are very different from those used to add a record.
 
To implement a `get` method similar in efficiency to the one we had
for our file based database, we introduce a local helper method
`_getstate`. This is the method that actually reads from the
database and picks up the data. `get` uses this to see if a record
exists for the oid in question, and if it does uses `preloadState`
as before. `_load` uses this helper method to get the state and
return it. Note that now `_load` needs to raise an error if the
record doesn't exist; before, we were letting `oneOf` do that for
us. Doing it this way has the happy consequence that the error
message presented to the end users will be more intuitive. Note
also that if `oneOf` detects too many rows, ''that'' error will not
be caught, and the resulting error message will tell the user exactly
what they need to know (there are too many records for this oid in
the database).
 
Now, of course, our application needs to ''use'' the new domain
objects: {{{
#!python
from peak.api import *
from hello_model import Customer, Group
from helloworld.model import Customer, Group
 
class HelloWorld(commands.Bootstrap):
 

    to -- displays a greeting
"""
 
    CustomerDM = binding.Make('hello_storage:CustomerDM',
    acceptURLs = False
    CustomerDM = binding.Make('helloworld.storage:CustomerDM',
        offerAs=[storage.DMFor(Customer)])
    GroupDM = binding.Make('hello_storage:GroupDM',
    GroupDM = binding.Make('helloworld.storage:GroupDM',
        offerAs=[storage.DMFor(Group)])
    #Does this actually do anything useful?:
    #RDM: Do the next to actually do anything useful?
    CustomerDB = binding.Obtain(PropertyName('corporate.customerdb'),
        offerAs=[PropertyName('corporate.customerdb')])
    MessageDB = binding.Obtain(PropertyName('helloworld.messagedb'),

    def _run(self):
        if len(self.argv)<2: raise commands.InvocationError("Missing name")
        storage.beginTransaction(self)
        print >>self.stdout, self.Customers[self.argv[1]].group.text
        name = self.argv[1]
        print >>self.stdout, self.Customers[name].group.greetingtemplate
        storage.commitTransaction(self)
 
 

    Groups = binding.Obtain(storage.DMFor(Group))
 
    def _run(self):
        if len(self.argv)<2: raise commands.InvocationError("Missing arguments")
        parts = ' '.join(self.argv[1:]).split(':')
        if len(parts)!=2: raise commands.InvocationError("Bad argument format")
        groupname = parts[0].strip(); message = parts[1].strip()
    
        if len(self.argv)<2:
            raise commands.InvocationError("Missing arguments")
 
        parts = ' '.join(self.argv[1:]).split(':',1)
        if len(parts)!=2:
            raise commands.InvocationError("Bad argument format")
 
        groupname, template = [part.strip() for part in parts]
 
        storage.beginTransaction(self)
        if groupname in self.Groups:
            group = self.Groups[groupname]
        else:
            group = self.Groups.newItem()
 
        group = self.Groups.get(groupname)
 
        if group is None:
            group = self.Groups.newItem()
            group.name = groupname
        group.text = message
 
        group.greetingtemplate = template
 
        storage.commitTransaction(self)
 
}}}
 
The changes here are relatively small, almost trivial. The biggest
change is that we are getting the message from the text attribute
of the `group` attribute of a `Customer` object. The changes in
`for` are mostly to the usage string and some variable name changes,
just to be consistent.
change is that we are getting the message from the greetingtemplate
attribute of the `group` attribute of a `Customer` object. The
changes in `for` are mostly to the usage string and some variable
name changes, just to be consistent.
 
If we test this program, though, we'll quickly find a design error: {{{
% ./hello for vip:"I am so happy to see you, %s"

advantage of this to deal with the case where our bosses are
serious about not wanting tailored messages. We'll allow
for messages that don't have a substitution point: {{{
%!python
#!python
class Customer(model.Element):
 
    class name(model.Attribute):
        referenceType = model.String
        referencedType = model.String
 
    class group(model.Attribute):
        referenceType = Group
        referencedType = Group
 
    def greeting(self):
        if '%' in self.group.text:
            return self.group.text % self.name
        else: return self.group.text
        if '%' in self.group.greetingtemplate:
            return self.group.greetingtemplate % self.name
        else: return self.group.greetingtemplate
}}}
 
Now our `print` line in `helloworld.py` becomes:{{{
print >>self.stdout, self.Customers[self.argv[1]].greeting()
Now our `print` line in `commands.py` becomes:{{{
print >>self.stdout, self.Customers[name].greeting()
}}} Which is more Demeter-proof anyway.
 
And now everyone is happy:{{{
% ./hello to Jeff
I am so happy to see you, Jeff
% ./hello for vvip:"Greetings, Your Excelency!"
% ./hello for vvip:"Greetings, Your Excellency!"
% ./hello to Jackie
Greetings, Your Excelency!
Greetings, Your Excellency!
}}}
 
Up: IntroToPeak Previous: IntroToPeak/LessonThree Next: IntroToPeak/LessonFive

PythonPowered
ShowText of this page
EditText of this page
FindPage by browsing, title search , text search or an index
Or try one of these actions: AttachFile, DeletePage, LikePages, LocalSiteMap, SpellCheck