Additions are marked like this.

two separate commands, but then we wouldn't have an excuse to

from helloworld.model import Message

    Messages = binding.Make(

        'helloworld.storage.MessageDM', offerAs=[storage.DMFor(Message)]

the subcommands will be able to `Obtain` the DM from their context,

as discussed in the last chapter.

The only other thing it's got is a `usage` class variable. To see

how this is used, try typing `./hello` at your command prompt: {{{

tasks associated with writing a script.

''same'' `MessageDM` instance as the one in the associated

Hmm. So, if something defined in a particular "property namespace"

affects the way the peak command behaves, that must mean that the

peak command has some place to get those properties, which means

it probably has an ini file. Sure enough, a little poking around the

peak directories will reveal a `peak.ini` file. In that file we

can find a section called `[peak.running.shortcuts]`, containing a

bunch of properties called `runIni`, `help`, and many other commands.

to = importString('helloworld.commands.toCmd')

#!python

        'import:helloworld.commands.toCmd',

after all, no arguments might be required. It ''will'' automatically

#!python

`[peak.running.shortcuts]` section, doesn't mean that the settings in

to = importString('helloworld.commands.toCmd')

provides a way to turn that behavior off, though. We just need to

override a flag in our `HelloWorld` class: {{{

 

    # rest of HelloWorld class goes here...

 

Now, we know we're going to have to rewrite our `storage.py` to

allow us to write to the database, but let's start this part of the

virtually independent of how we go about initiating the save

to = importString('helloworld.commands.toCmd')

for = importString('helloworld.commands.forCmd')

}}} and another `AbstractCommand` subclass in `commands.py`

#!python

 

 

 

 

 

        storage.commitTransaction(self)

a new "empty" object, using `newItem()`. (Actually, it can have a

preloaded default state, but we'll ignore that for now). Then we

modify it just like we would any other writable object we got from

the Data Manager, and the transaction machinery takes care of

getting the data written to the backing store at transaction

commit time.

OK, it's time to do some serious surgery on our Data Manager.

First, we need to exchange our `QueryDM` base class for a base

class that supports updating the database. That would be

`storage.EntityDM`.

object is added to the DM and needs to store the data for that

Let's write the new `storage.py`: {{{

from helloworld.model import Message

    filename = binding.Obtain(PropertyName('helloworld.messagefile'))

    def data(self):

        data = {}

        file = open(self.filename)

        for line in file:

            fields = [field.strip() for field in line.split('|',1)]

            forname, text = fields

            data[forname] = {'forname': forname, 'text': text}

        file.close()

        return data

    data = binding.Make(data)

        return self.data[oid]

        self._save(ob)

        return ob.forname

    def _save(self,ob):

        self.data[ob.forname] = {'forname':ob.forname, 'text':ob.text}

}}}

That was easy. The `_new()` method is responsible for both saving state and returning the object ID of the new object. This is because `_new` is responsible for assigning object IDs. In this case, we simply return `ob.forname`, since that's what we're using as an object ID, after calling `self._save(ob)`. Let's run the script, and try it out: {{{

% ./hello for Fred: Hi, guy!

% ./hello to Fred

Greetings, good sir.

 

}}}

Oops. All we did was update our in-memory `data` dictionary. We didn't save it to disk, so the change didn't stay in place for long. How can we fix that?

If we look at the `storage.IWritableDM` interface (see `peak help storage.IWritableDM`), we'll see that it includes a `flush()` method. `flush()` is called as part of the transaction commit process, and the default implementation of this method in `EntityDM` is what calls our `_save()` and `_new()` methods for the appropriate objects. If we define our own version of `flush()` that first calls the standard `flush()` and then writes our `data` array to disk, we'll be all set: {{{

#!python

    def flush(self,ob=None):

        super(MessageDM,self).flush(ob)

        file = open(self.filename,'w')

        for forname, data in self.data.items():

            print >>file, "%s|%s" % (forname,data['text'])

        file.close()

}}}

 

But wait. What if there's an error while writing the file? What is going to happen to the original file? Since we're opening the existing file for output, we'll have already erased our original data. That's not good.

 

We need a mechanism for writing files that can roll back or commit, just like the transaction as a whole. PEAK has a `peak.storage.files` module with two classes we can use for this: `TxnFile` and `EditableFile`. Because we're dealing with such a small file, and can load it all in memory at once, we'll use `EditableFile`, which offers a more convenient interface for such files. Let's take a look at the part of the output from `peak help peak.storage.files` that covers `EditableFile`: {{{

    class EditableFile(TxnFile)

     | File whose text can be manipulated, transactionally

     |

     | Example::

     |

     | myfile = EditableFile(self, filename="something")

     | print myfile.text # prints current contents of file

     |

     | # Edit the file

     | storage.beginTransaction(self)

     | myfile.text = myfile.text.replace('foo','bar')

     | storage.commitTransaction(self)

     |

     | Values assigned to 'text' will be converted to strings. Setting 'text'

     | to an empty string truncates the file; deleting 'text' (i.e.

     | 'del myfile.text') deletes the file. 'text' will be 'None' whenever the

     | file is nonexistent, but do not set it to 'None' unless you want to replace

     | the file's contents with the string '"None"'!

     | By default, files are read and written in "text" mode; be sure to supply

     | a 'fileType="b"' keyword argument if you are editing a binary file. Note

     | that under Python 2.3 you can also specify 'fileType="U"' to use "universal

     | newline" mode.

     |

     | 'EditableFile' subclasses 'TxnFile', but does not use 'autocommit' mode,

     | because it wants to support "safe" alterations to existing files.

}}}

 

Yep, that looks like what we need. We should be able to easily load and save

our data by reading or writing to the `EditableFile` object's `text` attribute,

especially since we will already be inside a transaction whenever we use

the data manager.

Okay, so let's fix up `storage.py` to use `EditableFile`: {{{

#!python

from peak.api import *

from peak.storage.files import EditableFile

from helloworld.model import Message

class MessageDM(storage.EntityDM):

    defaultClass = Message

    filename = binding.Obtain(PropertyName('helloworld.messagefile'))

    file = binding.Make(

        lambda self: EditableFile(filename=self.filename)

    )

    def data(self):

        data = {}

        for line in self.file.text.strip().split('\n'):

            fields = [field.strip() for field in line.split('|',1)]

            forname, text = fields

            data[forname] = {'forname': forname, 'text': text}

        return data

    data = binding.Make(data)

 

    def _load(self, oid, ob):

        return self.data[oid]

    def _new(self, ob):

        self._save(ob)

        return ob.forname

 

    def _save(ob):

        self.data[ob.forname] = {'forname':ob.forname, 'text':ob.text}

 

    def flush(self,ob=None):

        super(MessageDM,self).flush(ob)

        self.file.text = ''.join(

            ["%s|%s\n" % (forname,data['text'])

                for forname, data in self.data.items()

            ]

        )

We hardly changed a thing. Instead of opening `self.filename` to read and write the data, now we simply split or join `self.file.text`. The `EditableFile` will automatically handle writing the new data to a different filename, then renaming it and replacing the old file. It'll also automatically discard the new file if the transaction is aborted for any reason.

Speaking of aborting, there's actually still a bug in this DM. If a transaction is aborted, the DM may or may not have called `_new()`, `_save()` and/or `flush()`, one or more times already. The `EditableFile` will take care of resetting itself if a transaction is aborted, but our `data` dictionary could wind up out-of-sync with the file.

An easy way to do this would be to override the `abortTransaction()` method, similar to what we did for `flush()`, and delete the `data` dictionary if the transaction is aborted: {{{

    def abortTransaction(self, ob):

        self._delBinding("data")

        super(MessageDM,self).abortTransaction(ob)

Now, if the transaction is aborted, the `data` attribute gets deleted, and the next time we try to use it, our `binding.Make()` wrapper will re-run the function that creates the dictionary from the `EditableFile`. `EditableFile` does something similar to this, so when we access its `text` again, it will have reverted to whatever was last stored on disk, not what we changed it to.

There are a few other things to notice about our revised DM.

We're still getting the filename from that same configuration

variable. Now, however, we are turning that into an `EditableFile`.

Again we use binding.Make to create a descriptor that will return

a real value (and cache it) when the class attribute is actually

accessed. We used a lambda expression here instead of a function,

as this is more readable when there's only a single expression

being executed.

 

Anyway, with these changes in place, our `for` method should

now be working: {{{

 

% ./hello for Jeff: Hi, guy!

% ./hello to Jeff

Hi, guy!

== Questioning Existence, and Tuning Performance ==

 

At this point certain readers may be getting antsy because there's a

flaw in the `forCmd` implementation. As we wrote it, the `for` command

assumes that it's always creating a ''new'' `Message`, even

though the `forname` may already exist in our primitive "database".

 

For our current example, this doesn't actually cause any problems:

because of the way we're updating the "database", it doesn't matter

if the item is new or an update. But, we don't want to rely on

this implementation quirk, and when we move to an SQL database

later on, it will matter quite a bit whether we're adding or

updating.

 

To fix this, we need to change our `for` command to check whether

the name exists, and then either update the existing `Message`

object, or create a new one, as appropriate. In order to

do that, we need to be able to ask the DM whether or not a given

key exists. Since we're using the `forname` as the object id, we

can handily provide a way to do it by adding a `__contains__`

method to the DM:

{{{

    def __contains__(self,oid):

        return oid in self.data

}}}

Now we can update our `forCmd._run()` method in `commands.py`: {{{

 

        if len(self.argv)<2:

            raise commands.InvocationError("Missing arguments")

 

        parts = ' '.join(self.argv[1:]).split(':',1)

        if len(parts)!=2:

            raise commands.InvocationError("Bad argument format")

 

        forname, message = [part.strip() for part in parts]

 

 

 

With this change, updating the database should still work: {{{

Sharp-eyed readers will notice that the `__contains__` method we wrote does

''double'' the normal work for retrieving an item, because it actually "loads"

data, by accessing the "database". Then, if the item exists in the database,

the `_load()` method will access the database again. For our in-memory

database, this is no big deal, but it will be more important when start

using SQL. Let's change our approach. We'll replace the `__contains__`

method with a `get` method: {{{

#!python

    def get(self,oid,default=None):

 

        if oid in self.data:

            return self.preloadState(oid, self.data[oid])

 

        return default

}}}

 

This method will either retrieve the object, or return the `default`,

which is the standard python signature for `get`. To support only

retrieving the object once (as well as various other situations),

DM's have a `preloadState(oid,state)` method. This method creates

a pre-loaded object, using `state`, ''instead of'' calling `_load`

to get the state. (Remember, what we have stored in `data` is a

dictionary containing the values for the various object attributes,

which is the state from the DM's point of view).

 

So, our `get()` method can load the state from our "database", and

then preload it into the object it returns.

 

There is actually still a minor inefficiency here: we're always

checking the "database", even if the object we want is already loaded

into memory. We can make this slightly more efficient by changing it

to: {{{

#!python

    def get(self,oid,default=None):

 

        if oid in self.cache:

            return self.cache[oid]

 

        elif oid in self.data:

            return self.preloadState(oid, self.data[oid])

 

        return default

}}}

DM's have a `cache` attribute that holds onto currently loaded objects,

so that multiple requests for a given object ID will always return the

same object. So, by checking it here first, we can avoid doing the

lookup in `self.data` if the requested object is already loaded.

 

These minor changes are of little or no consequence to our current

app, but will have more impact when we move to using SQL, as every

`self.data` lookup is going to end up as an SQL query.

 

Let's finish out our refactoring by updating `forCmd` to use our new

`get()` method: {{{

#!python

    def _run(self):

 

        if len(self.argv)<2:

            raise commands.InvocationError("Missing arguments")

 

        parts = ' '.join(self.argv[1:]).split(':',1)

        if len(parts)!=2:

            raise commands.InvocationError("Bad argument format")

 

        forname, message = [part.strip() for part in parts]

 

        storage.beginTransaction(self)

 

        msg = self.Messages.get(forname)

 

        if msg is None:

            msg = self.Messages.newItem()

            msg.forname = forname

 

        msg.text = message

 

        storage.commitTransaction(self)

}}}

 

There. That even simplifies the logic a little. Note, by the way,

that we do ''not'' pass `Messages.newItem()` as the `default`

argument to `get()`, because that would do two wrong things: 1)

it'd create a new object that would be added to the database at

transaction commit, even if we didn't need it, and 2) it wouldn't

set `forname` on the new message. We could work around problem #2,

but not problem #1. Using the `newItem()` method of a DM always

creates an object that the DM will attempt to save when the transaction

commits, even if you don't keep the object around that long. So:

never call `newItem()` unless you want the object to be added to

the database. (Note: it's ''possible'' to write a DM that doesn't

behave this way, and only saves an object if it's referenced from

other objects or some kind of "root" object. We're just not going

to show you how in this tutorial!)

 

Anyway, that about wraps it up for creating a practical `EntityDM` subclass.

 

== Points to Remember ==

 

Here's the recap for what we've learned in this lesson (once again, it's quite a lot!):

 

 * Subcommands

 

  * The `peak` script is based on `commands.Bootstrap`, a `commands.AbstractInterpreter` subclass that runs a "subcommand" specified as its first argument.

 

  * `commands.Bootstrap` looks up non-URL commands in the `peak.running.shortcuts` property namespace

 

  * Raising `commands.InvocationError` in a command's `_run()` method causes the command's `usage` message to be displayed, along with the text of the `InvocationError` instance.

 

 * Configuration

 

  * Application configuration files load after `peak.ini`, and only explicit settings in an application config file override those in `peak.ini`

 

  * Using an asterisk (`*`) as the last part of a property name, defines a rule for all undefined properties in that property namespace. So, defining a rule for `peak.running.shortcuts.*` provides a value for any undefined property whose name begins with `"peak.running.shortcuts."` (Important: note the trailing dot).

 

 * Writable DM Basics

 

  * The `newItem()` method of a writable data manager (`storage.IWritableDM`) returns a new, "empty" object which will be saved when the transaction is committed.

 

  * The `storage.QueryDM` class does not support adding or modifying items, but `storage.EntityDM` does

 

  * The `_new()` method of an `EntityDM` must add the new object to the underlying database, and return the object ID to be used for the object

 

  * The object ID returned by `_new()` can be a newly-generated ID, or it can be a primary key field.

 

  * The `_save()` method of an `EntityDM` must update an existing object in the underlying database

 

 * Extending `EntityDM`

 

  * The `flush()` method of a writable data manager is called to write in-memory state to external databases. It can be overridden so that `_new()` and `_save()` can batch the data to be written, and then `flush()` can perform the batch in bulk.

 

  * If you are already accessing an underlying database (e.g. to check if an object exists, or when performing a mass query), you can use the `preloadState()` method to retrieve an object from the DM. The DM will not call its `_load()` method, but instead use the state that you supply to the `preloadState()` call.

 

  * DM's have a `cache` attribute, that caches "ghosts" and active objects used in the current transaction, to ensure that multiple requests for the same object ID will return the same object.

 

  * If you need to be able to tell if an item exists in a DM, it can be handy to implement a `get()` method, that checks the DM's `cache` attribute before checking the underlying database, and finally using `preloadState()` to retrieve the now-loaded object.

 

 * Miscellaneous

 

  * `peak.storage.files.EditableFile` provides an easy way to transactionally alter the contents of a file small enough to be held in memory. It works with the PEAK transaction framework to ensure that existing data isn't lost in the case of a write failure or other error, and that rolled-back changes are in fact rolled back.