[TransWarp] Tips on using model, storage, config, and
naming (was Re: First attempt to use the storage package in PEAK)
Phillip J. Eby
pje at telecommunity.com
Sun Dec 29 11:00:09 EST 2002
At 10:31 AM 12/29/02 +0200, Roché Compaan wrote:
>Jeez, you think of everything ;-)
I try. Sometimes I drive Ty nuts with my, "but what about X?" questions,
>But I'm glad you put so much effort
>into a proper validation framework - I found that without it you quickly
>end up with spaghetti code especially when you have to validate changes
>accross fields and accross objects. I wont buy into a framework that
>validates fields atomically.
Yeah, that's pretty useless for meaningful domain models. The work that
I'm doing on the MOF will be an interesting test case in itself for
validation: MOF has a lot of meaningful semantic constraints, like that
something can't subclass itself or contain itself, directly or indirectly.
But the MOF definition itself includes the notion of constraints that are
either "immediate" (checked in real-time) or "deferred" (checked manually
or perhaps at a transaction boundary), and I think it will be best to
implement these capabilities directly in peak.model, since such constraints
are part of the MOF model, and thus part of the UML and CWM metamodels.
> > Ideally, if one could specify constraints in "relatively declarative"
> > it would make validating constraints that much easier. Actually, I
> kind of
> > wonder if maybe I shouldn't be looking into OCL, because then one could
> > that into the UML model for an application from the start.
>OCL would be great, but expressing constraints in OCL shouldn't be a
>requirement. I think one should be able to express constraints in python
>as well, not free from, but in a form that can be mapped back to OCL.
Right. The simple way to do that would be to create object classes
representing OCL constructs, that can be created straightforwardly in a
Python expression. Then, if that works out well enough, it should be
possible to create an OCL translator that creates the same objects. That
overall approach worked pretty decently for the design and implementation
of SkinScript in ZPatterns.
But first, I need to learn OCL. :)
>Thanks for all the great tips on the config and naming packages. I love
>the fact that configuration variables can be aliased, are in one place
>and that components can easily discover them.
Eh? I think I know what you mean by "aliased", although technically
they're not aliased; it's just that you can define one set of properties in
terms of another.
But "in one place" is something I'd disagree with; you can put them in one
place if you want, but every object can define its own value for a
someVar = binding.bindTo("otherVar",
Instances of MyComponent will supply the value of 'someVar' in response to
a request for 'some.property', so any child component of a MyComponent
instance will use that value as well. You will see some instances of this
pattern's usage in PEAK source code. You'll also see some things like this:
someVar = binding.Acquire(PropertyName('other.property'))
What this does, is that if 'someVar' isn't manually set, it will look up
'other.property' in its parent components. But the component and its child
components will use the value of the 'someVar' attribute as their own value
of 'other.property'. This means that if you were to do this:
aComponent = MyComponent(someVar="some value")
Then you are effectively setting the value of 'other.property' to "some
value" for 'aComponent' and its children.
As for whether components can "easily discover them", that's an interesting
comment. There isn't any discovery capability for properties; you can look
one up if you know its name, but there's no way to find out what properties
do or don't exist. So, for example, you can't iterate through all possible
URL scheme names. The property namespace is potentially infinite. You
could do this, for example:
foo.* = # some code here to create a new scheme handler based on the scheme
This would let you use URL schemes of the form "foo.bar:", "foo.baz:",
etc. This kind of capability is incompatible with true "discovery",
although Ty and I have batted around the idea of making it possible to at
least discover that a rule for 'foo.*' exists. This would really just be
*rule* discovery, not *property* discovery.
> I still have a hell of a
>time maintaining instances of Zope apps where mailhost and database
>connections store configuration variables in themselves.
Yes, that was definitely a "misuse case" for PEAK, in the sense of
something we definitely didn't want. Storing that kind of configuration in
a persistent object as part of an application is a twisty entanglement of
problem domain and solution domain.
>At the moment PEAK addresses almost all my concerns as an application
>developer and I can really see that it will make maintaining
>applications orders of magnitude easier.
That was certainly the "big idea" behind PEAK, as distinct from TransWarp
(whose primary mission was application generation). I'm glad that you're
already finding it useful.
>There is one big concern that
>is not necessarily the responsibility of a framework like PEAK and has
>more bearing on the implementation of the problem domain namely, 3rd
>party customisation. I mention it in the hope that you can already see a
>pattern that will work well with PEAK.
The config package gives you all the hooks you need to deal with this sort
of thing, but of course you have to implement your basic application or
framework in a suitable way. This could be as simple as using:
at the end of a module's code. This allows people to create "patch
modules" which can "advise" your module's code, wrapping functions around
your functions, redefining class hierarchies, etc. It also allows people
to "subclass" your module, creating their own instance of it that is
You can also define plenty of hook points using configuration variables, e.g.:
OtherClass = binding.bindToProperty('myapp.Something.CollaboratorClass')
Something.CollaboratorClass = importString('some.package:SomeClass')
And of course this is then overrideable in an end-user's application
> The best solutions I've seen so
>far is CMF Skins and by far the best one is the hyperdb used by Roundup
>(a python based issue tracker to which I contributed some code a while
>back - http://roudnup.sf.net/). You most probably know CMF Skins so I'll
Actually, no, I'm not that familiar with it. I read something a long time
ago but I've never really used it so it has already dropped out of my brain
>When you install Roundup the core is installed in
>lib/python/site-packages. Creating an instance of an issue tracker is a
>two stage process. First you add an instance with a command-line script
>to anywhere on the file system. This installs a default class schema and
>html template in the directory you specified.
One way we expect this sort of thing to work with PEAK is that an
application "instance" would look like this:
...configuration file of arbitrary format goes here...
And the 'basicApp' interpreter would look like this:
#... ".ini" format configuration goes here
app_config.format = # code to load the schema needed to parse the instance
Finally, there would be a 'PEAK_App' script, written in Python, provided as
part of PEAK, which would basically set PEAK_CONFIG to point to argv
(i.e., the 'basicApp' script), and then invoke the
'peak.running.app_config.format' schema on argv (the application
instance config) to create a root component for the application, and
execute it (probably via a 'run()' method or some such).
The idea here is that you'll use a more user-oriented configuration format
(such as Zope 3's ZConfig syntax) with an application-specific schema for
its content. The configuration file itself becomes the executable for the
application instance. Applications themselves are a PEAK .ini-format
configuration file that specifies the "meta configuration", if you will.
You'll see that this concept allows you to "inherit" from an application
definition, as well. For example, you could create this "extendedApp" script:
[Load Settings From]
#... various settings to override those in basicApp
And then you could use "extendedApp" as the interpreter for instances of
this new extended application definition.
In principle, there is no limit to how deep the inclusions or meta-levels
can be. In practice, however, I think that two metalevels of configuration
(instance and application), with "n" levels of inclusion and the use of a
PEAK_CONFIG-specified environment configuration, should be plenty of
>Now I only have to modify the html template for the user, initialise the
>instance and start using my customised instance.
This would of course be possible with the above techniques as well.
>Although the hyperdb is quite flexible wrt to custom schemas and user
>interfaces the downside is that it has relatively few hooks where custom
>methods can be plugged in.
>Whereas CMF Skins allows for generous customisation of the user
>interface and provides you with a perfect place to add custom logic
>through scripts and external methods it only solves part of the problem.
>One still needs something that can accommodate 3rd party modifications
>to class schemas that won't be heavily disrupted by upgrades of the app.
I think that simplest way to do this sort of thing in PEAK would probably
be to have a property that specifies where to import the "model.Model" that
describes the application's problem-domain object model. The
solution-domain components would have 'bindToProperty()' attributes to
retrieve the model. Something like this:
model = binding.bindToProperty('myapp.Model')
defaultClass = binding.bindTo('model/TheClassIWant')
Model = importString('some.package:MyDomainModel')
Of course, it may be simpler to do it using a wildcard rule, and something
defaultClass = binding.bindToProperty('myapp.Model.TheClassIWant')
Model.* = # code to load the class named after the 'myapp.Model.' prefix
>I am still thinking of ways that will work best with PEAK but haven't
>come up with something that satisfies me yet. I think the way you
>illustrated how to include the MySQLConnection without modifying code in
>the storage package might be the answer but I'll have to play with my
>Contact example some more to see how well the patterns works with domain
Perhaps the examples I've laid out above will help.
>I've attached the mysql driver if you want to include it - it was
>fairly simple to write since MySQLdb implements the DB API.
I'm afraid MySQL gives Ty and I the creeps, and we don't feel comfortable
including a driver for it with PEAK for a number of reasons. Because we do
try to "think of everything", we are concerned about its locking and
transactional semantics and how they would interact with PEAK's caching,
locking, and transactional patterns. We couldn't say whether or not it
will work correctly, and so are totally uncomfortable including it in an
"enterprise application kit".
In the environment where I work, *PostgreSQL* is considered a lightweight
database for prototyping purposes, and Sybase or Oracle are what you use
for production applications. One of our apps has a 30GB database on a RAID
array, and its dedicated database server hardware is a 64-bit,
four-processor machine, with an identical warm standby server continuously
replicated from the first. PEAK really is intended to work at this sort of
"enterprise" scale, and using MySQL for production in such an environment
would be quite insane, IMO.
But Gadfly and SQLite don't pretend to do or be more than what they are,
and their limitations are both more obvious and more clearly documented, so
I don't have a problem with including them so that people can do database
prototyping. I worry that too many people use MySQL in production
situations where they *really* shouldn't be, and I don't want to encourage
that trend. Most people, however, wouldn't be tempted to use Gadfly or
SQLite for production unless they really could live with the
limitations. But many are tempted by MySQL's claims that speed is all that
matters and transactions aren't important. That's why it gives people
whose companies spend hundreds of thousands of dollars on RAID,
replication, and hardware redundancy for mission-critical, 24/7 worldwide
apps, a really creepy feeling about having *anything* to do with it. :)
So, while I'm sure that you would never use it for an application
environment it wasn't suited for, I'm not comfortable with making any
implication that PEAK supports or endorses its use in any way, because
PEAK's intended audience *does* include people who would abuse MySQL, and
in the process would give other people an excuse to trash PEAK, Python, and
Open Source in general for their lack of attention to "enterprise-level"
More information about the PEAK