[PEAK] Traversal, views, and templates = PWT, Reloaded
python at adytumsolutions.com
Mon Aug 23 22:06:06 EDT 2004
Have you had a chance to view the source of nevow yet? They made use of
a lot of Zope/TAL ideas in very clever and very clean ways. It's the
best templating I have seen in a while... and I'm a big fan of Zope/TAL
The process they went through in refining TAL and the product the
developed as a result (mostly Donovan Preston) may provide you with
some nice insights and shortcuts...
svn checkout svn://divmod.org/svn/Nevow/trunk Nevow
Quick overview and sample code snippets:
Dedicated site with downloadable sample apps (part of the tutorial):
I'd be very curious to hear your thoughts on what they did as well...
PS Just in case: nevow is a complete rewrite of woven from twisted.web.
On Aug 22, 2004, at 2:51 AM, Phillip J. Eby wrote:
> As I thought further about the plan in my last post, I discovered some
> interesting possibilities for simplifying typical DOMlet usage in
> This is a fairly long post, but important to read if you're currently
> using peak.web.templates (.pwt's), or would like to in the future. It
> describes a significant overhaul for PWT to take advantage of the
> coming "view registration" and "traversal namespace" mechanisms in
> peak.web. It also deals briefly with some likely directions for PWT
> widget development, and how the "view registration" mechanism will be
> Moving to Views in PWT
> Currently, we have 'text', 'text.notag', 'xml', and 'xml.notag'
> DOMlets for displaying text. They work by doing the equivalent of
> 'str(current_object)', so they don't really help for e.g. embedded
> templates. But, now that we have "alternative" traversals available,
> there doesn't seem to be a reason to distinguish between text and XML,
> or between text and some sort of embedded view or the attributes of an
> Let me be more specific. Suppose we have a DOMlet called "show", and
> what "show" does is traverse to the specified data item, adapt it to
> IDOMletNode, and invoke its 'renderFor()' method. The immediate
> consequence of this is that any view or attribute that represents a
> DOMlet or template, is automatically embedded at the point of contact.
> And, if we define default DOMlet adapters for strings and Unicode
> objects, then any string attribute of an object is trivially
> But what about non-strings? What about numbers and dates? Ah, that's
> where the view mechanism comes in. Let's say we register a view named
> 'short_date' for type 'datetime.datetime', that uses the request's
> locale to look up a short date format, and formats the given date as a
> string. Poof! We now have an instant date formatting widget, which
> we could use in a template as
> 'pwt:domlet="show:due_date/@@short_date"'. That is, "apply the 'show'
> DOMlet to the 'short_date' view of the current object's 'due_date'
> Not bad, not bad at all. But what happens if there's no 'due_date'
> attribute, or no 'short_date' view? Or a permission problem? And, do
> we need a 'show.notag' variant?
> Hm. It's almost enough to make you want more 'pwt' attributes besides
> 'domlet' and 'define'. For example, we could have 'pwt:insert' and
> 'pwt:replace', which either insert something within the current
> element, or replace the entire element (ala the '.notag' DOMlets).
> Then, we could allow the default DOMlet to be "show", so normally you
> would only be doing things like 'pwt:replace="due_date/@@short_date"'
> instead of 'pwt:domlet="show.notag:due_date/@@short_date"'.
> The basic idea here is that pwt:replace would work like pwt:domlet
> does now, but pwt:insert would create the DOMlet as a nested element
> within the element where the pwt:insert attribute appears. The
> factory for this nested DOMlet would have to support a new
> 'INestedDOMletFactory' interface, or some such, to indicate that it
> can be used in this fashion. Then, all children of the element with
> the 'pwt:insert' attribute will be placed inside the nested DOMlet,
> and the nested DOMlet will be placed inside the element with the
> I'm not sure if it will make practical sense to allow both 'insert'
> and 'replace' on the same element, although in principle it could make
> sense to do so.
> By adding this third attribute (fourth, if we allow 'domlet' for
> backward compatibility), we can now simplify the common case of
> template interpolation to 'pwt:xxx="some_attr/@@some_view"'. Of
> course, actual widgets like the current 'list' and the contemplated
> 'tree', 'menu', etc. widgets will still need to be specified by
> prefixes, such as 'pwt:replace="list:some_attr"'.
> Hmm... wait a minute. Suppose we just made the 'pwt:' attribute
> namespace into a property lookup for a new kind of DOMlet factory?
> Then, it would make sense to say something like
> 'pwt:list="some_attr"'. And even 'pwt:define' could be implemented by
> this mechanism. As far as I can tell, the XML spec allows for '.' in
> attribute names, so 'pwt:myapp.something' should be a valid attribute.
> Interesting. At this point, PWT is no longer a minimalistic
> two-attribute syntax, but is more convenient to type, while remaining
> far more extensible than other attribute-based syntaxes. We'll have
> to have some sort of prioritization mechanism, though, so that
> multiple attributes on the same element are ordered sensibly. In
> principle, XML attributes *should* remain in the order you specify
> them, but I believe that technically speaking no XML spec guarantees
> that they *will*, so if you run a PWT file through editors or
> processing of various kinds your order might get munged. Having a
> priority mechanism between transformers will keep the mechanism
> PWT vs. ZPT?
> You know what's *really* weird? Once PWT "dissolves" in this fashion,
> there's nothing stopping you from implementing Zope's entire "TAL"
> attribute language, if that's what you really want. Indeed, ZPT
> already has 'tal:content' and 'tal:replace' that are similar to my
> hypothetical 'pwt:insert' and 'pwt:replace'. The difference is that
> ZPT requires you to specify that a replacement or content is
> "structure" or "text". For me, I would just as soon use views to make
> that distinction, treating strings and Unicode as text by default.
> Although I like a lot of the features in ZPT's TAL and TALES
> languages, I'm worried that many of their features stray into
> scripting-land, while other features seem to lack important
> functionality. Specifically, I like PWT's ability to use chunks of
> XML as parameters much better than ZPT's tendency towards string
> manipulation. For example, in ZPT, 'tal:on-error' is an expression to
> use in place of the current content if there is a failure anywhere
> within the current XML element. If I implemented a 'pwt:on-error', it
> would work in almost exactly the opposite way:
> 'pwt:on-error="NotFound,NotAllowed"' would indicate that the marked-up
> element would be used in case of the corresponding errors occurring
> while executing the nearest *surrounding* insert/replace operation.
> So, although the idea of implementing ZPT's TAL/TALES/METAL in PWT is
> somewhat attractive from a documentation point of view (i.e., not
> having to write docs!), I think I'm going to leave it to be a
> "community-contributed" extension if anybody really wants it, and
> focus on making an excellent PWT system. The key is that PWT-style
> attributes should generally act like function invocation, taking a
> data path and a collection of specially-marked child XML elements as
> So, it sounds like the best course of action is to do another draft of
> the DOMlet factory architecture, such that individual XML attributes
> can perform transformations on an XML element, and the handlers for
> those attributes have some kind of priority scheme to determine
> evaluation order. There then need to be various architectural
> affordances to allow implementation of things like error handlers,
> parameters, and so forth.
> I expect that it will become relatively uncommon to create custom
> DOMlets, and when you do create them, you'll have to put them in some
> XML namespace other than the 'pwt' one. Still, once we have basic UI
> widgets, they can be made to do lots of different things. I believe
> I've mentioned previously how a simple "selected items" widget can
> implement such diverse things as menus, dropdowns, tabbed interfaces,
> and such, all by using different HTML snippets as parameters to the
> widget. So, there will probably be half a dozen or so commonly used
> widgets for lists, trees, data tables, form layouts, tabbed pages, and
> so forth. Once we have these, there'll be little call for DOMlets as
> such. More likely, you'll create much simpler "view" components to do
> text formatting, or create .pwt templates to define layout macros or
> XML views.
> Another Idea: Insert vs. Replace
> A common theme throughout this has been the need for choice between
> applying something to an element, and applying to its contents. I
> wonder if we could use XML namespaces for this? Perhaps 'this' and
> 'content' used as prefixes would make for nice spelling. An example:
> <ul content:list="some_attr">
> <li this:is="listItem" content:replace="item_name">Text will go
> In other words, "the content of this UL element will be replaced with
> a 'list' rendering of the data found at 'some_attr', and the list
> will receive a 'listItem' parameter that is the '<li>' element, which
> will have its contents replaced by the 'item_name' attribute of each
> item in the list." When this is rendered, it'll look something like:
> The idea here is that using a DOMlet from the 'content' namespace
> applies it to the collection of child elements of the element where it
> appears, while using one from the 'this' namespace applies it to the
> element itself. This would be orthogonal to the definition of DOMlets
> themselves, which would still be pulled from a single property
> namespace. In other words, the same attributes would be available
> under both 'content' and 'this', and have the same meaning either way,
> except for what object they apply to. (Except for some DOMlets that
> might not be able to function on a collection of child nodes.)
> Let's try an example that generates text differently...
> <div this:list="some_attr">
> <span content:is="listItem">This is the item: <span
> this:replace="item_name" /><br /></span>
> When this is rendered, it'll look something like:
> This is the item: foo<br />
> This is the item: bar<br />
> etc. The 'div' goes away because the 'list' is replacing it, rather
> than replacing its contents. The outer span doesn't appear, because
> its *content* is the "listItem" parameter. And of course the inner
> span is replaced entirely by each list item's name. As you can see,
> we used the exact *same* three DOMlets ('is', 'list', and 'replace'),
> to produce an entirely different effect.
> It's not a *perfect* syntax, though. For example, 'this:list' doesn't
> really convey very well that the element will be replaced.
> 'replace:list' would be more meaningful there, but using 'replace:' as
> a namespace for parameter definition doesn't make much sense. The
> other hangup is that you'll have to define two XML namespaces in each
> template document, unless we have some way to do it by default.
> How well does it work for error handling? Using
> 'content:on-error="NotFound"' would mean that the content would be
> used as an error handler for 'NotFound' errors occurring in an
> enclosing block, possibly the same element. And
> 'this:on-error="NotFound"' would mean that the entire item is used as
> an error handler for some enclosing block, but *not* the same element.
> So this:
> <span this:replace="some_attr" content:on-error="NotFound">No such
> means that if accessing 'some_attr' causes a 'NotFound' error, the "No
> such attribute" message will appear instead of the atttribute value.
> Or, you could do this:
> <span this:replace="some_attr">
> <em this:on-error="NotFound">No such attribute!</em>
> Which registers the '<em>' block as the 'NotFound' error handler for
> the enclosing 'this:replace'. In practice, you may want to use an
> application-specific error handler component, so you might say
> <span this:replace="some_attr">
> <span this:on-error="NotFound,NotAllowed"
> Some text that's maybe used by the handler... maybe even a
> <span content:is="someParam">parameter</span> or two for the
> In other words, "replace the outermost 'span' element with the value
> of 'some_attr', but if 'NotFound' or 'NotAllowed' are raised, invoke
> the 'my.app.handler' DOMlet that was constructed using the next inner
> span and its contents."
> Whew. That last example was a bit squirrelly, but mainly because
> there's a lot going on. Individually, though, each attribute seems to
> make a fair amount of sense.
> This also gives us a higher-level organization to use for prioritizing
> attributes: we can divide attributes into "registration" attributes
> that just package up the element or its contents and register it in
> some way with a containing element, and "replacement" attributes that
> replace the element or its contents. An element can only have one
> "replacement" attribute, but any number of "registration" attributes.
> The registration attributes get invoked after the replacement
> attribute, in no particular order (since the registrations are
> independent of one another). It should be an error to specify more
> than one replacement on an element, even if one is for the content and
> the other is for the element itself. (I can't think of any use cases
> for that; can you?)
> Attributes, Layout, and Parameters
> We've come a long way, so I'll go back to recap the concept of the new
> template system:
> * There will be two XML namespaces, colloquially referred to as
> 'this' and 'content'
> * There will be a single property namespace used to look up the
> implementations of attributes in these two XML namespaces.
> * Attribute implementations will be either "registration" or
> "replacement" attributes. At most one "replacement" attribute is
> permitted per XML element
> * The "replacement" attribute, if any, will be used to create the
> DOMlet for that element. If the 'content' namespace is used, then the
> created DOMlet will actually be nested just inside the element where
> the attribute appears, and all of the child nodes that would have been
> added to the element are instead added to the "replacement" DOMlet.
> * The "registration" attributes, if any, will be supplied with the
> element where they appear, unless they are in the 'content' namespace,
> in which case they will be supplied an element corresponding to the
> entire contents of the element where they appear. (If there was a
> "replacement" attribute, this "contents" element will be the
> "replacement" DOMlet, otherwise an extra "tagless element" node will
> be created.)
> * An 'is' "registration" attribute will replace the current
> 'pwt:define' attribute, and a 'replace' attribute will replace most
> uses of the existing 'pwt:domlet' attribute. Other new "replacement"
> attributes will be used instead of the values previously specified in
> 'pwt:domlet' attributes. E.g. 'content:list="foo"' will do the same
> thing that 'pwt:domlet="list:foo"' used to do, and
> 'this:replace="bar"' will take the place of
> * New "registration" attributes ('this:on-error' and
> 'content:on-error') will be created for handling errors in the
> execution of the nearest enclosing "replacement" attribute. They will
> basically register their target node as an error handler for the
> appropriate types, with the nearest enclosing DOMlet that accepts
> error handler registrations. We'll probably also have a "replacement"
> attribute called 'optional', to refer to a replacement that should
> simply be omitted if it isn't found or isn't allowed for the current
> user. I expect that handling such missing/not-allowed items would be
> such a common use case for 'on-error' handlers that having a shortcut
> available is a good idea.
> The above items don't cover layouts/macros, which we'll also need a
> way to handle. A simple solution is to have the 'replace' attribute
> supply the invoked DOMlet with parameters for all of the 'is'-tagged
> items within the 'replace' attribute's scope. It could do this by
> creating an object whose attributes were the parameters. Then, normal
> 'replace="attrname"' attributes can be used in the "macro" or "layout"
> template to insert the supplied arguments.
> It would be nice if we could also pass in arbitrary paths as named
> parameters. One possible way of doing so would be to add a 'with:'
> namespace to define parameters, such that 'with:foo="bar"' means
> "supply the enclosing replacement with the current value of "bar" as a
> parameter named "foo". For example:
> <div this:replace="@@standard_layout" with:title="summary">
> <div this:is="menu" ...>
> <div this:is="body" ...>
> Here, we've said that the entire thing is going to be replaced with
> the 'standard_layout' view of the current object. The
> 'standard_layout' view will be supplied with three parameters: 'title'
> (which will be the 'summary' attribute of the current object), and
> 'menu' and 'body' which will be DOMlets representing the corresponding
> sections of the page as defined. In the standard layout template, we
> may have something like:
> <head><title content:replace="title">Title goes here</title></head>
> <body><h1 content:replace="title">Title goes here</h1>
> <div this:replace="menu" ...> ... </div>
> <div this:replace="body" ...> ... </div>
> and so on. If the template needs access to the object that was
> "current" at the time it was invoked, it can use '..' as a path to
> move above the "parameters" object. We can also make the parameters
> lazy, in that path-expression parameters don't need to be evaluated
> until or unless they're actually used. (DOMlet parameters are
> implicitly lazy, since they have to be invoked for anything to
> happen.) [Open issues: how do we deal with multi-valued parameters?
> Missing ones? What happens when parameters are passed back to a
> DOMlet that was passed in as a parameter? And so on...]
> Hm. This is looking pretty good, though. So far the syntax has been
> very natural, just rolling off my head and onto my fingers. :) I
> find my thought process goes, "Do I want to replace this or its
> contents?" and then I type 'this' or 'content', followed by the thing
> I want to do. Then I go back to look at DOMlet parameters and tag
> them with 'this:is' or 'content:is', and finally fill in the
> 'with:paramname="data"' attributes for any object arguments the
> replacement operation needs.
> What I haven't seen yet is how annoying it may or may not be to
> specify the XML namespaces -- all *three* of them! I imagine that
> I'll probably just cut and paste them from previous templates, but I
> think maybe I should set up the parsing machinery so the namespaces
> default to existing with the correct definitions. If you're not using
> tools that care about the namespaces being defined, it's a waste of
> time to define them. But, if you do use tools that need them to be
> present and correct, you can easily add them in.
> Browsers will be unaffected, of course, because "replacement" and
> "registration" attributes are all removed from the document model
> during parsing. This means that PWT isn't truly round-trippable the
> way ZPT is. Actually, even if we left the attributes in, and added an
> equivalent to ZPT's "define-macro" to mark the insertion of other
> templates, and wrote a program to split such a page back into its
> component templates, we'd *still* be missing any elements that were
> "replaced" via a 'this:' attribute. Anyway, the main use case I'm
> aware of for round-tripping templates is to feed back actual sample
> data into your visual design, and in the worst case that can be done
> with careful copy and paste. The main goal of PWT's design is to make
> it so darn easy to mark up an existing XHTML document and turn it into
> a layout that you don't mind doing it repeatedly if necessary. :)
> [Open issue: we'll probably want i18n "replacement" attributes to
> support translation, too.]
> Widgets and Models
> Here are some ideas for where PWT widgets are headed. Widgets will
> basically be parameterized "replacement" attributes that perform
> rendering of what we might call "data shapes". By data shapes, I mean
> that some data is shaped like a tree, other data is shaped like lists
> or forms or tables. This is rather similar to MS Research's concept
> of "triangles, circles, and rectangles", where trees are triangles,
> objects are circles, and data rows are rectangles, except that those
> "shapes" are a bit vague compared to what I have in mind.
> In the sections above, I've pretty much laid out what's needed for
> basic "circles" and for lists thereof. But we don't really have
> "rectangles" because we don't have, for example, an interface that
> defines a collection of rows with metadata about columns. And we
> don't have "triangles" because we don't have an interface for tree
> browsing. We also don't have "form" or "menu" interfaces.
> It seems to me that a lot of applications can get by with just a few
> basic shapes, and relatively few widgets per shape. For example, I've
> mentioned before that a "menu" widget, using a "shape" that consists
> of a list of items and some way to tell which items (if any) are
> selected, can be used to implement list boxes, dropdowns, navigation
> menus, tabbed interfaces, and so on. "Form", "Grid" and "Tree"
> widgets can do quite a bit as well. The really nice thing about
> widgets implemented as DOMlets is that they can be parameterized with
> arbitrary dynamic content, so the visual rendering of your data is
> completely under your control. Anyway, here are my thoughts so far on
> what "shapes" will make sense, with the abstractions that they will
> * Tree -- "children", "expanded/unexpanded", "selected/unselected",
> "depth", ...?
> * Tabular Data -- "columns", "sorting", "items per page",
> "summaries", ...?
> * Menus -- "items", "selected items"
> * Forms -- "field names", "fields", "help", "errors",
> * "Decks" (alternate content, displayed conditionally, e.g.
> "wizards" and certain kinds of "tabbed" interfaces) -- "items",
> "selected item"
> As you can see, this is all still fairly vague at the moment. It
> would be nice if these "model" interfaces could be defined in a way
> that allowed them to be extended to GUI interfaces in the future, by
> adding notification (from the models to their GUI views), although the
> notification part is out of scope for peak.web itself. But widgets
> *will* need a way to receive "user events", in the sense that they
> will have data they need from the HTTP request (e.g. query string and
> form variables) in order to do their magic.
> If anybody has any experience with GUI "models" that's relevant here,
> suggestions on how best to structure these interfaces might be
> View Registration
> The view registration mechanism is going to be based on contextual
> protocols. First, we'll add a new API, 'config.registeredProtocol(ob,
> key, baseProtocol=None)'. The idea is that if you want to register a
> view named "foo" for instances of TargetType, you'll do something
> p = config.registeredProtocol(context, 'peak.web.views.foo')
> protocols.declareAdapter(lambda ob: MyViewClass, [p],
> Well, that's not exactly true. You probably won't do this in code,
> because it'll be a pain. Notice that you have to have a context
> object (the web application, basically) before you can register; you
> can't just declare that MyViewClass is an adapter, because the
> protocol to adapt *to* is defined at runtime.
> So, instead you'll use one of two configuration formats. Either .ini:
> [Web views for some_module.TargetType]
> foo = some_module.MyViewClass
> Or XML:
> <view name="foo" for="some_module.TargetType"
> I've skipped a bit far ahead here. The result of adapting to our
> named protocol needs to be an 'IViewFactory': an interface with one
> method: '__call__(environ, ob, name)'. The result of calling that
> method is actually used by the traversal machinery as the target of a
> '++view++foo' or '@@foo' traversal. (Or of a 'foo' traversal, if the
> object has no 'foo' attribute or item.)
> So, your view definition can be either a class or a function. If it's
> a class, the view will be an instance, otherwise it'll be the return
> value of the function. Here's a view that formats a floating point
> number to two places:
> def floating_format(environ, ob, name):
> return "%.2" % ob
> It should be able to be defined using e.g.:
> [Web views for __builtin__.float]
> two_places = some_module.floating_format
> Of course, you'll probably want to use dotted names for views in order
> to avoid name collisions between different components that are
> assembled into one application. Now let's look at a fancier way of
> specifying views:
> def resource_view(permission, resource_path):
> def get_view(environ, ob, name):
> allowed = web.allows(environ, ob,
> if not allowed:
> raise web.NotAllowed(
> getattr(allowed,'message',"Permission denied")
> return web.getResource(environ, resource_path)
> return get_view
> and its use:
> [Web views for my_app.SomeComponent]
> index.html = resource_view(security.Anybody,
> This simple arrangement lets you map page (view) names to template (or
> other) resources. Notice, by the way, that view factories are
> responsible for doing security checks! We can get away without them
> for views that e.g. format a date or a number, but other types of
> views will typically require permission checks.
> Anyway, that's the basic idea. Neither the .ini nor XML formats for
> configuration are finalized yet; they're only tentative at the moment.
> I'd like the XML format to follow ZCML where possible, but in some
> cases that's just not going to happen, because the ZCML names imply
> implementation details that differ. For example, ZCML expects a view
> to have a "class", but our views can be defined by functions. I'm
> going to need to review this more before I finalize the XML format.
> Amusingly enough, I was doing exactly that review when I stumbled upon
> the traversal namespaces stuff that started me on writing my earlier
> post. But, I won't be going back to it just yet, as it's after 2 AM
> and I need some sleep first. :)
> Finally, as a reward for reading this far, I give you one last idea:
> what if you could build a static site publishing system from a
> peak.web application? That is, you run a script to build a static
> version of the site. The tricky bit would be knowing what pages need
> rendering, though; you don't really want to parse HTML and follow
> links. So... you define a view name to be used, maybe
> 'static_site_children'. And then you define its implementation for
> each of the different classes and locations in your application, so
> that you need only request that view on each location in the site.
> Intriguing, isn't it? Somebody could create a dynamic blogging
> peak.web application, but then you could just add a static view
> definition to it and then run a "staticize" tool over the application
> object whenever you needed to republish.
> Okay, that's more than enough for now. I look forward to your
> comments, questions, and feedback.
> PEAK mailing list
> PEAK at eby-sarna.com
Duncan M. McGreggor mailto:duncan at adytum.us
Systems & p 301.698.5032
Applications Engineer m 301.801.0349
AdytumSolutions, Inc. http://adytum.us
More information about the PEAK