[TransWarp] A layout framework, and thoughts on DOMlet data specifiers

[Phillip J. Eby]

In order to enable applications to re-use page layout, peak.web needs 
DOMlets that support this, similar to Woven or ZPT macro facilities, or to 
Struts' "Tiles" framework, or OFBiz's "Regions" framework.

The basic idea is simple: have a way to specify a DOMlet that means "use 
such-and-such layout to format my contents".  The "define" attributes in 
the body of the DOMlet will provide values to fill the "regions" in the 
"layout" that is chosen.

On the layout side of the equation, we assume that any peak.web template 
may be used as a layout; it will simply contain "region" DOMlets that will 
render the content supplied by the caller, or else display their existing 
content as a default.

So far, so good, and it even seems reasonable to assume that the logical 
names for the DOMlets involved should be "layout" (used by the caller to 
"lay out" the defined content), and "region" (to identify a place in the 
layout where the caller's defined content to be inserted).

But the devil is in the details.  Do we perform the substitution at runtime 
or compile-time?  If at runtime, do we also select the layout at runtime, 
or do we select it at compile time?  In either case, how is a layout selected?

When I had conceived of resources as being properties of a skin, 
incorporating layout templates into the system seemed almost 
trivial.  Since DOMlet factories are properties, all that seemed needed was 
to specify a DOMlet name that mapped to another template.  This doesn't 
work any more, at least not directly.  Currently, you'd have to take the 
extra step of mapping a property to a resource lookup, in order to do this 
now.  Or, presumably, we could have our hypothetical 'layout' DOMlet accept 
a resource path as its parameter, and do the lookup.

There are some downsides to this approach.  First, resource paths have to 
start with '/++resources++/some.package', which means that layout specs are 
verbose.  Second, the '++resources++' part is in principle changeable, so 
it's potentially not "portable" between apps if a template uses it.  This 
goes directly against one of the goals for the layout system: that it 
should be easy to plug an "off-the-shelf" component into your application, 
preferably without having to rewrite all its templates to fit into your GUI 
design.

This same ++resources++ problem also exists when you want to refer to say, 
an image resource or other static file, so it's not exclusive to the layout 
issue.  I wonder if perhaps we should just permit '//' to be used as a 
shortcut for '/++resources++/'  (or whatever the local app's equivalent 
is).  This would fix the portability problem and at least part of the 
verbosity problem.  We still need '++resources++' or its equivalent in 
order to allow access via the web, but for internal use the shortcut isn't 
bad.  Interestingly, it echoes the '//' of URLs' "naming authority", and 
it's followed by a qualified name.  It might be that DOMlet specs would now 
look "too much" like URLs, but we'll just have to try it, I think.

Okay, so let's look at the next part of the URL: the package 
name.  Certainly, it's important for dealing with most static resources, so 
as to disambiguate similarly named resources (e.g 'editForm') existing in 
multiple packages.  But, for layouts, we seem to want to obtain these from 
the application.  If package A and B each request 'standard_page_layout' 
from their own package names, how can the creator of application C get them 
both to use C's template for the layout?

This seems to imply that there should be a standard for what packages 
layouts are considered to come from, or else there needs to be a way to 
access them that isn't dependent on package namespaces.

We could of course, forgo the use of standard paths for layout selection, 
use a property name, and add some way to conveniently map from a property 
name to a resource.  What concerns me about this approach, however, is that 
if every DOMlet ends up having its own rules for interpreting path 
arguments, the overall system becomes a little too context-dependent for 
easy understanding when reading template text.

Perhaps we need to support using PEAK URLs in DOMlet specs, although that 
almost seems even more chaotic, increasing the dynamic expressiveness of 
the language too much.  I mean, it definitely seems crazy to allow one to 
specify a database connection as the target of a DOMlet!

Of the current PEAK URL types, about the only one that seems to make any 
sense is the "config:" scheme, which would allow use of properties.  This 
would end up looking like:

<sometag pwt:domlet="layout:config:foo.bar.baz/">

Which is beginning to look an awful lot like code to me.  I don't see as 
much trouble with:

<sometag pwt:domlet="layout://foo.bar/baz">

Hm.  It almost seems to me as though we could use Zope 3's idea of path 
parameters, e.g.:

<sometag pwt:domlet="layout:foo.bar.baz;ns=property">

But that's pretty ugly too.  OTOH, some of the ideas for specifying such 
properties wouldn't work because they were intended for externally usable 
URLs, so

<sometag pwt:domlet="layout:(property)foo.bar.baz">

Might be workable too.

I should back up here for a moment, to expand on my goal a bit.  I think we 
want to have some set of standard namespaces that are used in templates, 
and accessible through an easy-to-use, easy-to-read syntax.  If at all 
possible, DOMlets should use only this mechanism to identify the subject of 
their operations, and there should be a small number (ideally a fixed 
number as well) of kinds of data specifications.

However, I'm not positive this is achievable.  For example, our proposed 
"region" DOMlets will look up a named definition, but unless we have a 
"parameter" namespace defined just for this one kind of DOMlet, this won't 
be consistent with our overall pattern.  Indeed, every time a new DOMlet 
comes up with special requirements, we'll be extending that set of 
namespaces, *and* we'll also be increasing needless verbosity, since it's 
likely that most of these DOMlets with "special" namespaces will only ever 
use the special namespace.  (E.g., region DOMlets would always use the 
parameter namespace, making it redundant to say so.)  So, it seems to make 
little sense to try to have a uniform addressing mechanism in the data 
specifier.

That having been said, it may still be useful to try to increase the 
expressiveness of DOMlets' default interpretation of data specifiers, as we 
will do by allowing // in place of /++resources++.  Perhaps we should make 
it possible to specify property names, somehow, or perhaps allow a property 
name to specify a path that should be used.  Perhaps this would look like:

<html pwt:domlet="layout:(layouts.standard)">

Meaning "use the path specified by the template's 'layouts.standard' 
property".  Since an application could define this on a global or per-skin 
basis, we now have a simple method to achieve indirection for anything that 
needs this kind of configuration.

I'm not sure what I think of this yet, as I don't know of any obvious uses 
for it besides layouts.  I definitely lean towards making it a compile-time 
substitution replacing the property name with the path found in that 
property name.  Anyway, it seems mainly useful as a way to allow templates 
to refer to "application-level" resources without having to know who the 
"application" is.  In principle, it could also be used for relative paths, 
but that doesn't seem very useful for anything.

For any given DOMlet, there are several interesting "contexts" from which 
things might be looked up.  At base, there are its 'data' and 'state' 
objects, and there is also the DOMlet itself, and its surrounding component 
hierarchy.  Currently, data specifiers just traverse the 'data', and there 
is no way to look up properties, utilities, or do other acquisition-like 
things, on the data, the state, or the DOMlet itself.

No matter which way I turn on this subject, I bump into something I don't 
like.  If I make it possible to specify different kinds of lookups, 
templates will become more code-like.  If I don't make it possible, then 
DOMlets that can reasonably support  more than one kind of lookup will 
spawn variants to handle them.  *sigh*.

Maybe I'm going about this entirely the wrong way.  Maybe the solution 
isn't to allow different interpretations in general, but rather to support 
DOMlets that do.  To put it another way...

<div pwt:domlet="property:layouts.standard">
     <div pwt:domlet="layout">
         <div pwt:define="region1" domlet=":..">
             ...
         </div>
     </div>
</div>

The idea here is that by using a DOMlet to "escape" into another namespace, 
you could still use '..' to "back up" to the previous traversal 
location.  Alas, as you can see, it requires *lots* of extraneous markup to 
get there and back again.  :(  OTOH, it's intended to be a way to 
supplement normal traversal in unusual situations.  On the *other* other 
hand, it's really not useful for normal situations.

I think we'll have to feel our way on these points for a while, as the best 
compromise isn't clear yet.  At this point, I'm leaning towards saying that 
the interpretation of a data specifier is strictly up to the DOMlet, though 
it's recommended that they be TraversalPaths if at all practical.

So, back to the layout framework for a minute.  We haven't said what the 
"data" context is when "define"s run.  Do they receive the "data" that the 
layout was accessing, or the data that the calling template was processing 
when the layout was invoked?  I.e., if I have:

<html pwt:domlet=":spam">
<div pwt:domlet="layout://foo.bar/standardLayout">
     <div pwt:define="region1" pwt:domlet="text:baz">
        ...

Then, do I know that in "region1" I am referring to 'spam/baz'?  Or could I 
be referring to 'spam/diddly/baz' if there is a reference to 'diddly' in 
the 'standardLayout' template, that encloses the "region" DOMlet for 
region1?  E.g., if the layout template looks like this:

<html pwt:domlet=":diddly">
     <div pwt:domlet="region:region1">
         ...

should we assume that the defined "region1" is then invoked with 
'spam/diddly' instead of 'spam'?  If we do this, it *might* avoid the need 
for the regions to repetitively traverse to subobjects that are naturally 
required by those regions.  On the other hand, it seems to "leak" 
information from the template to the caller, in a way that makes the caller 
dependent on easily broken details of the template's 
implementation.  That's probably worse than some possibly-redundant lookups.

Another benefit of keeping the current data context when inserting regions 
is that you can read a template and understand what data it uses, without 
having to refer to the layout template.  For this reason, it seems that 
execution state (the 'state' parameter of 'renderFor()') should similarly 
revert when using a region.  Thus, once again, the behavior of DOMlets 
appearing in a region definition will in no way be influenced by anything 
appearing in a layout template.  While this arrangement is less flexible, 
it is easier to reason about, and there is no way for a change in the 
layout template to negatively impact a working template that simply uses 
the layout.

Alright.  I think that we've now completely specified how the "layout" 
framework should operate, with the exception that we need a convenient way 
to provide some kind of indirection for selecting layouts, e.g. via a 
'(property.name)' syntax, or some other mechanism.

Thoughts, anyone?