[PEAK] Resources and traversal in peak.web

[Phillip J. Eby]

As I've been thinking about the design of the upcoming peak.web XML 
configuration format (translation: as I've been studying Zope 3's ZCML to 
find good stuff we can steal), it has occurred to me that maybe it's time 
to replace the ad hoc ++resources++ and @@name handling in 
peak.web.  (Translation: I found some good stuff in ZCML/zope.app we can 
steal!)

Zope 3's traversing system (at least as of a few months ago) uses a very 
simple way to let path traversals operate in a sort of "multidimensional" 
fashion.  It basically works like this:

  * Path segments that don't begin with '@' or '++' are normal traverals

  * Path segments that begin with '++some_identifier++' are split into the 
'some_identifier' part and the remaining part.  Then, 'some_identifier' 
(the "namespace") is used to look up a traversing function, that's then 
given the remaining part (the "name") to perform the lookup.

  * Path segments that begin with '@@' are treated as though they began 
with '++view++'; in other words, '@@' is a shortcut for accessing the 
"view" namespace

Zope 3 provides a variety of built-in traversal functions (technically, 
adapters) to do common operations like item and attribute traversal, 
setting skins or virtual host locations, accessing resources, and so on.

One thing that's been bugging me for a while in the design of 
peak.web.templates is that certain kinds of operations really need to be 
able to do this sort of "traversal in another dimension".  For some reason 
I never really made the connection between Zope 3's namespace mechanism and 
this issue; possibly because at the time I started the design, Zope 3's 
namespace mechanism was *much* more complicated and had lots of other weird 
features.  Now that it's streamlined down to "look up a function, call the 
function", it's easier to see the elegance of it.

By contrast, 'peak.web' currently has a small amount of special handling 
for '@@': traversing decorators that wrap containers usually recognize this 
to mean that the URL seeks a method or attribute of the container, rather 
than an item within the container.  We also have special handling for a 
name '++resources++', that can be used to access resources like templates 
and images, but only at the "application root" URL.

But these special cases are rather awkward, to say the least.  For example, 
the need to be able to refer to ++resources++ at the root means that 
peak.web has to make the application's "skin" the root object, which then 
checks for the special name before allowing traversal on the *real* 
application root.  This would be unnecessary if a ++resource++ namespace 
were available anywhere, as a natural byproduct of traversability.  This 
would also make it possible to access arbitrary resources from 
peak.web.templates.  Also, with the more general mechanism, we could use 
URL parts like '++skin++foo', to allow easy skin selection via URL.

As with Zope, this mechanism could be extensible, so that an application 
with special needs could define namespaces of its own.  In practice, 
though, there will probably be only a handful of built-in namespaces that 
will get most of the use.  Zope has:

    * view -- this will be used to access most web pages, and template 
fragments

    * attr -- access an attribute of the object (i.e. 'getattr(ob,name)')

    * item -- access an item of the object (i.e. 'ob[name]')

    * acquire -- search up the object's component hierarchy for the name

    * skin -- set the current skin to the named skin

    * resource -- look up the named resource in the current skin

    * vh -- alter request contents for virtual hosting purposes

    * help -- enter the online help system

Of these, I don't see much use in 'acquire' or 'vh'.  (Actually, I see the 
use of 'vh' but I wonder what its full security implications are.)

It seems that the obvious way in PEAK to configure these traversal 
mechanisms is in a property namespace, perhaps 'peak.web.namespaces'.  New 
namespaces or namespace definition overrides can then occur at any 
component, or in any resource directory (via resources.ini).  The current 
'web.traverseName()' function would also be the natural place to implement 
this, as it's used for all traversals.

As part of this traversal refactoring, I'm thinking we would also:

  * Change the definition of web.IHTTPHandler to match the current WSGI 
spec's input and output parameters

  * Add a view registration mechanism, based on a name and a source type or 
protocol.  The mechanism needs to be contextual based on the traversed URL, 
*not* the target component's physical component hierarchy.  In this way, 
objects retrieved from a single database can still have different views in 
different contexts.  It's tempting to use a generic function for this, but 
the "simplest thing" will probably be to treat each distinct view name as a 
protocol to which the target object is adapted, registering those protocols 
in the property namespaces of components that represent traversal steps, 
and configuring them such that child protocols are implied by parent 
protocols (so that by default the parent's definitions apply).

  * Define the standard traversal mechanism (i.e. when the namespace is 
unspecified) to be "getattr, then getitem, then defined views", allowing us 
to ditch 'peak.web.places.Decorator' and friends in favor of this default 
traversal algorithm.

  * Consolidate the skin and layer services into the interaction policy, 
rather than keeping them as separate services.  They're so easy to 
configure using an .ini file that there seems little point in creating 
alternative implementations, especially since one can just set 
'peak.web.skins.*' or 'peak.web.layers.*' to do a lookup in a custom 
service if you need it.


These changes have the potential to break existing peak.web applications, 
if they define custom decorators or rely on being able to see names 
beginning with '@@' or '++'.  But, they should continue to simplify the 
peak.web code base, and the development process for new peak.web apps, 
especially once the XML site definition format is available.  And, they 
will get us into a position where we can begin building p.w.t widgets for 
menus, layout systems, etc.

View registration will let us define name mappings that adapt the target to 
a protocol *and* apply a peak.web.template to format it at the same 
time.  This will probably show up as a 'views' domlet namespace, as in 
'pwt:domlet="views.menu"' to replace the enclosed XML block with the 
results of adapting the current object to a "views.menu" contextual 
protocol and applying the registered rendering template.  Perhaps the 
enclosed block will be used as a default in the event there is no such 
view, or perhaps a pwt:define setting will be required to define such a 
default.  I'm not sure yet if this will interact with layouts in some way, 
but, for doing things like creating menus, dropdowns, tabbed interfaces, 
form views and such, this should be very handy.

Of course, because arbitrary views are potentially accessible via URLs, we 
probably need some way for templates to be designated as "fragments", so 
that if you try to access '@@menu' on an object you don't get the object's 
menu representation as an HTML fragment!  We might want to use a different 
extension for such resource files, and use a template subclass that either 
refuses to render to HTTP, or places itself within a standard layout 
template.  Hm, that's three kinds of templates, then:

   * Standalone page template: renders the same whether it's accessed 
directly via HTTP, or embedded in another template.

   * "Widget" template: only renders when embedded in another template, 
refuses to render via HTTP, registering a '403 Forbidden' error if you try.

   * "Fragment" template: renders as-is when embedded in another template, 
but if accessed directly via HTTP, it wraps itself in a "standard layout" 
template.

The last of these is kind of a convenient special case for objects that can 
be displayed within another page, or by themselves.

I'm not sure how these should be designated, although we could just use 
extensions like '.page', '.widget', and '.fragment'.  Or maybe they would 
be '.page.pwt', '.widget.pwt', and so on.  (The resource subsystem can 
handle multi-part extensions like this, and longer extensions take 
precedence over shorter ones.)

Anyway, if y'all have any better ideas for the naming convention, let me know.