[TransWarp] peak.web Design Notes
Phillip J. Eby
pje at telecommunity.com
Sun Jun 15 09:43:50 EDT 2003
(Comments and questions welcomed.)
* Get 80% of the flexibility and power of Zope 3, for 20% or less of the
complexity, by keeping to a bare minimum of required components and
interfaces. That's *not* the same thing as having a bunch of interfaces
but providing default implementations! Eliminate the need for registries
by using adaptation wherever possible.
* Leverage code. peak.web is for Python developers, not end users. It's
okay to have XML or ZConfig mechanisms that can replace coding later, but
we can and should build with the assumption that application logic and
navigation is made of code, not data.
* Separate presentation from navigation/security/logic. Presentation
components should be in a separate namespace from application components,
and presentation components should not contain application code. Ideally,
HTML templates shouldn't even contain presentation code, only markup to
indicate what code or data is to be applied where.
* Leverage "application-ness". The developer is building a web
*application*, so components should allow refinement of interfaces to ones
that are application-specific. For example, the developer should be able
to use a custom "user" object with application-specific methods, or add
their own collaborator objects like a "cart" or a "clipboard".
* Leverage Apache. If Apache can serve it directly, let it. We don't need
to innovate in the static-file space. Apache can handle ETags, last
modified, persistent connections and a whole bunch of other stuff that can
speed things up for the end user, that we don't want to have to copy.
* Leverage zope.publisher. A lot of knowledge is embodied in the existing
Request and Response classes; we don't want to have to duplicate this either.
(New terms marked with *asterisks*.) An application's *publisher*
interface creates a *publication* component for each web hit. A
publication has bindings for the *request*, *response*, *user*, *skin*, and
any other application-specific bindings (e.g. cart, clipboard, etc.), and
the root *navigator*. A navigator is responsible for translating a URL
path component (i.e. a name) to another navigator or to a
*renderer*. (This includes validating the user's permission to access that
name.) A renderer is responsible for writing output to the response, and
may use *resources* accessed via the publication's skin. Navigators and
renderers are accessed by adaptation, using contextual interfaces supplied
by the publisher (thus allowing publisher-specific adaptations to be
The publisher is accessed by adapting a specified application object to an
appropriate publisher interface, e.g. IHTTPPublisher. The publisher must
provide contextual interfaces (see "Big Example 2" in the PyProtocols
reference manual) for at least IHTTPNavigator and IHTTPRenderer. This is
so that the application-specific navigators and renderings can be
registered as adaptations from application classes, while still having
fallback to more generally registered navigators and renderers.
The publishing interface should provide a way to look up a "skin" by name,
a way to create a Publication from a zope.publisher Request object, and a
way to access the original application root.
The Publication interface is an extension of
zope.publisher.IHTTPPublication. Zope's "publication" objects can be
reused from one web hit to another, but ours are one-per-hit. Anything
that can live for more than one web hit should be in the publisher. Of
course, an application's publication class can define bindings that
retrieve things from the publisher.
The Publication object is central to the architecture, in that it provides
all the "glue" to make things work. It contains the request, response,
user, current skin, and any application-specific context or user-preference
information needed by the application's navigators and renderers. It
provides these things as named bindings, e.g. "user" for the user object,
"skin" for the skin, etc. The application's policy for obtaining these
objects is encoded in the definitions of the bindings. So, to create a new
authentication policy, I subclass a Publication class and change the
bindings. Or, I can change the Publisher class to create the Publication
with a specified user object or skin or whatever. The mechanism and policy
are up to the developer.
This doesn't mean we may not end up with an IAuthService or something like
that, with the 'user' Once binding using the IAuthService to look up the
user. It's just that we don't *need* all that extra machinery up front, to
do simple things. Changing to complicated things can always be done later,
if we really need it.
It should be really easy to break up the bindings into lower-level bindings
like "skinName", "userName" and the like, in order to do this. I can
envision having bindings like 'userName =
bindTo("request/cookies/__username")' in the Publication class to do this
kind of thing. Maybe a 'userDM' binding that refers to a DM that the user
name is looked up in to get the user object. Skins wouldn't be that different.
Publication responsibilities also include setting up and finishing each web
hit, and traversing to an object and invoking it. So an application can
add app-specific code to these things as well, if it needs to. The default
implementation will be to wrap the hit in a transaction, to traverse
objects by adapting them to Navigators (using the Publisher's Navigator
protocol), and invoke objects by adapting them to Renderers (using the
Publisher's Renderer protocol).
A navigator is responsible for mapping names to objects. While doing this,
it has access to the Publication object, so it can check the user's
permission to access a given name, or look up resources from the skin,
etc. Since navigators can be application-specific, this can take whatever
form is desired. Ideally, navigators would also have the ability to get
metadata like a location title or URL, and the ability to iterate over a
"menu" of choices that the user has permission for. But these are not
required for the core, and individual applications could define their own
app-specific interfaces for this if needed.
A renderer is responsible for sending an HTTP response, by returning a
value or using response.setBody(), streaming output, etc. The typical
renderer is some type of page template, but it may also be a wrapper on
some object's method (or methods). (For example, one might define an
"action" renderer that accepted form data and then set various features of
a persistent object.) Renderers also get access to the Publication object,
so that they can get access to Resources via the Skin. Renderers may
themselves be Resources from the Skin, that were found via a Navigator or
registered as an adaptation for a particular application class.
A resource is a presentation component, like a page template, image, or
i18n message catalog. Resources are accessed via a skin, so that they can
be changed for different presentation contexts. (E.g. WAP vs HTML,
high/low bandwidth, "themes", etc.)
A skin is a namespace of presentation components. In the Zope 3
architecture a skin is a collection of "layers" that contain the actual
components. We will probably want to do something similar, but it should
also be possible to define a skin by just using a class and putting in
bindings for the resources, thus making it easy to initially create an
"unskinned" application. (It can always be upgraded later in a
straightforward fashion.) Skins can and should cache access to their
contained resources; this is especially important since there is no per-hit
resource caching. (Although, if a resource is expected to be used multiple
times in a hit, it can always be referenced as a binding on the
Publication, e.g. 'someResource = bindTo("skin/somewhere/aResource")', and
then navigators/renderers can refer to publication.someResource.)
Page Templates and Views
A page template is a resource that represents a web page or portion
thereof. I think I would like us to use a modified version of Twisted's
"Woven Web Widgets" page template approach. Like ZPT, Twisted uses only
attributes -- not special tags -- to mark up a page. Unlike ZPT, however,
Woven allows a more declarative approach. Instead of coding conditionals
and variable references and such, Woven has "model", "view", and "pattern"
markers that contain only names. "model" and "view name the subcomponents
to be selected for data and presentation, respectively, and "pattern"
identifies portions of markup to be supplied as parameters to the view
component when rendering.
The result is impressive: you can create *structural* markup on an existing
web page to identify patterns that exist in it. By contrast (if I
understand ZPT correctly), if you identify a loop on an existing page using
ZPT, you have to mark all of its current contents as dummy elements. For
Woven, you just pick an entry as an example and tag it with a
'pattern="listItem"' (and the loop's container with a 'view="list"') to do
the same thing.
Because views are Python objects, you can define your own views to do any
kind of widgety things you want. You could make your own calendar widget,
tree widget, menu bars, tab bars, breadcrumb viewer, etc. And these
widgets could take all of their appearance *by example* from the input
page. That means I could rough something out in text, render it via the
program, then save the page from the program and fiddle with it in a
WYSIWYG editor, and roundtrip it back with a fancier look to it. But even
if we don't or can't roundtrip a page, the process of inserting "model",
"view" and "pattern" attributes isn't nearly as tedious as reconstructing a
lot of actual code embedded in a page.
There are some downsides to Woven. For one, I don't see any way to pull
out and use its rendering engine, so we'll have to write our own. (On the
bright side, remember that it only has to interpret three attributes!) I'm
not sure how fast it can be made to run, although I suspect it might be
capable of going faster than ZPT. In fact, I can see a way to perhaps
compile Woven-style templates into Python code, that would be run as long
as a given view didn't need to manipulate the HTML being output. That
would be faster than ZPT, or indeed any templating engine that doesn't
compile to Python or use some kind of C speedup mechanism. Views of course
will already be "compiled to Python" since they will be written in it by
the view developer. :)
Open Issues and Next Steps
There's a lot of stuff to decide, and a lot of stuff to write, not
necessarily in that order (since we may have to try some things to learn
what we like). One risk here is that we don't know how well contextual
adaptation really works yet, nor have we established standards and
practices for its use.
But many of the things we don't know, don't need to be addressed in the
early stages. The core interfaces are the Publisher, Publication,
Navigator, and Renderer, and by themselves they are sufficient to create a
web application. We can solidify the basic interfaces for these without
yet nailing down skins or page templates, or defining a permissions scheme,
or creating static resource classes, naming conventions for resources, etc.
There is some conceptual overlap between our Navigator and Woven's
"models", and between our Renderer and Woven's "view". It's not clear to
me yet whether we can make them 100% overlaps. It'd be nice, though,
because then page templates would be "secure", in the sense that a page
template couldn't access any data that the navigator didn't permit it
to. This could be leveraged so that parts of a page just don't appear if
they're not accessible. Also, it would mean that anything that could be
"navigated" would be browsable-to and potentially renderable. We could
also possibly use lookupComponent() with the values in the "model" and
"view" attributes, which means that we wouldn't have to design Yet Another
Interface for looking up submodels and subviews.
It seems to me that we should in fact adapt objects to navigators to do
normal lookupComponent() path traversal, and optionally supply a "lookup
context" or "security context" to them as well. I've often found it
annoying that 'bindTo' paths can't go through mapping objects like
dictionaries. But if the lookup process adapted such objects to
navigators, they could go through them as well. With a little bit of work,
we could integrate security into this as well. And if page templates had
an "owner" that was used for the "view" lookups (the logged-in user would
be used for the "model" lookups), then we could even support "untrusted"
page template authors, that would be locked down with even more
restrictions than ZPT can muster.
So it sounds like I need to review the current lookupComponent() mechanism
with an eye to making it use adaptation to a navigation protocol, where the
protocol has a means of providing security or other context information for
the traversal. This data (protocol and context) would be able to be
supplied to lookupComponent(), and if not supplied should have reasonable
defaults that assume they are being used from "trusted" code. Adapters
that don't know how to do security checking should deny access to anything
if they are run in an "untrusted" mode, except for simple mappings or
sequences, which can leave the checking to their contents. Finally,
getParentComponent() and its brethren need to be able to supply security
context as well, so that access above a certain point can be blocked as
well. (This also allows "pseudo-roots" to exist, which could be useful
when referring to models or views in a page template; the root view should
be the skin, and the root model would be a context with standard names like
"request", "response", "here", etc.)
At that point, we will have our navigation facility! Publication objects
will only need to use normal 'lookupComponent()' operations (with extra
parameters) to securely traverse application components. It would also be
possible to use the application components as their own navigation
components, for simpler applications. (More complex apps, especially those
that are reusing components from other apps, may want to keep navigation
and security separate from the actual app components.)
One other issue that would need to be addressed is the use of URLs in view
or model attributes; presumably they should be disallowed, or else we'll
need to security-enable the naming system as well. (Although that might
not be a bad idea!) If we security-enabled URLs, we could perhaps provide
access to formulas or other interesting models or views via URLs. But that
seems awfully close to puting code in the presentation again.
Finally, to make all this work we'll need to actually have page templates
and other resource classes, although naturally we'll be able to do a lot of
initial development and testing without them.
Apart from adding page templates and resources, though, this is nearly all
refactoring of existing code and interfaces, and sounds pretty doable for
0.5 alpha 3.
More information about the PEAK