[TransWarp] Re: [Zope3-dev] EventService, references, and subscription semantics

[Phillip J. Eby]

Okay, maybe I'm missing something, but it seems to me like the simplest 
decomposition would be:

1. 'IReferenceable' - interface with a 'getReference()' method, returns a 
reference to the referencable thing.

2. 'IReference'  - subclass of IReferenceable, whose implementations' 
'getReference()' methods return 'self'.  Includes a 'dereference()' method, 
as well as '__eq__()' and '__hash__()' (general comparison doesn't really 
make sense for references, but dictionary lookup and equality checking 
would be useful).  'IReference' objects should be immutable and 
pickle-able, but should probably *not* be Persistent (the overhead would be 
wasteful).

3. Things that need references, use 'getAdapter()' to get an 
'IReferenceable' implementation for the thing they want a reference to.

4. You're done.  :)  Since 'IReference' is a subclass of 'IReferencable', 
if you were passed a reference, you've got what you need.  If you didn't 
have a reference, then you'll get something that can give you one.  All the 
lookup policies that y'all are talking about can be encapsulated in the 
adapter that implements IReferenceable, and configured via the normal 
channels.  If 'getAdapter()' is insufficiently placeful, make the adapter 
do placeful lookups and delegate to the appropriate authorities.

How's that sound?  I think it would also be nice if there was a way to 
convert a reference to a short text string that could be turned back into 
an object, for non-pickling purposes, but that might be a little too 
ambitious.  OTOH, Ty and I have successfully used the following pattern for 
references in other applications:

Consider a reference to be a URL of the form "scheme:opaquedata".  Since 
the RFC's say a URL scheme may include '"."' characters, a dotted path to a 
class or function in a Python module may be used.  Implement a general 
lookup function which loads the scheme implementation, and then calls it 
with the full URL string, to receive an object.

This concept has served us especially well for referencing various kinds of 
objects such as SQL and LDAP database connections, logfiles, lockfiles, 
etc.  We use it in conjunction with a simple configuration file format that 
maps symbolic names to URL references.  Code then refers to symbolic names 
such as "applicationDatabase", which are looked up in the configuration 
file and converted to a URL, before being converted into the actual object 
needed by the application.

A similar pattern, implemented in Zope3, would extend the reach of ZCML 
considerably, to reference arbitrary objects inside *or* outside the 
ZODB.  Interestingly, it could be implemented in a way that would be 
backward compatible with existing "module.class" references, since they do 
not use the '":"' character currently.

I was just in the process a few days ago of designing a more general 
version of our current reference facilities to incorporate into 
TransWarp.  If there is interest on the Z3 side, perhaps we could create a 
joint specification for object naming based on this URL pattern, or even a 
joint implementation for Zope and TransWarp.  I've been studying the Java 
JNDI naming specs, and a lot of what's there could be applied to Zope 
naming, but in the short-term I only intend to work on the URL part.

The relevant part from JNDI is that one can configure a "package path" for 
URL schemes, which is a set of package prefixes that are added to the 
scheme name in succession until a loader is found.  So, if the package path 
were '["Zope.Naming", "TW.Naming"]', the URL '"foo.spam:bar/baz"' would 
cause a lookup of 'Zope.Naming.foo.spam.', then 'TW.Naming.foo.spam.', and 
the first item found would be taken.  I think that for Python, the 
convention should try the bare 'foo.spam' first, *if* it contains a dot.

Of course, configuring the package path could be done from ZCML...  :)  (as 
well as from Python code and perhaps an environment variable)

I was originally thinking that I would implement this logic in a TW.Naming 
package for TransWarp.  However, I would not be averse to creating a 
separate distribution package - PyNDI perhaps? - that could be shared by 
Zope, TransWarp, and any other Python application that could use this sort 
of capability.  ZCML could then convert reference strings to PyNDI.URL 
objects, which would implement IReference.  IReference and IReferenceable 
would actually be defined and exported by the PyNDI package.

Over the somewhat longer term, PyNDI could be expanded to implement JNDI's 
other concepts such as naming contexts, which I think could be very useful 
in relation to Zope concepts of traversal and namespace manipulation.

What do y'all think?  I was planning to implement a rudimentary form of 
this sometime in the next couple of weeks, anyway.  The only real question 
is whether I should implement it as TW.Naming or PyNDI...  :)  (Well, 
actually, I imagine that the ZC folks will want to add a bit more "process" 
before implementation, but that's okay by me.)


At 09:01 AM 2/26/02 +0000, Steve Alexander wrote:
>Shane Hathaway wrote:
>>
>>That sounds reasonable for this case, but I don't think we've had enough
>>time to think to be designing a service yet.  Maybe we have, but I'd like
>>us to leave room for adding an event subscriber that either provides its
>>own reference, or doesn't need to use a reference at all.
> >
>>Hmm, maybe that's not a problem: the addSubscriber() method of the event
>>service (or whatever the method is called) might accept either a reference
>>object or some other kind of object; if the object is not a reference then
>>the event service tries to make one for it.  That's a similar pattern to
>>what getAdapter() does.
>
>I suggest that this isn't the job of the subscribe() method of the 
>EventService. The ReferenceService (as in the diagram in my last email) 
>could just return the object if it already implements IReference.


>>I think IReferenceLookup is the wrong name--I think what you want is
>>something like IReferenceGenerator.  The IReferenceGenerator is only used
>>when creating a reference, not when dereferencing one, which the name
>>"IReferenceLookup" tends to imply (although the implementation might, at
>>its option, refer to the IReferenceGenerator that created it in order to
>>dererence.)
>
>Sure. It is good to have more accurately descriptive names :-)
>
>So, heres what we have so far. I guess I should put this on a wiki soon.
>
>
>Either an object can be its own IReference, or the object's context 
>provides a utility to say how to make an appropriate kind of reference, or 
>the IReferenceGenerator utility/service provides IReferences, based on 
>sensible zcml configured defaults.
>
>
>       Root  :-- IReferenceGenerator (service/utility)
>      /    \
>     Bar   Foo
>    /        \
>  Baz        LDAPUserFolder   :-- IReferenceGeneratorForContext (utility)
>                \
>                 LDAP Principal Object
>
>
>So, object Baz gets the LDAP Principal Object (let's call it "user"). It 
>wants to store a reference to user. Baz might well be an IEventService, or 
>other kind of ISubscribable object.
>
>   1: Baz gets the IReferenceGenerator service
>
>   2: Baz passes user to the IReferenceGenerator service
>
>   3: If user implements IReference, the IReferenceGenerator service
>      simply returns user as its own IReference.
>
>   4: Otherwise, the IReferenceGenerator service examines user's context
>      wrapper, and walks the wrapper looking for an
>      IReferenceGeneratorForContext utility.
>
>   5: The IReferenceGenerator service passes user to the
>      IReferenceGeneratorForContext utility, which returns an appropriate
>      IReference object.
>
>   6: If the IReferenceGeneratorForContext service returns None, or if it
>      can't be found, the IReferenceGenerator service does whatever it
>      is configured (by zcml directives perhaps) to do, such as return
>      a simple default IReference for the object.
>
>
>What about comparing IReferences. I think I'd like IReference to implement 
>__cmp__ or something, and have some appropriate semantics.
>
>--
>Steve Alexander
>
>
>_______________________________________________
>Zope3-dev mailing list
>Zope3-dev at zope.org
>http://lists.zope.org/mailman/listinfo/zope3-dev