[TransWarp] Re: [ZODB-Dev] SQL/ZODB integration (was Re: ZODB and new style classes?)

[Phillip J. Eby]

At 02:52 PM 7/1/02 -0400, Jim Fulton wrote:
>
>I spoke to someone at EuroPython who was working on an EOF persistence
>framework. He seemed to want an observer mechanism to enforce integrity
>constraints.  BTW, I think that there was some interest on a standard Python
>observer mechanism that could, possibly allow the persistence mix-in
>to be avoided.
>
>I'd like to formulate a cleaner interface between the caching architecture
>and the persistance framework. For example, currently, the persistence
interface
>includes cache-specific data, which is silly.
>

If you think that's silly, here's a *really* silly idea for implementing
observers...  if you have an object you want tracked, create a per-instance
subclass of its class.  :)  Initially, it would just have __getattribute__
and __setattr__ methods overridden...  But you can also stick properties
into it.  So in your initial "data load", you might load some attributes
into the object dictionary, but others you'd load into this "shadow class"
as properties that would load the attribute on demand.  (You'd of course
also drop the __getattribute__ method at that point, unless you wanted to
track access times or something like that.)

At first blush the idea sounds horrendously inefficient, and it might in
fact be, if only in space terms.  In other ways, it could be *more*
efficient than the current mechanisms, in that you can drop the overhead of
certain operations by removing them from the observer class if they aren't
needed while the object is in a certain state!

Note that as long as somebody is using isinstance() and not doing direct
class comparison, this seems like it should work just fine with any
new-style classes, and since the observer class doesn't add any C-structure
state, it should be a compatible mixin with any object, possibly including
(in theory) built-in types.  It's possible that advanced metaclass usage
might not be 100% compatible, however.

I'm only half serious about this idea, or maybe half joking, depending on
how you look at it.  But it certainly solves the "avoid the mixin" issue,
and really de-couples the persistence mechanisms and policies from the
object to be persisted!

I'm tempted to write a pure-Python prototype, just to try the idea out, and
see what kind of overhead it adds.  Since my framework doesn't need the
access time, though, it could ditch the __getattribute__ as soon as any
state was loaded, and __setattr__ as soon as the object entered the
"changed" state.  Which should make the resulting performance no different
than adding a single level of inheritance to the objects' class hierarchy.

The main coding complexities will be in specifying the various kind of
descriptors that will be needed, and their fallback behaviors.  For
example, if an object accesses an attribute of self.__class__, and it's
shadowed, should the shadowing descriptor redirect the access to the true
__class__?  (Probably.)  Anyhow, it's certainly food for thought.