[TransWarp] Re: Eventually actually using TransWarp

[Phillip J. Eby]

At 11:14 PM 2/26/02 +0000, Steve Alexander wrote:
>Hi Phillip,
>
>I was just reading through CVS commit logs for TransWarp, and was reminded 
>of an earlier email where you wondered whether anyone was actually 
>interested in TransWarp after all.
>
>I'm *very* interested in this work, especially with combining it with 
>Zope3's component architecture.

Me too.  :)  Thanks for the input.


>Unfortunately, all my bandwidth is consumed by Zope3 at present.

As is mine by TransWarp.  :)  There's currently a shortage of developers at 
my workplace, and a big project that *really* needs all 3 TransWarp 
framework layers (domain, dispatch and data) finished pronto.  So I'm going 
all out at the moment, with a goal of having the dispatch layer stuff 
written this week.  As of today, the data layer is basically done, and the 
design for how Specialists will interact with Elements and the data layer 
is done.  The domain layer will also get a little cleanup and refactoring, 
but it's mostly okay already.

(I don't know whether I've posted it anywhere, but the "dispatch" layer 
serves roughly the functions which were implemented with SkinScript and 
"providers" in ZPatterns.  The "data" layer doesn't really have an analogue 
in ZPatterns; ZPatterns made you create your own data layer!)


>However, I'm following your checkin logs, and I'll be working with 
>TransWarp as soon as time allows.

Fantastic.  Currently, I'm not working on coding Persistent implementations 
of the data or dispatch layers, although I've been doing some checking to 
make sure I'm not designing myself into a corner that would make it 
difficult to put persistent Specialists, TypeManagers, etc. into a ZODB 
instance.  It looks like the trickiest bit will be adding 
__getstate__/__setstate__ methods to SEF.Base that drop out "Once" 
attributes and convert the SEFparent weak reference to/from a direct 
reference in the state.

I think it should also be possible to create a PersistentModel module based 
on DataModel that would replace TypeManager's 'recordCache' with a BTree, 
and convert Record to a PersistentMapping, thus directly creating a 
multiple-indexed variant of the old ZPatterns Rack.  There'd probably be a 
little extra work needed to ensure that "non-existent" mappings didn't hang 
around, however.  Actually, now that I think of it, that means I need to do 
the same for any TypeManager.recordCache class other than 
WeakCache!  Oops.  I guess TypeManager needs a two-tier caching strategy; 
I'll hash out a design with Ty tomorrow, I guess.  On the bright side, once 
that's solved, it'll be that much easier to implement a PersistentModel 
data layer...

Anyway, once I finish the Specialists and related items of the dispatch 
layer, TransWarp should be as functional for Z3 as ZPatterns is for Z2, 
*except* where ZODB storage is concerned.  TW will only be usable 
out-of-the-box for filesystem-based Specialists and application 
objects.  ZODB support would have to come in two parts: TW.SEF.Persistent 
(letting Specialists and suchlike objects to live in the ZODB), and 
TW.Database.PersistentModel (allowing Records to be stored in ZODB).  These 
could actually be independent of each other, since one could use Once 
objects in your TypeManager that would connect to a ZODB and pull out the 
desired BTree.  But it might be easier to debug if larger chunks of the 
application were in ZODB all at once.

I don't know when/if ZODB support will become a priority, however.  It may 
end up being a side project for me if I don't have any production needs 
driving its development.  However, if I have a "customer" for it (not 
financially speaking, but in the sense that someone who has ZODB needs can 
help me direct my efforts), then things might progress a little easier.

Hm.  I think I know a way to quickly address the two-tier caching issue, 
but only if I drop support for caching non-existence of a record.  If I 
don't cache a record until its contents are changed (by writing, or by 
loading data)...  Hm.  Only problem then would be sort of a "Schrodinger's 
Record" problem, where if you look up the same set of keys twice, but don't 
look into either record and so don't load any data...  now there's a 
collision.  But I could at least *detect* that, by ensuring that when 
writing to the cache, those slots are empty, and when removing from the 
cache, that it is the same record which is removed.  And that would also 
work if recordCache were actually a BTree of PersistentMapping-like 
objects...  A mapping not stored in the recordCache would just die away 
quietly.  Ah well...  more work will be required on that at a later time, 
but for now I can just implement cache collision detection and dropping 
non-existent records, so that the data layer can use caching policies other 
than WeakCache.