[TransWarp] Basic "storage jar" design

[Phillip J. Eby]

At 11:05 PM 6/30/02 +0200, Roché Compaan wrote:

>On Sun, 2002-06-30 at 16:43, Phillip J. Eby wrote:
> >
> > Yes, *but* it is not necessarily the case that you'll be putting a
> > department object from *that* department jar there.  Suppose you were
> > working in an RDBMS, but the source of department existence was an LDAP
> > directory.  You might set aPerson.department =
> > aDepartmentFromAnLDAPJar.  When saving aPerson, you ask the
> > SQLDepartmentJar for an oid, and it has to create a thunk or stub 
> reference
> > in the SQL database that is referenceable as a department key, but has 
> some
> > kind of linkage to the LDAP-based department info.  That's what the 
> thunk()
> > method is for.  As I noted, it's not something you'll support often, 
> but Ty
> > and I have multiple apps which do this sort of cross-DB referencing for 
> one
> > or two object types.
>
>I thought it had something to do with cross-DB referencing.  But is the
>SQLDepartmentJar really necessary?  Can't PersonJar (SQL-based) just ask
>DepartmentJar (LDAP-based) for an oid?

No, because another LDAP-based jar might ask LDAPDepartmentJar for an LDAP 
oid!  SQLPersonJar has to ask SQLDepartmentJar for an oid, because 
SQLPersonJar knows it is writing to a field which is a foreign key 
reference to the table used by SQLDepartmentJar, and so it *must* have an 
OID from the SQL department jar.  Similarly, an LDAP jar which needed to 
have a DN reference to a department would have to ask an LDAP-based jar for 
the oid.


>PersonJar doesn't really have to
>know that DepartmentJar is LDAP-based, it is only asking DepartmentJar
>for an oid.

Actually, it *does* have to know that it's *SQL-based*.  See above.  It's 
just that the SQL-based department jar has to know if it supports thunking 
references to LDAP departments, and if so, it finds or creates the 
department stub in the SQL database, and returns the SQL oid for the stub, 
NOT the LDAP oid.  (YMMV, however, in that you may have an SQL schema that 
uses LDAP identifiers, or you actually want a foreign key reference that is 
just a DN, instead of a reference to an SQL-based stub.)


> > >So if an object's state is set to "loaded" by __setstate__ you still
> > >have an empty instance.  The only difference being that it's state is
> > >set.  When does data retrieval happen for this instance, especially
> > >since its "loaded" state will prevent it.  What am I missing?
> >
> > If the state is loaded, it's not a ghost, and it has everything it needs.
>
> > >"__getitem__" returns an object from the cache or a ghost if its not in
> > >the cache.
> >
> > Yes.  preloadState() is similar, except that it *may* return a non-ghost,
> > fully loaded object.
>
>Then I think one can actually drop the method "ghost" and just call
>preloadState(oid, state="ghost").

Huh?  You lost me.  preloadState and __getitem__ both *call* ghost() as 
part of their implementaton.  ghost() is the (private) method that *you* 
write to create a ghost object.  You can't get rid of it.

Again, ghost() isn't part of the API for a jar, and nothing but the jar 
itself calls it.


>Btw, where can I look if I want to have a look at the ZODB 4 code?  I
>have a Zope3 checkout and I noticed the interfaces you mentioned are in
>there.  Is there any other place I should be looking as well?

if you look at http://cvs.zope.org/ZODB4/ you'll see that there's also a 
standalone ZODB distribution that is just selected Zope 3 packages and a 
few extra files.  ZODB 4 and Zope 3 are being developed concurrently.  One 
thing that's nice is that the ZODB 4 package includes Interface and 
Transaction packages, so that's only one package to install to support most 
things PEAK needs.  PEAK will probably just end up requiring ZODB 4 to be 
installed, even if you're not using its persistence features, just to be 
sure to get the Interface package.


>Well maybe "DataManager" is not loaded metaphorically but it is
>intuitively understandable (which was a big barrier for ZPatterns).

Yeah, but I also have to consider whether I'll upset the faithful ZPatterns 
users by changing the terminology on them again.  ;)


> > Anyway, final terminology can wait a bit, since there's no code as yet.
>
>But let's not neglect it :)

If you have a strong opinion on what it should be, let's hear 
it.  :)  Right now, I'm torn between "jar" and "rack".  Rack is existing 
ZPatterns terminology that many know and love, and it does almost exactly 
the same thing.  Jar has both _p_jar and Monty Python "storage jars" going 
for it, although I could perhaps argue that '_p_jar' stands for 
"persistence: just a Rack" if I wanted to tip the balance in favor of Rack.  :)

In short, I'm undecided.  Somebody convince me of a direction to take on 
the naming.  :)