[TransWarp] TableDM proof of concept, request for comments

[Phillip J. Eby]

At 07:39 PM 10/9/03 -0400, Phillip J. Eby wrote:
>I don't care about OQL per se.  Think of it more as QBE...  e.g.  "find me 
>all bulletins with a category whose kind is 'foo'" might be expressible as:
>
>Bulletin.where( category = Category.where(kind='foo') )
>
>Or, find me people at least 18 years old...
>
>Person.where( age = GE(18) )
>
>..who have at least one dog:
>
>Person.where( age = GE(18), pets = EXISTS( Pet.where(kind=PetKind.DOG) )

A further explanation of the above...

I see a future point at which model.Element classes will have a 'where' 
attribute that's a "selector class".  The selector class is automatically 
generated from the element's type metadata.  The separate class is needed 
because the base element class isn't going to accept criteria in its data 
fields.  But, it should be possible to create an automated mapping from the 
raw field info, to a criteria class.

Then, if you create an interface that represents a particular database 
schema, such as 'IBulletinsForSQLite', and there are parameterized 
interfaces like 'IPersistTo(IBulletinsForSQLite)', then you can define 
adapters from each model class to that interface, and define adapters from 
e.g. 'Bulletin.where' to 'IQueryOn(IBulletinsForSQLite)'.

So, then you could have a "database" object that was keyed by selectors, e.g.:

      bulletins = bulletinsDB.find( Bulletin.where(category=someCategory) )

The argument to 'find()' is an instance of the 'Bulletin.where' class, and 
the bulletinsDB adapts it to 'IQueryOn(IBulletinsForSQLite)', in order to 
build the necessary SQL.

Using the adaptation framework, we could define generic adapters for simple 
mappings, but still override the adapters if we wanted to create a custom 
arrangement for a particular DB schema or backend.

One other piece that seems doable is to have say, a 'PARAMETER' concept, so 
that you could say something like:

     queryTpl = Bulletin.where( category = 
Category.where(kind=PARAMETER('kind')) )
     query = adapt( queryTpl, IQueryOn(IBulletinsForSQLite) 
).buildQuery(dbConnection)

     results = query(kind="foo")

That is, you could generate a "compiled form" of a particular query, bound 
to the database you're using.  Of course, this'd probably all be done with 
bindings, and preferably in such a way that the built query object can take 
advantage of prepared statements (if the target database offers such a 
thing).  However, at the usage level that should be transparent.  (But 
maybe the 'buildQuery()' method would have options to control certain 
aspects of that.)

Most likely, 'IQueryOn' will include methods to both build a "top-level" 
query, and to add a subquery or join to an existing query.  In this way, a 
.where type's adapter can adapt its subparts to IQueryOn and use them to 
help construct the overall query.

The net result is that a database will likely have lots and lots of query 
"methods" that are pre-compiled SQL, ready to pass straight to 
DBAPI...  *but* which are compiled based on a DB schema and DBAPI driver 
that are selected at *runtime*!

It may be that we don't have separate IQueryOn/IPersistTo adapters, since 
in essence they are both simply a mapping from domain object to DB 
schema.  OTOH, they adapt from different types (SomeType vs. 
SomeType.where).  Probably, you'll do something like:

     protocols.advise(
         instancesProvide = [IPersistTo(someSchema)],
         asAdapterForTypes = [SomeType]
     )
     protocols.advise(
         instancesProvide = [IQueryOn(someSchema)],
         asAdapterForTypes = [SomeType.where]
     )

Or better yet, you'll call a simpler API that will do this for 
you.  Probably you'll subclass some handy base that will get the necessary 
information from class attributes, e.g.:

class MyMapping(TableMapping):

     elementType = SomeType
     tableType   = someSchema.SomeTable

     mapping = (
         MapField(
     )

And when the class is created it'll automatically declare itself as all the 
right things.  There are some questions in my mind, however, about how to 
assemble table types.  I'm assuming that they'll be defined similarly to 
model.Element classes, but with only "primitive" or "structure" types 
allowed for the fields.

I don't want to have to specify the schema over and over, or the protocols 
to declare, so probably the TableMapping base (or whatever it's called) 
should allow you to create an empty subclass for purposes of sharing, e.g.:

class BulletinsSQLiteMapping(TableMapping):
     schema = IBulletinsForSQLite

class UserMapping(BulletinsSQLiteMapping):
     # ...

class CategoryMapping(BulletinsSQLiteMapping):
     # ...

etc.  The same would probably be true for any options that these mapping 
shortcut classes need for SQL generation.


Ideally, all of these mappings would be in relation to the *schema*, 
without necessarily being affected by the database back-end.  In practice, 
though, it's hard to make a truly cross-platform schema, due to differences 
in available datatypes, and even differences in allowable column names(!) 
among databases.

Still, it should be possible to derive one schema from another, by making 
its target protocols 'protocols.Variation()'s of the original schema's 
protocols, and then judiciously subclassing tables and mappings where they 
need to be different.  But you might need module inheritance to do it 
well.  :(  That aspect requires some more thought.


Selectors can be used for DML queries, too.  Imagine 'someDB.delete( 
Bulletin.where(...) )', or 'someDB.update( Bulletin.where(...), 
Bulletin.set(...) )', for example.  That latter use requires a more complex 
arrangement, of course.

Anyway, there's still much to think about, many holes to plug, and *many* 
concepts to simplify from these rough sketches.  I don't even have a 
complete picture of the scope yet.  One related thing I've been thinking 
about is the idea that objects shouldn't have _p_oids at all, but are 
instead retrieved by selectors.  It's annoying right now to have to 
duplicate data into the _p_oids, when most objects in business application 
schemas already have "natural" primary keys as part of their requirements!

Anyway...  it's late, I've gotta sleep.  More on this another day.