[PEAK] API Tradeoffs (was RE: A ridiculously simple mapping layer)

[Phillip J. Eby]

At 11:53 AM 7/27/04 -0400, Phillip J. Eby wrote:
>Right now I'm going over the requirements for both providing all the 
>current functionality of DM's, while expanding the basic 
>capabilities.  And I'm also trying to balance things in my mind between 
>making the class conversion a general-purpose AOP facility, and making it 
>extremely specific to data mapping.

Hm.  I'm beginning to sense a pattern here...  whenever there are tradeoffs 
required as to the degree of flexibility an operation requires, consider 
using generic functions.

That is, I'm trying to balance the class conversion mechanism between 
extremes of simplicity and generality, trying not to pay a high price for 
early generalization, but not ruling it out in the future.  So, what if the 
mapping from abstract modules to workspace classes was done by a generic 
function?

This would in effect create a general-purpose AOP framework, of the form:

     componentClass = genericAdvisingFunction(abstractModule)

As it happens, generic functions are composable, so 
'genericAdvisingFunction' can be created from an arbitrary number of 
specific cases, just as aspects should be composable.  And, because generic 
functions (at least the ones in PyProtocols) allow testing arbitrary 
conditions, it's effectively possible to use them to specify the equivalent 
of AspectJ's *static* pointcuts.  That is, one could define a single 
advising function that applies to several methods of multiple classes, 
based on a pattern of the class and methods names, or any of several other 
things.

There are some additional features I'd need to add to generic functions to 
make this practical, mainly in the area of allowing actual composition of 
generic functions.  But they're not particularly a big deal, and I figured 
I'd implement them eventually anyway.

The main tradeoff of using generic functions to resolve the "generality vs. 
simplicity" conflict is economy of expression.  That is, expressing things 
in terms of generic functions is often quite verbose and repetitive.  But, 
this is countered by the relative ease of defining higher-level functions 
that operate by creating closures and adding them to one or more generic 
functions on your behalf, so that you can express yourself in 
domain-specific terms.  For example, by calling functions to 
'declareField()' or 'mapTable()' or some such, rather than defining methods 
directly.  However, the fact that the implementation is in terms of generic 
functions means that you can define your own high-level "macros" or even 
code directly to the generic function when needed.

The other tradeoff that generic functions introduce are the issues of 
desigining the function signature(s) and method combination rule(s).  That 
is, generic functions are *so* generic that you could quite literally use 
one generic function for *everything*, just by giving it an extra parameter 
to say what it is you're supposed to be doing.  :)

Naturally, that's a bad idea.  But having too many functions can also be 
problematic, because a given domain-specific operation like 'mapTable()' 
might need to add methods to generic functions for loading an object, 
saving an object, reading or modifying an object's fields, and so on.  And 
that means that the DSO (domain-specific operation) needs access to all 
those functions, and we don't want you to have to pass them in.  You want 
to say something like 'mapTable(schema,...)', where that schema object 
carries all the needed generic functions.  (Of course, objects could 
provide DSO's as methods, e.g. 'schema.mapTable(...)', but that's just a 
different way of spelling the same thing.)

Anyway, this seems to me to mean that a common case for generic functions 
is that they will live on some instance, possibly "inheriting" an initial 
definition from their class.  For example, a "relational workspace" class 
would define several generic functions for the operations needed by 
objects, and supply the initial default implementations for the parts of 
those operations that are not schema-specific.  Schema-specific methods 
would then be added as a result of calling DSO methods like 
'workspaceClass.mapTable(...)', but could also be defined directly by 
calling 'workspaceClass.someOperation.addMethod(predicate,function)'.

To make things even weirder, some of those generic functions will be the 
runtime behavior of the workspace (e.g. get and set object attributes), but 
some will be "build-time" behavior, i.e. defining how to transform model 
classes into implementation classes.

Ow.  That makes my own head hurt.  :)  I think I'm going to need some 
"separation of church and state" (behavior and state?) there.  More 
specifically, I need to get a clearer idea of how and when the functions at 
each level are defined.  I just caught a glimpse of how everything is 
"generic functions all the way down", and it made me dizzy.  However, I 
expect it's probably something that can be gotten used to.  After all, 
since version 2.2, Python is "metaclasses all the way down", but most of 
the time nobody notices or cares about that fact.  :)

There's a lot more that needs to go into this train of thought, but I'll 
need to continue it later.