[PEAK] API for generic functions?
Phillip J. Eby
pje at telecommunity.com
Mon Nov 8 21:44:16 EST 2004
Now that I've implemented most of the generic function API from my last
post (dispatch.SimpleGeneric, dispatch.when, dispatch.as, etc.), I'm
wondering if maybe it needs to be tweaked a little bit.
Currently, you can use 'dispatch.when()' to either create a new generic
function or update an existing one. However, if you wish to create a
single-dispatch generic function, you must explicitly create it first,
using SimpleGeneric.
I'm wondering if maybe this should be changed, such that you must always
explicitly define a generic function before adding methods. There are
certain advantages to that, especially if the definition is done with a
decorator on a function. For example, you can cause source-parsing
documentation tools (like HappyDoc) to emit correct documentation for the
generic function that way. And, you can specify arguments to the decorator
to configure the generic function. For example, you could select the
method-combination strategy to be used:
[dispatch.generic(summarize=sum)]
def priority(ctx, job):
"""Determine priority of 'job' by summing applicable scoring rules"""
[priority.when("job.isRush()")]
def rush_priority(ctx,job):
"""Add 20 to the priority for a rush job"""
return 20
[priority.when("job.owner.name=='Fred'")]
def we_like_fred(ctx,job):
"""Add 10 more for people we like"""
return 10
Now, there are three functions in the above code, and 'priority' would
invoke the other two as applicable, adding the returned scores
together. Or, we could change 'reduce' to 'max' so the function returns
the highest priority assigned by any rule. Meanwhile, the rules
'rush_priority' and 'we_like_fred' are available to be reused in other
generic functions, or just as standalone functions.
Meanwhile, single-dispatch generic functions might be defined thus:
[dispatch.single('source')]
def getStreamSource(source,ctx):
"""Return a 'naming.IStreamSource' for 'source' and 'ctx'"""
[getStreamSource.when(naming.IStreamSource)]
def already_a_source(source,ctx):
return source
[getStreamSource.when([str,unicode])]
def lookup_stream_URL(source,ctx):
# etc...
The idea here is that you would name the argument that dispatch will occur
on; a feature we don't actually have yet, but which would be nice to have,
since it would allow single-dispatch generics to be used as methods, not
just multi-dispatch ones. (Of course, neither kind of generic function can
be used as a method yet, anyway.)
Method definition with the approach above could get a little awkward, though:
class NormalRules:
[dispatch.generic(summarize=sum)]
def priority(self, job):
"""Determine priority of 'job' by summing applicable scoring
rules"""
[priority.when("job.isRush()")]
def rush_priority(self,job):
"""Add 20 to the priority for a rush job"""
return 20
class Favoritism(NormalRules):
priority = NormalRules.priority
[priority.when("job.owner.name=='Fred'")]
def we_like_fred(self,job):
"""Add 10 for people we like"""
return 10
I guess it's not too bad. However, what about a use case where we're not
in the class, but want to add another case to Favoritism directly?
[Favoritism.priority.when("job.owner.name=='Bob'")]
def we_really_like_bob(self,job):
return 100
Looks okay, but it won't work right without a lot of behind-the-scenes
implementation muck, that will also slow down access to generic functions
used as methods, unless I write that part with Pyrex.
At this point, I suppose we could get rid of 'dispatch.when()'
altogether. That seems to work alright, so let's start looking at the
advanced API for defining fancy method combinations.
CLOS "short-form" method combinations could look something like this:
[dispatch.method_combination(operator.add)]
def add():
"""Summarizes results by adding them together"""
In addition to taking an operator and a function, there'd be two keyword
args to 'method_combination', unary_identity=True and
order=dispatch.MOST_SPECIFIC_FIRST. (The default
values.) 'unary_identity' means, if there's only one applicable method,
just return its value, rather than iterating over applicable
items. 'order' should let you specify MOST_SPECIFIC_FIRST,
LEAST_SPECIFIC_FIRST, DEFINITION_ORDER, or REVERSE_DEFINITION_ORDER
For more advanced method combinations (like before/after/around, it would
also be necessary to specify "method groups" that are used to select
specially-qualified methods. Method groups would be defined by something like:
around = dispatch.MethodGroup("around")
primary = dispatch.MethodGroup("primary", qualifiers=(), required=True)
after = dispatch.MethodGroup("after", order=dispatch.LEAST_SPECIFIC_FIRST)
before = dispatch.MethodGroup("before")
And then they would be used something like:
[dispatch.method_combination(groups=[before,after,around,primary])]
def standard_combination(primary,before,after,around):
"""CLOS-like method combination"""
run_primary = dispatch.method_chain(primary)
if before or after:
befores = dispatch.method_list(before)
afters = dispatch.method_list(after)
def inner(*__args,**__kw):
if before:
for b in befores(*__args,**__kw): pass
result = run_primary(*__args,**__kw)
if after:
for a in afters(*__args,**__kw): pass
return result
else:
inner = run_primary
return dispatch.method_chain(list(around)+[inner])
Here, we define a method combination that runs any 'around' methods
"around" the 'inner' method, which runs the 'before' and 'after' methods
around the chained 'primary' methods. (In other words, the standard CLOS
method combination approach.)
Qualifiers defined by method combinations (e.g. 'before', 'after', and
'around') should effectively add methods to the generic function instance,
so that 'someGeneric.before("condition")' then works.
CLOS allows multiple qualifiers per method, but I'm not sure if we need
that. It could always be added later, if we build the innards right. CLOS
also expects methods to be qualified when using "short-form" method
combinarions, such that if we were doing:
[dispatch.generic(strategy=add)]
def priority(ctx, job):
"""Determine priority of 'job' by summing applicable scoring rules"""
it would then expect individual methods to be defined with the equivalent of:
[priority.add("job.isRush()")]
def rush_priority(ctx,job):
"""Add 20 to the priority for a rush job"""
return 20
But I don't think I like this. CLOS presumably does this for redundancy,
so that you don't forget that the value you return is going to be added (or
and'ed or or'ed or whatever). There's something to be said for that. I
guess maybe we just need to be able to specify the names nicely, like:
[priority.add_when("job.isRush()")]
def rush_priority(ctx,job):
"""Add 20 to the priority for a rush job"""
return 20
That looks better to me. Maybe you'd define it with:
[dispatch.method_combination(operator.add, method="add_when")]
def add():
"""Summarizes results by adding them together"""
for the short-form qualifier. A long form qualifier would look something like:
around = dispatch.MethodGroup("around")
primary = dispatch.MethodGroup("primary", qualifiers=("add_when",),
required=True)
[dispatch.method_combination(groups=[around,primary])]
def add(primary,around):
"""CLOS-like 'add' combination"""
if len(primary)==1:
inner = primary[0]
else:
primaries = dispatch.method_list(primary)
def inner(*__args,**__kw):
return reduce(operator.add,primaries(*__args,**__kw))
return dispatch.method_chain(list(around)+[inner])
Which shows rather nicely how the short forms will be converted to long
forms. Another example:
around = dispatch.MethodGroup("around")
primary = dispatch.MethodGroup("primary", qualifiers=("and_when",),
required=True)
[dispatch.method_combination(groups=[around,primary])]
def And(primary,around):
"""CLOS-like 'and' combination"""
if len(primary)==1:
inner = primary[0]
else:
primaries = dispatch.method_list(primary)
def inner(*__args,**__kw):
for value in primaries(*__args,**__kw):
if not value:
break
return value
return dispatch.method_chain(list(around)+[inner])
This would be then be used like:
[dispatch.generic(strategy=And)]
def allowed(...):
"""Blah"""
[allowed.and_when("some condition")]
def allow_only_if_not_blah(...):
return False
Whew. I think I'm about worn out for the moment. Let me try to make a
list of what I'd need to implement all this:
* A 'MethodGroup' class that knows how to test whether a (possibly
qualified) method belongs in that group, and strip off the qualification
wrapper if any.
* A 'method_combination' decorator that constructs a method-combining
function and a list of qualifier method names, from the short form or the
long form invocation.
* 'method_list' and 'method_chain' function constructors, where each take a
list of functions and return a function that either iterates over the
functions' return results, or allows each function to call the next with
'dispatch.next_method()', respectively. They should also have the option
of allowing the list/chain to resolve any prioritization ambiguities,
rather than raising AmbiguousMethod when two applicable methods have
overlapping rule definitions.
* MOST_SPECIFIC_FIRST, LEAST_SPECIFIC_FIRST, DEFINITION_ORDER, and
REVERSE_DEFINITION_ORDER functions to handle sorting.
* GenericFunction needs a __getattr__ or other mechanism to access
"qualifier methods" for the qualifiers that are applicable to it. This
mechanism also needs to work with GenericFunctions accessed from a class,
e.g. 'Favoritism.priority.add_when' should be a different object than
'NormalRules.priority.add_when', with different behavior.
* Make "qualifier methods" detect when they are used in a class, and if so,
delay doing the actual registration of the corresponding function until the
class has been created, at which point the registration should be done with
a condition requiring the first argument to be an instance of the
applicable class, for that particular function to be applicable.
* Tweak protocols.advice() functions to accept frames as well as depth, to
make it easier to mix class and function advice, as some of the above need
to do.
Too bad most of these things are much easier to describe than to code. :)
Interesting side note: all of this method combination stuff should also be
applicable to database-backed generic functions. That is, if you create a
database with a bunch of business rules in it (following the San Francisco
"Key/Keyable" pattern), you could still use these method-combining
techniques to combine the found, applicable rules, as long as you had a way
to map the persistent form of a rule into 'dispatch'-compatible predicates
and callables.
Hmmm... you know, you could actually map almost all of GenericFunction's
internal DAG onto relational DB tables, especially since relational
databases can (at least in theory) do some of the traversal optimization
themselves that GenericFunction objects do. That is, right now we build
indexes that map from various value classifications to what cases are
applicable. We then do the equivalent of joining these tables to extract a
collection of applicable rules. You could do almost the exact same thing
in a database. Verrry interesting, indeed. Maybe I should patent that
idea, in the grand tradition of today's wonderful "do X, but on a
computer/in a database/on the Internet" patents. ;)
More information about the PEAK
mailing list