[PEAK] Generic function status update (was Re: API for generic functions?)

[Phillip J. Eby]

Basic single and multi/predicate dispatch generic functions are now ready 
for use in PyProtocols and PEAK CVS.  They do not support advanced method 
combining, but basically all the other promised features (minus C speedups, 
and lambda/listcomps in rules) are now available.  There have been some 
minor changes to the last API proposal API, as you will see below.

The executive summary?  The portions of the generic function API that are 
described in this message are now at at least alpha stability, and you can 
begin experimenting with them if you like.  At some point I will add 
advanced method combination and C speedups, but you don't need those things 
to do all sorts of cool things with generic functions.  I intend to go back 
into PEAK for a while and start using generic functions to 1) replace 
single-method interfaces in selected cases, 2) begin implementing advanced 
features like conditional views for 'peak.web', refactoring 'peak.security' 
to be generic function-based, and building other metadata registries on 
generic functions.  (Having some of these things written will help 
establish performance parameters for deciding what parts of the system 
should have Pyrex/C versions.)


At 09:44 PM 11/8/04 -0500, Phillip J. Eby wrote:

>     [dispatch.generic(summarize=sum)]
>     def priority(ctx, job):
>         """Determine priority of 'job' by summing applicable scoring rules"""
>
>     [priority.when("job.isRush()")]
>     def rush_priority(ctx,job):
>         """Add 20 to the priority for a rush job"""
>         return 20
>
>     [priority.when("job.owner.name=='Fred'")]
>     def we_like_fred(ctx,job):
>         """Add 10 more for people we like"""
>         return 10

This is now implemented in CVS, except that 'dispatch.generic()' only 
accepts a method-combiner function.


>Meanwhile, single-dispatch generic functions might be defined thus:
>
>     [dispatch.single('source')]
>     def getStreamSource(source,ctx):
>         """Return a 'naming.IStreamSource' for 'source' and 'ctx'"""
>
>     [getStreamSource.when(naming.IStreamSource)]
>     def already_a_source(source,ctx):
>         return source
>
>     [getStreamSource.when([str,unicode])]
>     def lookup_stream_URL(source,ctx):
>         # etc...

This ended up being called 'dispatch.on("argname")' rather than 
'dispatch.single("argname")', but otherwise it works as shown.


>     class NormalRules:
>
>         [dispatch.generic(summarize=sum)]
>         def priority(self, job):
>             """Determine priority of 'job' by summing applicable scoring 
> rules"""
>
>         [priority.when("job.isRush()")]
>         def rush_priority(self,job):
>             """Add 20 to the priority for a rush job"""
>             return 20
>
>
>     class Favoritism(NormalRules):
>
>         priority = NormalRules.priority
>
>         [priority.when("job.owner.name=='Fred'")]
>         def we_like_fred(self,job):
>            """Add 10 for people we like"""
>            return 10

This now works, too, except for my above-mentioned caveat about 'generic()'.


>I guess it's not too bad.  However, what about a use case where we're not 
>in the class, but want to add another case to Favoritism directly?
>
>     [Favoritism.priority.when("job.owner.name=='Bob'")]
>     def we_really_like_bob(self,job):
>         return 100
>
>Looks okay, but it won't work right without a lot of behind-the-scenes 
>implementation muck, that will also slow down access to generic functions 
>used as methods, unless I write that part with Pyrex.

I decided not to implement this one.  You'll have to explicitly do:

     [Favoritism.priority.when("job.owner.name=='Bob' and self in Favoritism")]
     def we_really_like_bob(self,job):
         return 100

when you are outside a class body.  Note also that any rules added to a 
'dispatch.generic()' inside a class body automatically have an implicit 
'and self in ThisClass' added to their condition(s).  (Technically, it's 
not 'self', but the first named positional argument.)


>At this point, I suppose we could get rid of 'dispatch.when()' altogether.

And I did implement this; 'dispatch.when()' is gone, as well as 
'dispatch.defmethod()'.  These changes were necessary to implement full 
support for all standard Python argument types including positionals, 
keywords, defaults, and nested argument tuples.  The functions used with 
'dispatch.on()' and 'dispatch.generic()' serve to set the generic 
function's name, docstring, argument signature, and default argument 
values.  The old way of just using 'dispatch.when()' to implicitly create a 
generic function didn't allow for this "function prototype" mechanism.


>   That seems to work alright, so let's start looking at the advanced API 
> for defining fancy method combinations.

I haven't started in on the method-combining APIs yet.  They remain 
important for advanced uses of generic functions, but the basic functions 
now implemented should be quite usable.

Also, I implemented a somewhat tricky optimization for both kinds of 
generic functions, that was basically needed in order to support efficient 
argument handling, but which should also speed them up in 
general.  Specifically, when you use the 'dispatch.on()' or 
'dispatch.generic()' decorators, you will get back an actual 
honest-to-goodness Python *function* object, not an instance of a special 
'dispatch' type.  An actual Python function is generated, that wraps the 
dispatch logic.

This lets us leverage Python's built-in argument processing (including 
error messages for missing/extra/malformed arguments), not to mention 
numerous "fast-path" optimizations in the Python interpreter for calling 
regular functions.  Last, but not least, it means that documentation tools 
like pydoc and epydoc will "see" generic functions as normal functions or 
methods, and therefore have a better chance of documenting them correctly.