[PEAK] Events and listeners

[Phillip J. Eby]

After letting the rough draft of peak.events sit for a while, I think I 
know how I'd like to have events and listeners work in PEAK, in general.

The draft version of peak.events had callbacks that took no arguments.  The 
current draft of peak.running.timers has callbacks with several 
arguments.  I think I'd like to compromise on a format that takes two 
arguments:

class IEventSink(protocols.Interface):
     """Accept an event from an event source"""

     def __call__(source, event=None):
         """Accept 'event' from 'source'; return truth to "consume" event"""

Here's the design theory:

* Lots of useful event sources really need to provide some data as part of 
the event.  Although the draft mechanism for values, queues, etc. provides 
a way to get the "current value" from certain kinds of sources, it would be 
much more efficient to simply provide the value with the callback.

* We only need to provide *one* value, though, because it can be an 
arbitrary object, tuple, dictionary, etc.  We don't need a strongly-typed 
event mechanism for this interface, because that's defined by the specific 
thing you're getting called back by.  And, we make it optional since some 
event sources don't have any event data besides "I just happened."

* But, we keep the source as a separate parameter, so that the 'event' can 
be a simple value (like a string or integer).  Otherwise, each new event 
source would have to come up with an event type or data structure 
convention to holding the source as well as what other data was 
needed.  And, it makes it possible to quickly call back the source to e.g. 
re-add the callback.

* Some event services and systems need a way to "consume" events.  That is, 
a way to stop their propagation through a sequence of alternate 
handlers.  For example, most GUI frameworks have a mechanism like this, and 
I also want the PEAK logging framework to be able to have handlers that 
process log events this way.  So, if an event source works that way, the 
convention will be "return truth to "consume" an event", since the natural 
way for most handlers to end is by not returning a value... i.e., None, and 
consuming an event should not be the default case.

I think this is simple enough of a signature to work for pretty much 
anything we need across an enormous variety of events.  Unfortunately, 
callback parameters are pretty much wasted on threads, since there isn't 
any way to get them back into the underlying iterator.  Unless of course we 
gave threads a "mailbox" that held incoming events, and supplied it to the 
iterator, e.g.:

     def doSomething(mbox,otherArg,...):
         yield anEventSource; resume()
         for source,event in mbox:
             ...

OTOH, if you need this, it could perhaps be expressed as:


     def doSomething(otherArg,...):
         mbox = EventQueue(anEventSource)
         yield mbox; resume()
         for source,event in mbox:
             ...
             yield mbox; resume()

with some loss of efficiency.

This brings up another issue: circular references.  Callbacks mean circular 
references, if the thing that's being called has a reference to the thing 
doing the calling.  The 'source' parameter might allow mitigating this 
issue in some circumstances, though not in the general case.  For example, 
consider the EventQueue above; if 'anEventSource' keeps a reference to one 
of the queue's methods in order to call it back, then of necessity the 
EventQueue can't be garbage collected, even if there are no references to 
it remaining.  Indeed, the queue would keep filling up with events from 
anEventSource long after anybody was taking any of the events *out* of the 
queue.  Matters are made worse by the lack of try/finally in generators, 
making it impossible to force a queue to be "closed".

We could let the EventQueue hold on to 'anEventSource', but only give 
'anEventSource' a reference to the queue when there's at least one callback 
(reader) present on the queue.  This lets garbage collection take place, 
but at the price of ignoring events while you're not actively listening to 
them.

So the only way I see to deal with the circularity issue in general is to 
have listener-adapters that hold a weak reference to their target, know 
what attribute of the target to invoke, and can deal with the target going 
away.

On the other hand, some kinds of listeners, like logging handlers, never 
really go away, since they aren't a pipeline to some thread that may or may 
not be paying attention.  So they don't need weak reference 
management.  That probably means that we'll need to ensure that event 
sources based on other event sources have a way to supply weakrefs to their 
upstream subscription, rather than build weakref management into the 
event-producer side of the equation.  Thus, event "transformers" that 
consume events via callbacks and then produce events of their own, need to 
have weakref management for their input subscriptions.

Interestingly, this need would go away if there was a means for passing 
values back into a running thread, since this would allow one to receive 
the value that was being "waited on".  Indeed, the idea of subscriptions 
could go away entirely (except for "handler"-type event sinks), since it 
would suffice to have a thread wait for, and directly receive, the values 
it needed from an event source.

OTOH, the existence of a way for the data to "come back" into the thread 
would effectively be a queue, so it's not clear that it'd solve the problem 
in the general case.  But, since callbacks would be of a one-shot nature, a 
dead link would clear itself quickly.  That is, even if the thread were no 
longer runnable, the event source it was waiting on last would drop its 
reference as soon as its next firing.  So, that actually would break the 
circularity as soon as the event fired.

This seems to make a strong argument for using only one-shot callbacks on 
event sources that you "pull" from.  If you needed a callback to happen 
repeatedly, you just loop yielding to that event source, which re-adds the 
callback each time you actually need it.  Since nothing else happens while 
a thread is between yields, you can't miss any events.  Of course, if 
you're yielding to one event, you could miss an occurrence of 
another.  But, that's what 'AnyOf' will be for, to monitor multiple event 
sources at once.

[much rambling and many unworkable approaches deleted]

I've got to give some more thought to inter-thread communication in order 
to be able to finish this up.  I'll do that on the way home, though.