[TransWarp] Requirements for the future peak.messaging package

[Phillip J. Eby]

Comments, questions, and suggestions welcome...

* CRITICAL: Asynchronous "fire-and-forget" messaging; programs that produce 
messages in volume are implicitly very busy and should not be made to wait 
around for the messaging system to connect, etc.  This can be accomplished 
either via socket to a local daemon, or via filesystem queues processed by 
a daemon or thread.

* CRITICAL: Transaction integration for injecting or removing messages from 
a queue.  When a transaction is committed, all messages that were 
provisionally sent should be "really" sent, and all messages that were 
retrieved should be actually removed from the queue they came from.

* CRITICAL: Strong multi-processing support; queueing mechanisms should 
support simultaneous injection or retrieval by multiple processes on 
multiple machines.

* CRITICAL: Messages must be secure from being read in transit by 
unauthorized parties, and it should not be possible for unauthorized 
parties to inject forged messages into the system.  (For filesystem-based 
transit mechanisms, it suffices for adequate permissions to exist, and 
private-key encryption and message digest mechanisms should suffice for 
network transit.)

* IMPORTANT: Persistence system integration.  It should be possible to 
treat application-domain objects as "messages" and inject them into or 
retrieve them from a messaging queue, with minimal wrapping overhead, e.g. 
by providing metadata arguments on a send method and receiving a 
(metadata,object) pair upon retrieval.  This will be very useful in 
relation to both timed queues and browse/search interfaces (see below).

* IMPORTANT: Timed queues.  Timed queues make messages available for 
retrieval only after a future timestamp is reached, allowing scheduling of 
future events via message send.

* IMPORTANT: Browse/search: queues may expose browsing or searching 
interfaces to allow inspection of queue contents for system monitoring 
purposes or to select messages for consumption (except ordered queues, 
which only allow consumption in sequence).

* HELPFUL: Publish/subscribe of events or log entries, with "live" 
listening - possibly non-durable messages with time-to-live

* HELPFUL: Low-latency request/response messaging (like XML-RPC but without 
TCP connection overhead for each request).  If this existed and were both 
reasonably secure and sufficiently lightweight, it could be a basis for 
developing many kinds of advanced services such as distributed lock 
managers, Linda or Jini-style tuplespaces, etc.

* HELPFUL: Ordered queues.  Ordered queues provide a total ordering 
according to a timestamp, and each message can only be received after the 
one before it has been removed from the queue (at least 
provisionally).  (Note that this implies only a single receiver may 
retrieve records from such a queue at any given point in time.)

* NICE TO HAVE: Keyed queues; i.e., messages corresponding to the state of 
an object can be identified by a key, and later messages supersede earlier 
messages, reducing the number of messages to be picked up and processed by 
a slow or intermittent recipient.

* NON-REQUIREMENT: Conformance to protocols or APIs of existing messaging 
services is not a requirement at this time.  However, if people have such 
services and wish to ensure that peak.messaging's conceptual model can be 
safely mapped to those services, we will entertain suggestions for how to 
do so.

* NON-REQUIREMENT: Sender non-repudiation of messages is not an 
infrastructure requirement, as long as applications can build on the 
infrastructure to include digital signatures or similar mechanisms as 
needed.  Most applications, however, do not need non-repudiation and 
shouldn't pay for the extra cryptographic overhead.