[PEAK] Messages to the future (in the Trellis)
Phillip J. Eby
pje at telecommunity.com
Thu Jul 26 11:57:43 EDT 2007
Now that the high-level API is complete and I've started work on
doc/test refactoring, I've started thinking about supporting mutable
data structures such as lists, sets, and dictionaries.
In the curent Trellis model, you can't detect when such things have
changed; you have to use custom objects (similar to ZODB's
PersistentList and PersistentDict) that can track their own changes.
However, implementing such structures correctly is rather tricky,
especially the mutation parts.
Within a given Trellis pulse, values aren't allowed to actually
change, because then different rules might see different values. So,
when you implement a mutable data structure, it has to defer making
any actual changes until the next trellis pulse.
In the API examples I've written previously, I've used a number of
different hacks to handle this, including my recent "hub/spoke" examples.
But after a lot of thought, it seems to me that what we need is a
simple way to manage "messages to the future". For example, here's a
simple Set implementation:
added = trellis.todo(lambda self: set())
removed = trellis.todo(lambda self: set())
to_add = added.future
to_remove = removed.future
data = self.data
if data is None:
data = set()
for item in self.removed:
for item in self.added:
def __contains__(self, item):
return item in self.data
def add(self, item):
if item in self.to_remove:
def remove(self, item):
if item in self.to_add:
To work, it requires a new feature: "todo". The idea is that any
attribute marked "todo" is a data structure containing changes to be
made to the object, and you can also use it to create a "future"
version of the same attribute. This "future" version is a read-only
property that lets you access the *future* version of the "todo" attribute.
If there is no future version available yet, a new value is
calculated using the todo attribute's rule.
Effectively, a 'todo' creates a 'receiver' cell whose default value
is created using the supplied rule. Accessing its future
automatically assigns a fresh value to it, if needed. And presto, we
can now operate on our future events directly, by simply referring to
their future versions. And, since the todo's are receivers, they
automatically reset to their empty value after everyone has seen the changes.
This makes it fairly simple to create mutable data structures, since
one can simply design whatever change records are appropriate (such
as added/removed in the case of a set), and have the manipulation
methods manipulate this future state. Rules for the current state
can then apply the changes, so that any attempt to read the current
state, brings it up to date with the planned changes.
At this point, I'm not 100% certain this is the way to go, but it
looks quite promising. I'll have to see how useful the approach is
for other data types, though. For dictionaries, one could do
something like updated/removed, or added/changed/removed. Lists are
harder to describe changes to, however. Perhaps the best thing would
just be a list of co-ordinates and replacement slices, since all list
manipulation can be described in terms of setslice operations.
One other thing that's needed, that 'todo' doesn't provide, is any
way to tell that the object itself has "changed", so that if you use
one of these structures in another cell, any dependent rules can be
updated. Overall, however, in such a case you should probably just
refer directly to the change events, rather than treating the data
structure as if the whole thing changed.
More information about the PEAK