Ignore changes in the amount of whitespace
Differences between version dated 2005-01-15 23:45:18 and 2005-01-16 23:51:46
(spanning 3 versions)
Deletions are marked like this.
Additions are marked like this.
prevents users from having to "relearn how to write good code" in order to use
the new features successfully.
This PEP directly competes with PEP 245, which proposes a syntax for
Python interfaces. If some form of this proposal is accepted, it
would be unnecessary for a special interface type or syntax to be added to
Python, since normal classes and partially or completely abstract classes will
be routinely usable as interfaces. Some packages or frameworks, of course, may
have additional requirements for interface features, but they can use
metaclasses to implement such enhanced interfaces without impeding their
ability to be used as interfaces by this PEP's system for creating extenders.
Of course, given the number of previous failed attempts to create a type
declaration system for Python, this PEP is an act of extreme optimism, and it
will not be altogether surprising if it, too, ultimately fails. However, if
only because the record of its failure will be useful to the community, it is
worth at least making an attempt. (It would also not be altogether suprising
worth at least making an attempt. (It would also not be altogether surprising
if this PEP results in the ironic twist of convincing Guido not to include
type declarations in Python at all!)
type declarations.
PEP 246 presents a basic implementation model for automatically finding an
appropriate adapter so that one type can conform to the interface of another.
appropriate adapter, given an object to adapt, and a desired interface.
However, in recent discussions on the Python developers' mailing list, it
came out that there were many open issues about what sort of adapters would
be useful (or dangerous) in the context of type declarations.
PEP 246 was originally proposed for an explicit adaptation model where an
``adapt()`` function is called to retrieve an "adapter". So, in this model the
adapting code potentially has access to both the "original" object and the
adapted version of the object. Also, PEP 246 permitted either the caller of a
function or the called function to perform the adaptation, meaning that the
scope and lifetime of the resulting adapter could be explicitly controlled in a
straightforward way.
By contrast, type declarations would perform adaptation at the boundary between
caller and callee, making it impossible for the caller to control the adapter's
lifetime, or for the callee to obtain the "original" object.
Many options for reducing or controlling these effects were discussed. By and
large, it is possible for an adapter author to address these issues with due
care and attention. However, it also became clear from the discussion that
most persons new to the use of adaptation are often eager to use it for things
that lead rather directly to potentially problematic adapter behaviors.
Also, by the very nature of ubiquitous adaptation via type declarations, these
potentially problematic behaviors can spread throughout a program, and just
because one developer did not create a problematic adaptation, it does not mean
he or she will be immune to the effects of those created by others.
So, rather than attempt to make all possible Python developers "relearn how to
write good code", this PEP seeks to make the safer forms of adaptation easier
to learn and use than the less-safe forms. (Which is the reverse of the
current situation, where less-safe adapters are often easier to write
than some safer ones!)
Kinds of Adaptation
-------------------
Specifically, the three forms of type adaptation we will discuss here are:
Operational Conformance
Providing operations required by a target interface, using the
operations and state available of the adapted type. This is the
simplest category of adaptation, because it introduces no new
state information. It is simply a specification of how an instance
of one type can be adapted to act as if it were an instance of
another type.
Extension/Extender
The same as operational conformance, but with additional required state.
This extra state, however, "belongs" to the original object, in the sense
that it should exist as long as the original object exists. An extension,
in other words, is intended to extend the capabilities of the original
object when needed, not to be an independently created object with its
own lifetime. Each time an adapter is requested for the target interface,
an extension instance with the "same" state should be returned.
Volatile/View/Accessory
Volatile adapters are used to provide functionality that may require
multiple independent adapters for the same adapted object. For example,
a "view" in a model-view-controller (MVC) framework can be seen as a
volatile adapter on a model, because more than one view may exist for the
same model, with each view having its own independent state (such as window
position, etc.).
Volatile adaptation is not an ideal match for type declaration, because it
is often important to explicitly control when each new volatile adapter is
created, and to whom it is being passed. For example, in an MVC framework
one would not normally wish to pass a model to methods expecting views,
and wind up having new views created (e.g. windows opened) automatically!
Naturally, there *are* cases where opening a new window for some model object
*is* what you want. However, using an implicit adaptation (via type
declaration) also means that passing a model to *any* method expecting a view
would result in this happening. So, it is generally better to have the methods
that desire this behavior explicitly request it, e.g. by calling the PEP 246
``adapt()`` function, rather than having it happen implicitly by way of a type
declaration.
So, this PEP seeks to:
1. Make it easy to define operational and extension adapters
Over a long period of time, it became clear that there are really two
fundamentally different types of adaptation that are in common use. One type
is the "extender", whose purpose is to extend the capability of an object
or allow it to masquerade as another type of object. An "extender" is not
truly an object unto itself, merely a kind of "alternate personality" for the
object it adapts. For example, a power transformer might be considered
an "extender" for a power outlet, because it allows the power to be used with
different devices than it would otherwise be usable for.
By contrast, an "independent adapter" is an object that provides entirely
different capabilities from the object it adapts, and therefore is truly an
object in its own right. While it only makes sense to have one extender of
a given type for a given base object, you may have as many instances of
an independent adapter as you like for the same base object. For example,
Python iterators are independent adapters, as are views in a
model-view-controller framework, since each iterable may have many iterators
in existence, each with its own independent state. Resuming the previous
analogy of a power outlet, you may consider independent adapters to be like
appliances: you can plug more than one lamp into the same outlet, and different
lamps may be on or off at a given point in time. Many appliances may come and
go over the lifetime of the power outlet -- there is no inherent connection
between them because the appliances are independent objects rather than mere
extensions of the power outlet.
A key distinction between extenders and independent adapters is the "as a"
relationship versus the "has a" relationship. An iterable "has" iterators
and a model "has" views. But an extender represents an "as a" relationship,
like treating a Person "as an" Employee, or treating a string "as a" filename.
For example, Jason Orendorff's ``path`` module offers a string subclass for
representing file or directory paths, supporting operations like ``rename()``
and ``walkfiles()``. Using this class as an extender would allow routines to
declare that they want a ``path`` argument, yet allow the caller to pass in a
string. The routine would then be able to call ``path`` methods on the object
it receives. Yet, if the routine in turn passed that object to another routine
that needs a string, the second routine would receive the original string.
In some ways, this approach can actually improve on subclassing. The ``path``
module's string subclass inherits many string methods that have no meaning
for path objects, and it also has a different meaning for __iter__ than a
string does. While iterating over a string yields the characters in the
string, iterating over a path yields the files within the directory specified
by the path. Thus, a path object is not really substitutable for a string, and
today passing a path object to a routine that expects to be able to iterate
over the string's characters would break.
However, if it were implemented as an extender, the path type could supply only
methods that make sense for a path, and any given routine can choose to treat
the object "as" either a string or a path, according to its need.
2. Make it possible to define volatile adapters, but only by explicitly
declaring them as such in the adapter's definition.
3. Make it possible to have a type declaration result in creation of a
volatile adapter, but only by explicitly declaring in the adapter's
defintion that type declarations are allowed to implicitly create
instances.
By doing this, the language can gently steer developers away from
unintentionally creating adapters whose implicit behavior is difficult to
understand, or is not as they intended, by making it easier to do safer
forms of adaptation, and suggesting (via declaration requirements) that other
forms may need a bit more thought to use correctly.
PEP 246 was originally proposed for an explicit adaptation model where an
``adapt()`` function is called to retrieve an "adapter", where no distinction
was made between extenders and independent adapters. However, the adapting
code in this model always has access to the "original" object, so the
distinction didn't matter: you could always create new extenders or independent
adapters from the original object, so you had complete control. Also, PEP 246
permitted either the caller of a function or the called function to perform the
adaptation, so the scope and lifetime of the resulting adapter could be
explicitly controlled in a straightforward way.
By contrast, the type declaration syntax Guido proposed would perform
adaptation at the boundary between caller and callee, making it difficult for
the caller to control an independent adapter's lifetime, or for the callee to
obtain the "original" object in order to access a different extender or create
a new independent adapter.
The discussion that followed these matters also made it clear that although
PEP 246 provides excellent support for creating independent adapters, it offers
few conveniences for creating extenders, and it is extenders that are the
primary domain of type declarations. Guido, however, has also said [1]_:
I don't believe for a second that all [independent] adapters
are bad [in the context of type declarations], even though I
expect that [extenders] are always good.
Therefore, this PEP proposes additional support for creating extenders, in
addition to PEP 246's support for creating independent adapters. Also, we
propose that type declarations be limited to extenders by default, but allowing
independent adapters upon request. That is, we propose that independent
adapters registered for use with ``adapt()`` be required to specify an
additional option or make an additional API call to declare that they are
safe for use with type declarations, since it is not always appropriate to
create new independent objects just because a function or method call has
occurred.
Currently, it is very easy to write good independent adapters in Python,
because as independent objects it suffices to write the class with the
desired functionality. But it is much harder to write good extenders, because
their state needs to be "sticky", in the sense that it is attached to the
extended object.
Also extenders written as adapter classes are not *composable*. If two
interfaces have overlapping functionality, it's often necessary to create
separate adapter classes for each interface. Conversely, if two adapter
classes are written to support different interfaces, they cannot be
automatically combined to form a single extender for an interface that
includes the operations of the two original interfaces.
This PEP therefore focuses on describing an extension to PEP 246 that
automatically creates extender classes by combining simple declarations
in Python code. These declarations are based on defining extenders for
*operations*, rather than for interfaces as a whole. It is then possible
to automatically recombine operations to create an extender for an interface
whose operations are known. The result is an easy, intuitive way to create
extenders that "just work" without a lot of attention to the mechanics.
Adapter Composition
such as when attempting to compare implicitly adapted objects or use them as
dictionary keys.)
Therefore, this proposal seeks to have adaptation performed via type
declarations avoid implicit adapter composition, by never adapting an
operational or extension adapter. Instead, the original object will be
retrieved from the adapter, and then adapted to the new target interface.
Volatile adapters, however, are independent objects from the object they
adapt, so they must always be considered an "original object" in their own
right. (So, volatile adapters are also more volatile than other adapters with
respect to transitive adaptation.) However, since volatile adapters
must be declared as such, and require an additional declaration to allow them
to be implicitly created, the developer at least has some warning that their
behavior will be more difficult to predict in the presence of type
declarations.
However, these problems are a direct consequence of not distinguishing between
extenders and independent adapters. An extender should not be re-adapted;
instead, the original object should be retrieved from the extender and
re-adapted. PEP 246 is currently undergoing changes to allow supporting
this behavior.
Interfaces vs. Duck Typing
return value(s).
The problem with this concept is that interface implementations are typically
expected to be complete. In Java, for example, you say that your class
expected to be complete. In Java, for example, you can't say that your class
implements an interface unless you actually add all of the required methods,
even if some of them aren't needed in your program yet.
concepts in other languages with more sophisticated type systems (like Haskell
typeclasses or Dylan protocols), while still being a straightforward extension
of more rigid type systems like those of Java or Microsoft's COM (Component
Object MOdel).
Object Model).
This PEP directly competes with PEP 245, which proposes a syntax for
Python interfaces. If some form of this proposal is accepted, it
would be unnecessary for a special interface type or syntax to be added to
Python, since normal classes and partially or completely abstract classes will
be routinely usable as interfaces. Some packages or frameworks, of course, may
have additional requirements for interface features, but they can use
metaclasses to implement such enhanced interfaces without impeding their
ability to be used as interfaces by this PEP's adaptation system.
Specification
form the basis for compatible interfacing between packages; if each package
denotes the relationship between its types' operations and the operations
of the ``file`` type, then those packages can accept other packages' objects
as parameters declared as requiring a ``file`` instance.
as ``:file`` parameters.
However, the standard library cannot contain base versions of all possible
operations for which multiple implementations might exist, so different
# walk like a duck!
This is an example of declaring the similarity *inside* the class to be
adapted. In many cases, however, you can't do this because you don't control
extended. In many cases, however, you can't do this because you don't control
the implementation of the class you want to use, or even if you do, you don't
wish to introduce a dependency on the foreign package.
Whichever way the operation correspondence is registered, we should now be
able to successfully call ``walkTheDuck(Mallard())``. Python will then
automatically create a "proxy" or "adapter" object that wraps the ``Mallard``
instance with a ``Duck``-like interface. That adapter will have a ``walk()``
automatically create an extender object that wraps the ``Mallard``
instance with a ``Duck``-like interface. That extender will have a ``walk()``
method that is just a renamed version of the ``Mallard`` instance's
``waddle()`` method (or of the ``duckwalk_by_waddling`` external operation).
For any methods of ``Duck`` that have no corresponding ``Mallard`` operation,
the adapter will omit that attribute, thereby maintaining backward
the extender will omit that attribute, thereby maintaining backward
compatibility with code that uses attribute introspection or traps
``AttributeError`` to control optional behaviors. In other words, if we have
a ``MuteMallard`` class that has no ability to ``quack()``, but has an
tries to make it ``quack``, that routine will get an ``AttributeError``.
Adapter Creation
----------------
Extender Creation
-----------------
Note, however, that even though a different adapter class is needed for
different adapted types, it is not necessary to create an adapter class "from
Note, however, that even though a different extender class is needed for
different source types, it is not necessary to create an extender class "from
scratch" every time a ``Mallard`` is used as a ``Duck``. Instead, the
implementation can need only create a ``MallardAsDuck`` adapter class once, and
then cache it for repeated uses. Adapter instances can also be quite small in
implementation need only create a ``MallardAsDuck`` extender class once, and
then cache it for repeated uses. Extender instances can also be quite small in
size, because in the general case they only need to contain a reference to the
object instance that they are adapting. (Except for "extension" adapters,
which need storage for their added "state" attributes. More on this later,
in the section on `Adapters That Extend`_, below.)
object instance that they are extending.
In order to be able to create these adapter classes, we need to be able to
In order to be able to create these extender classes, we need to be able to
determine the correspondence between the target ``Duck`` operations, and
operations for a ``Mallard``. This is done by traversing the ``Duck``
operation namespace, and retrieving methods and attribute descriptors. These
descriptors are then looked up in a registry keyed by descriptor (method or
property) and source type (``Mallard``). The found operation is then placed
in the adapter class' namespace under the name given to it by the ``Duck``
in the extender class' namespace under the name given to it by the ``Duck``
type.
So, as we go through the ``Duck`` methods, we find a ``walk()`` method
subclass of ``Duck`` can reuse operations declared "like" that operation.)
If we find the entry, ``duckwalk_by_waddling`` (the function object, not its
name), then we simply place that object in the adapter class' dictionary under
name), then we simply place that object in the extender class' dictionary under
the name ``"walk"``, wrapped in a descriptor that substitutes the original
object as the method's ``self`` parameter. Thus, when the function is invoked
via an adapter instance's ``walk()`` method, it will receive the adapted
via an extender instance's ``walk()`` method, it will receive the extended
``Mallard`` as its ``self``, and thus be able to call the ``waddle()``
operation.
of the ``Mallard`` class, then the ``@like`` decorator will register the method
name ``"waddle"`` as the operation in the registry. So, we would then look up
that name on the source type in order to implement the operation on the
adapter. For the ``Mallard`` class, this doesn't make any difference, but if
we were adapting a subclass of ``Mallard`` this would allow us to pick up the
extender. For the ``Mallard`` class, this doesn't make any difference, but if
we were extending a subclass of ``Mallard`` this would allow us to pick up the
subclass' implementation of ``waddle()`` instead.
So, we have our ``walk()`` method, so now let's add a ``quack()`` method.
the ``__mro__`` (method resolution order) of ``Mallard`` in order to see if
there is an operation corresponding to ``quack`` that ``Mallard`` inherited
from one of its base classes. If no method is found, we simply do not put
anything in the adapter class for a ``"quack"`` method, which will cause
anything in the extender class for a ``"quack"`` method, which will cause
an ``AttributeError`` if somebody tries to call it.
Finally, if our attempt at creating an adapter winds up having *no* operations
Finally, if our attempt at creating an extender winds up having *no* operations
specific to the ``Duck`` type, then a ``TypeError`` is raised. Thus if we had
passed an instance of ``Pig`` to the ``walkTheDuck`` function, and ``Pig``
had no methods corresponding to any ``Duck`` methods, this would result in
Thus, if a given type doesn't have a more specific implementation of
``dict.update``, then types that implement a ``dict.__setitem__`` method can
automatically have this ``update()`` method added to their ``dict`` adapter
class. While building the adapter class, we simply keep track of the needed
automatically have this ``update()`` method added to their ``dict`` extender
class. While building the extender class, we simply keep track of the needed
operations, and remove any operations with unmet or circular dependencies.
By the way, even though technically the ``needs`` argument to ``@like`` could
result will either be an ``AttributeError`` at a deeper point in the code, or
a stack overflow exception caused by looping between mutually recursive
operations. (E.g. if an external ``dict.__setitem__`` is defined in terms of
``dict.update``, and a particular adapted type supports neither operation
``dict.update``, and a particular extended type supports neither operation
directly.) Neither of these ways of revealing the error is particularly
problematic, and is easily fixed when discovered, so ``needs`` is still
intended more for the reader of the code than for the adaptation system.
intended more for the reader of the code than for the extender creation system.
By the way, if we look again at one of our earliest examples, where we
externally declared a method correspondence from ``Mallard.waddle`` to
When you register an external operation, the actual function object given is
registered, because the operation doesn't correspond to a method on the
adapted type. In contrast, "internal operations" declared within the adapted
extended type. In contrast, "internal operations" declared within the extended
type cause the method *name* to be registered, so that subclasses can inherit
the "likeness" of the base class' methods.
Adapters That Extend
Extenders with State
--------------------
One big difference between external operations and ones created within a
mangled attribute names to avoid collisions with other external
operations' attributes.
So let's look at an example of how to handle adaptation that needs more
state information than is available in the adapted object. Suppose, for
So let's look at an example of how to handle extenders that need more
state information than is available in the extended object. Suppose, for
example, we have a new ``DuckDodgers`` class, representing a duck who is
also a test pilot. He can therefore be used as a rocket-powered vehicle by
strapping on a ``JetPack``, which we can have happen automatically::
during flight, which would then be confused about how much fuel it had or
whether it was currently in flight!
This pattern of adaptation is referred to in the `Motivation`_ section as
"extension" or "extender" adaptation, because it allows you to dynamically
extend the capabilities of an existing class at runtime, as opposed to just
recasting its existing operations in a form that's compatible with another
type. In this case, the ``JetPack`` is the extension, and our ``launch``
method defines part of the adapter.
Note, by the way that ``JetPack`` is a completely independent class here. It
does not have to know anything about ``DuckDodgers`` or its use as an adapter,
does not have to know anything about ``DuckDodgers`` or its use in an extender,
nor does ``DuckDodgers`` need to know about ``JetPack``. In fact, neither
object should be given a reference to the other, or this will create a
circularity that may be difficult to garbage collect. Python's adaptation
machinery will use a weak-key dictionary mapping from adapted objects to their
circularity that may be difficult to garbage collect. Python's extender
machinery will use a weak-key dictionary mapping from extended objects to their
"extensions", so that our ``JetPack`` instance will hang around until the
associated ``DuckDodgers`` instance goes away.
``DuckDodgers`` instance, and then the operation is invoked with references
to both objects.
Of course, this mechanism is not available for adapting types whose instances
Of course, this mechanism is not available for extending types whose instances
cannot be weak-referenced, such as strings and integers. If you need to extend
such a type, you must fall back to using a volatile adapter, even if you would
prefer to have a state that remains consistent across adaptations. (See the
`Volatile Adaptation`_ section below.)
such a type, you must fall back to either storing the additional state in the
object itself, using the object to key some other dictionary to obtain the
state, or declaring that your extender can live with potentially inconsistent
states.
XXX have a way to declare that state is kept in the extender for this scenario
Using Multiple Extenders
------------------------
Each external operation can use a different ``using`` type to store its state.
For example, a ``DuckDodgers`` instance might be able to be used as a
``Soldier``, provided that he has a ``RayGun``::
Using Multiple Extender States
------------------------------
Different external operations can use different ``using`` types to store
their state. For example, a ``DuckDodgers`` instance might be able to be used
as a ``Soldier``, provided that he has a ``RayGun``::
@like(Soldier.fight, for_type=DuckDodgers, using=RayGun)
def fight(raygun, self, enemy:Martian):
``using`` types with a common base class (other than ``object``), the
most-derived type is used for both operations. This rule ensures that
extenders do not end up with more than one copy of the same state, divided
between a base type and a derived type.
between a base type and a derived type. This is important because extenders
are really a form of inheritance, in that they extend the type of the object
they extend, so we don't want to end up with multiple objects representing the
same thing.
Notice that our examples of ``using=JetPack`` and ``using=RayGun`` do not
interact, as long as ``RayGun`` and ``JetPack`` do not share a common base
it in at least one of the operations.
If it is not possible to determine a single "most-derived" type among a set of
operations for a given adapted type, then an error is raised, similar to that
operations for a given extended type, then an error is raised, similar to that
raised by when deriving a class from classes with incompatible metaclasses.
As with that kind of error, this error can be resolved just by adding another
``using`` type that inherits from the conflicting types.
Volatile Adaptation
-------------------
Volatile adapters are not the same thing as operational adapters or extenders.
Indeed, some strongly question whether they should be called "adapters" at all,
because to do so weakens the term. For example, in the model-view-controller
pattern, does it make sense to call a view an "adapter"? What about iterators?
Are they "adapters", too? At some point, one is reduced to calling any object
an adapter, as long as it mainly performs operations on one other object. This
seems like a questionable practice, and it's a much broader term than is used
in the context of the GoF "Adapter Pattern" [3]_.
Indeed, it could be argued that these other "adapters" are actually extensions
of the GoF "Abstract Factory" pattern [4]_. An Abstract Factory is a way
of creating an object whose interface is known, but whose concrete type is not.
PEP 246 adaptation can basically be viewed as an all-purpose Abstract Factory
that takes a source object and a destination interface. This is a valuable
tool for many purposes, but it is not really the same thing as adaptation.
Shortly after I began writing this section, Clark Evans posted a request for
feedback on changes to PEP 246, that suggests PEP 246 will provide adequate
solutions of its own for defining volatile adapters, including options for
declaring an adapter volatile, and whether it is safe for use with type
declarations. So, for now, this PEP will assume that volatile adapters will
fall strictly under the jurisdiction of PEP 246, leaving this PEP to deal
only with the previously-covered styles of adaptation that are by definition
safe for use with type declarations. (Because they only cast an object in
a different role, rather than creating an independent object.)
Non-Method Attributes
---------------------
Miscellaneous
-------------
Sometimes, an interface includes not only methods, but also the ability to get
or set an attribute as well. In the case of attributes that are managed by
descriptors in the interface or type, we can use these for ``@like``
declarations by mapping to their ``__get__``, ``__set__``, and ``__delete__``
methods, e.g.::
@like(SomeClass.someAttr.__get__, for_type=OtherClass)
def get_foo_as_someAttr(self):
return self.foo
@like(SomeClass.someAttr.__set__, for_type=OtherClass)
def set_foo_as_someAttr(self, value):
self.foo = value
While creating an extender to map ``OtherClass`` to ``SomeClass``, we will
find the ``someAttr`` descriptor and check for operations defined for its
``__get__``, ``__set__`` and ``__delete__`` methods, using them to assemble
a ``property`` descriptor for the extender. In addition to using functions
as shown, we can also implement a shortcut like this::
like(SomeClass.someAttr, for_type=OtherClass)("foo")
To mean that any set/get/delete of ``someAttr`` on the extender should be
mapped to a corresponding action on the ``foo`` attribute of the ``OtherClass``
instance. Or, we can use this::
like(SomeClass.someAttr, for_type=OtherClass, using=FooData)("foo")
to mean that the ``foo`` attribute should be gotten, set, or deleted from
the ``FooData`` state instance, whenever the corresponding operation is
performed on the extender's ``someAttr``.
XXX property get/set/del as three "operations"
Special Methods
---------------
XXX binary operators
Backward Compatibility
======================
XXX explain Java cast and COM QueryInterface as proper subsets of adaptation
XXX explain Java cast and COM QueryInterface as proper subsets of extender
concept
Reference Implementation
.. [2] Optional Static Typing -- Stop the Flames!
(http://www.artima.com/weblogs/viewpost.jsp?thread=87182)
.. [3] XXX Adapter Pattern
.. [4] XXX Abstract Factory Pattern
Copyright
