Module Inheritance and Patching

The APIs defined here let you create modules which derive from other modules, by defining a module-level __bases__ attribute which lists the modules you wish to inherit from. For example:

        from peak.api import *

        import BaseModule1, BaseModule2

        __bases__ = BaseModule1, BaseModule2

        class MyClass:
            ...

        binding.setupModule()

The setupModule() call will convert the calling module, BaseModule1, and BaseModule2 into specially altered bytecode objects and execute them (in "method-resolution order") rewriting the calling module's dictionary in the process. The result is rather like normal class inheritance, except that classes (even nested classes) are merged by name, and metaclass constraints are inherited. So an inheriting module need not list all the classes from its base module in order to change them by altering a base class in that module.

Note: All the modules listed in __bases__ must either call setupModule() themselves, or be located in an on-disk .py, .pyc, or .pyo file. This is because PEAK cannot otherwise get access to their bytecode in a way that is compatible with the many "import hook" systems that exist for Python. (E.g. running bytecode from zip files or frozen into an executable, etc.) So if you are using such a code distribution technique, you must ensure that the base modules call setupModule(), even if they do not have a __bases__ setting or use any other PEAK code.

Function Rebinding

All functions inherited via "module inheritance" using setupModule() (including those which are instance or class methods) have their globals rebound to point to the inheriting module. This means that if a function or method references a global in the base module you're inheriting from, you can override that global in the inheriting module, without having to recode the function that referenced it. (This is especially useful for super() calls, which usually use global references to class names!)

In addition to rebinding general globals, functions which reference the global name __proceed__ are also specially rebound so that __proceed__ is the previous definition of that function, if any, in the inheritance list. (It is None if there is no previous definition.) This allows you to do the rough equivalent of a super() call (or AspectJ "around advice") without having to explicitly import the old version of a function. Note that __proceed__ is always either a function or None, so you must include self as a parameter when calling it from a method definition.

Pickling Instances of Nested Classes and the __name__ Attribute

One more bonus of using setupModule() is that instances of nested classes defined in modules using setupModule() will be pickleable. Ordinarily, nested class instances aren't pickleable because Python doesn't know how to find them, using only someClass.__name__ and someClass.__module__.

PEAK overcomes this problem by renaming nested classes so that they are defined with their full dotted name (e.g. Foo.Bar for class Bar nested in class Foo), and saving a reference to the class under its dotted name in the module dictionary. This means that someClass.__name__ may not be what you'd normally expect, and that doing del someClass may not delete all references to a class. But pickling and unpickling should work normally.

Note that some PEAK classes and metaclasses provide a "short form" of the class name for use when appropriate. For example, Feature classes have an attrName class attribute. In a pinch, you can also use __name__.split(".")[-1] to get the undotted form of a class' name.

Special Considerations for Mutables and Dynamic Initialization

Both inheritance and patching are implemented by running hacked, module-level code under a "simulator" that intercepts the setting of variables. This works great for static definitions like class and def statements, constant assignments, import, etc. It also works reasonably well for many other kinds of static initialization of immutable objects

Mutable values, however, may require special considerations. For example, if a module sets up some kind of registry as a module-level variable, and an inheriting module overrides the definition, things can get tricky. If the base module writes values into that registry as part of module initialization, those values will also be written into the registry defined by the derived module.

Another possible issue is if the base module performs other externally visible, non-idempotent operations, such as registering classes or functions in another module's registry, printing things to the console, etc. The simple workaround for all these considerations, however, is to move your dynamic initialization code to a module-level __init__ function.

Module-level __init__() Functions

The last thing setupModule() does before returning, is to check for a module-level __init__() function, and call it with no arguments, if it exists. This allows you to do any dynamic initialization operations (such as modifying or resetting global mutables) after inheritance has taken effect. As with any other function defined in the module, __proceed__ refers to the previous (i.e. preceding base module) definition of the function or None. This lets you can chain to your predecessors' initialization code, if needed/desired.

Note, by the way, that if you have an if __name__=="__main__" block in your module, it would probably be best if you move it inside the __init__() function, as this ensures that it will not be run repeatedly if you do not wish it to be. It will also allow other modules to inherit that code and wrap around it, if they so desire.

Package Inheritance

Packages (i.e. __init__ modules) can also set __bases__ and call setupModule(). Their package __path__ will be extended to include the __path__ contents of their __bases__, in an MRO-like order. This means that to derive a package from another, you do not need to create a separate inheriting module for every individual module or subpackage, only those that you wish to make modifications to. If package foo contains modules foo.bar and foo.baz, and you want your spam package to derive from foo, you need only create a spam/__init__.py that contains:

            import foo
            __bases__ = foo,

            # ...

            from peak.api import config
            config.setupModule()

At this point, spam.baz or spam.bar will automatically be importable, based on the foo versions of their code. This will work even if the foo versions don't call setupModule(), although in that case you won't be able to override their contents.

To override a module within the spam package, just create it, and use module inheritance to specify the base in the original package. For example, you can extend foo.bar by creating spam.bar as follows:

            import foo.bar
            __bases__ = foo.bar,

            # ...

            from peak.api import config
            config.setupModule()

Limitations of Package Inheritance

Because "package inheritance" is effectively just a __path__ hack, it is really only good for "single inheritance". Python will not automatically merge the modules or packages found on a package's __path__. So if you need multiple inheritance, you will need to create a module or subpackage for each module or subpackage that exists in more than one base package, and explicitly specify the right __bases__ for it. If a module or subpackage only appears in one base, however, and you have nothing to add to it, you can omit it from the inheriting package.

Using Relative Imports in Packages

If you want to extend a package, it's important to use only relative imports. This is because the code of a module is executed as-is in the derived module. If you do an absolute import, e.g. import foo.baz in package foo.bar, then spam.bar will still import foo.baz, not spam.baz. It's better to import baz for an item in the same package, or use peak.utils.imports.lazyModule like this:

            from peak.utils.imports import lazyModule
            baz = lazyModule(__name__, 'baz')

While this is more verbose in the simple case, it works for more complex relative paths than Python allows; for example you can do this:

            eggs = lazyModule(__name__, '../ni/eggs')

which isn't possible with a regular import statement.

Import Paths

To make using relative imports easier, __bases__ can include "/"-separated relative path strings instead of modules, e.g.:

            __bases__ = '../../foo/bar',

A / at the beginning of the path makes it absolute, so /foo/bar would also work.

To-do Items

• The simulator should issue warnings for a variety of questionable situations, such as...

  • Code matching the following pattern, which doesn't do what it looks like it does, and should probably be considered a "serious order disagreement":

                BaseModule:
    
                    class Foo: ...
    
                    class Bar: ...
    
                DerivedModule:
    
                    class Bar: ...
    
                    class Foo(Bar): ...
    

• This docstring is woefully inadequate to describe all the interesting subtleties of module inheritance; a tutorial is really needed. But there does need to be a reference-style explanation as well, that describes the precise semantics of interpretation for assignments, def, and class, in modules running under simulator control.

• Allow declareModule() to bootstrap non-existent modules; this might let us create "virtual packages" made by assembling other packages and modules.

• Need a strategy for handling "del" operations; they are currently untrapped. This might be okay under most circumstances, but need to consider edge cases.

• makeClass() should probably become part of the core API, where it can be used to resolve __metaclass__ conflicts during the first pass of importing a module (prior to running setupModule())

buildModule ( module,  code=None )

declareModule (
        name,
        relativePath=None,
        bases=(),
        patches=(),
        )

        from peak.api import *

        __bases__ = '../square', '../circle'

        config.declareModule(__name__, 'fill',
            bases = ('../../circle/fill',)  # relative to 'curvyBox.fill'
        )

        config.setupModule()

        import my_additions
        config.declareModule('third.party.module', patches=(my_additions,))

getCodeListForModule ( module,  code=None )

getLegacyCode ( module )

moduleBases ( module,  name='' )

patchModule ( moduleName,  _lvl=1 )

            from peak.api import config

            # ... body of module

            config.patchModule('moduleToPatch')

    'patchModule()' works much like 'setupModule()'.  The main difference
    is that it applies the current module as a patch to the supplied module
    name.  The module to be patched must not have been imported yet, and it
    must call 'setupModule()'.  The result will be as though the patched
    module had been replaced with a derived module, using the standard module
    inheritance rules to derive the new module.

    Note that more than one patching module may patch a single target module,
    in which case the order of importing is significant.  Patch modules
    imported later take precedence over those imported earlier.  (The target
    module must always be imported last.)

    Patch modules may patch other patch modules, but there is little point
    to doing this, since both patch modules will still have to be explicitly
    imported before their mutual target for the patches to take effect.

prepForSimulation (
        code,
        path='',
        depth=0,
        )

setupModule ( _lvl=1 )

setupObject ( obj,  **attrs )

toBases ( bases,  name='' )