[PEAK] PEAK-Rules for Python3

[Cara]

Despite what I said last time, I did a little bit more on this.

> Those results are incorrect; in fact there are only 3 shallow failures
> for Python 2.7 across all the projects you listed.  The correct way to
> run the full tests is with "setup.py test".  That will run both the
> doctests and the unittests for each project, and with the correct
> options for each file/project.
> 

Mea culpa.  I know next to nothing about setuptools and didn't
know that.  I was hoping to look at how broken the functionality is in
Python 3 without having to deal with the packaging, but since apparently
the only way to run the tests correctly is to fix the packaging, I had
to do that first.  This was rather involved even not counting all the
dead ends I went down.  After setting up a virtualenv with Python 2 and
the packages from PyPi, I had to figure out that setuptools no longer
distributes ez_setup and that I needed the old version in the
repository, get it, and symlink in each of the individual package
directories.  After that, I reran the conversion scripts, reapplied my
previous edits, and installed into a virtualenv successfully.

> The only projects with failures on 2.7 are PEAK-Rules and
> DecoratorTools, and in each case it's due to subtle repr differences
> or internal refactorings in 2.7.  Essentially, they work just fine on
> 2.7

This is also what I found.  Note that trying to run the tests on 2.7
will fail because of the missing ez_setup.
 
> Yes, and that's due to a fundamental limitation in Python 3, which
> doesn't have any way to select a metaclass while within a class
> definition suite, except by an even nastier global monkeypatching than
> the way decoratortools works in general.  This limitation is one of
> the many reasons I probably won't be trying to port my projects to
> Python 3.  I previously raised the issue on Python-dev (see e.g.
> https://mail.python.org/pipermail/python-dev/2012-April/118952.html ),
> and there is a PEP (#422) that would address the issue, complete with
> a patch ( http://bugs.python.org/issue17044 ), but which has been
> languishing for a couple of years.  A couple months ago, there was a
> new proposal ( http://code.activestate.com/lists/python-ideas/32123/
> ), which could probably also be made to work for most uses, but of
> course is also not implemented in any version of Python 3.

I'm aware of the problem and the PEPs that try to solve it, but I didn't
know it was the issue here.  I'd like to see the functionality added,
but CPython development is CPython development.  I can throw in a word
on python-dev if you want, for all that would do.
 
> > 1) Breaking backwards compatibility: Python has had, for instance,
> > class decorators since 2.6, which isn't even supported any more.  I
> > don't have any interest in rewriting the frame-hacking for
> > DecoratorTools.decorator_class() to make it work on Python 3, so I'd
> > just refactor all the relevant code to use the built-in class
> > decorators.
> 
> This isn't actually possible, anyway: you would not be able to define
> generic function methods in a class body unless you added an explicit
> metaclass or class decorator to the class, and rewrote the API or
> dropped the feature.  (I guess I should say, it isn't possible while
> retaining the features of class-level AddOns or of defining gf methods
> in class bodies.)

I'm not sure I understand why this is.
 
> If I were to drop compatibility with Python <2.6, BytecodeAssembler
> and the most of the other ast stuff could be mostly replaced with the
> Python 2.6+ "ast" module and the compile() builtin.  Such a thing
> would be vastly easier to make work with 3.x, I imagine.  In practice,
> I don't have time or interest at the moment sufficient to do it
> myself.  Such a thing might be better off as a fork, anyway.

I've gotten the Python 3 port to the point where everything installs and
most of the tests work.  Extremes, SymbolType, and Importing pass all
their tests in both 2 and 3.  There's an AmbiguousMethods error in
PEAK-Rules,

("peak.rules.core.AmbiguousMethods: ([Method(<built-in function
issubclass>, (<type 'type'>, <type 'type'>), 2, None), Method(<built-in
function issubclass>, (<type 'type'>, <type 'type'>), 3, None)], (<class
'peak.rules.core.TypeEngine'>, <class 'peak.rules.core.TypeEngine'>),
{})"

which I think comes from this section of code that I don't
understand in core.py:

ClassType = type
when(implies, (type,      (ClassType, type) ))(issubclass)
when(implies, (ClassType,  ClassType        ))(issubclass)
when(implies, (istype,     istype           ))(lambda s1,s2:
    s1==s2 or (s1.type is not s2.type and s1.match and not s2.match))
when(implies, (istype, type) )(lambda s1,s2: s1.match and
    implies(s1.type,s2))

I added the ClassType = type assignment to bypass shallow NameErrors.

There's a collection of errors in Python 3 for the AddOns and
DecoratorTools tests that I suspect relate to the class creation and
decorators issued you mentioned: the tests involving Demo in the doc
tests and TypeErrors in testMixedMetas, testSingleExplicitMeta, and
testOrder, plus some shallow errors.  From what you said, the changes to
metaclass handling make any kind of a direct port impossible because the
functionality isn't there on Python 3?

There's one error in testAlreadyTracing that appears on both Python 2
and 3:

Traceback (most recent call last):
  File "/home/cara/code/peak-rules/DecoratorTools/test_decorators.py",
line 116, in testAlreadyTracing
    'decorate_assignment', 'enclosing_frame', '<lambda>',
'failUnlessEqual',
AssertionError: Lists differ: ['tes[66 chars]>', 'assertEqual',
'_getAssertEqualityFunc', '[363 chars]ual'] != ['tes[66 chars]>',
'failUnlessEqual', 'decorate_assignment', [171 chars]ual']

First differing element 4:
assertEqual
failUnlessEqual

First list contains 9 additional elements.
First extra element 16:
...

And then there's a large collection of errors in BytecodeAssembler on
Python 2 I haven't assessed, plus it segfaults (no surprise there) on
Python 3.  I posted the log for Python 3 here:
http://pastebin.com/UhCfyHJf .  The log for Python 2 is too large to put
up anywhere.

Cara