[PEAK] Organizing documentation of Python code

[Ian Bicking]

Phillip J. Eby wrote:
> But source-reading tools are few and far between, because you have to be 
> able to write a parser or figure out how to use Python's internal parser 
> in order to write one.  Really, HappyDoc is the only practical 
> source-reading tool out there that I know of.  But even though it reads 
> the source, it *also* discards the ordering information, and doesn't 
> understand descriptors or interfaces or metaclasses.

Docutils has a source parser that's pretty decent, I worked on it some 
during the PyCon docutils sprint.  I don't know the current state of 
things, but there's been a lot of activity recently (probably a lot of 
it still focused on stand-alone document generation, but that's just 
because that's what people are using it for now).

Anyway, the parser had a couple features:

* It could read strings anywhere in the code (though it didn't pay 
attention to function bodies as a policy decision, I think).  So you 
could do:

class Whatever:
     """The owner attribute means blah blah"""
     owner = None

Actually, I think it put the string after the assignment, to match 
docstrings, though I think that's all backwards.  It captures both the 
variable being assigned to, and the expression.  It also captures 
strings that aren't attached to anything and aren't before an attribute 
assignment.  It also captured instance variable assignment in __init__.

It also keeps track of all the ordering of the module.  I used it at one 
point to do separators, like:

class Whatever:
     ...
     """These functions are only for use in subclasses"""
     def ...


Unfortunately it can't read comments at all.  But otherwise all the 
interesting information is there.  Right now it is creating a DOM, which 
is the docutils DOM plus some source-code-specific nodes.  It needs 
something to translate that DOM into a standard docutils DOM, which can 
then be rendered.  It also needs a lot of work and thought to deal with 
inter-module references, but it would still be useable without that. 
Oh, and subclassing -- that's a hard one too (being able to display a 
picture of a class's methods, including all superclass definitions). 
And, for PEAK, you'll probably want to be able to extend the whole thing 
for special PEAK constructs.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org