Parsing Functions

The basic idea of the options framework is that you have an object whose attributes will store the options parsed from a command line. The options that are available, depend on what options you declared when defining the object's class, or its base class(es).

All of the options API functions assume that you have already created the object you intend to use to store options. (This works well with the common case of a peak.running.commands command object that is calling the parsing API from within its run() method.)

These functions also accept keyword arguments that are passed directly to the optparse.OptionParser constructor. So, if you need precise control over some optparse feature, you can supply keyword arguments to do so. (Most often, these arguments will be used to set the usage, description, and prog values used for creating help messages. See the section on Parser Settings for details.)

options.make_parser (ob, **kw)

Make an optparse.OptionParser for ob, populating it with the options registered for ob.__class__.

options.parse (ob, args, **kw)

Parse args, setting any options on ob, and returning a list of the non-option arguments in args. args should be a list of argument values, such as one might find in sys.argv[1:].

options.get_help (ob, **kw)

Return a formatted help message for ob, explaining the options registered for ob.__class__.

Declaring Options

Most of the time, you will want command-line options to set or modify some attribute of your command object. So, most options are specified as attribute metadata, e.g. via the metadata argument of an attribute binding, or through attribute metadata APIs like binding.metadata() or binding.declareAttributes(). For simplicity in this document, we'll be mostly using binding.metadata(), but you can also specify options like this:

class MyClass(commands.AbstractCommand):

    dbURL = binding.Obtain(
        PropertyName('myapp.dburl'),
        [options.Set('--db', type=str, metavar="URL", help="Database URL")]
    )

(That is, by including option declarations as metadata in an attribute binding.)

Anyway, there are three kinds of options that can associate with attributes:

• options.Set()
• options.Add()
• options.Append()

Each kind of option performs the appropriate action on the associated attribute. That is, a Set() option sets the attribute to some value, while an Add() or Append() option adds to or appends to the attribute's initial value.

Let's take a look at some usage examples:

>>> opt_x = options.Set('-x', value=True, help="Set 'x' to true")
>>> opt_v = options.Set('-v', '--verbose', value=True, help="Be verbose")
>>> opt_q = options.Set('-q', '--quiet', value=False, help="Be quiet")
>>> opt_f = options.Set('-f', '--file', type=str, metavar="FILENAME")
>>> opt_p = options.Set('-p', type=int, metavar="PORT", help="Set port")
>>> opt_L = options.Append('-L', type=str, metavar="LIBPATH", sortKey=99)
>>> opt_d = options.Add('-d', type=int, help="Add debug flag")

All of these option constructors take the same arguments; one or more option names, followed by some combination of these keyword arguments:

type

A callable that can convert from a string to the desired option type. If supplied, this means that the option takes an argument, and the value to be set, added, or appended to the attribute will be computed by calling supplied_type(argument_string) when the option appears in a command line. If the callable raises a ValueError, the error will be converted to an InvocationError saying that the value isn't valid. All other errors propagate to the caller.

value

If supplied, this means that the option does not take an argument, and the supplied value will be set, added, or appended to the attribute when the option appears in a command line.

help

A short string describing the option's purpose, for use in generating a usage message.

metavar

A short string used to describe the argument taken by the attribute. For example, a metavar of "FILENAME" might produce a help string like "-f FILENAME   Set the filename" (assuming the option name is -f and the help string is "Set the filename"). Note that metavar is only meaningful when type is specified. If no metavar is supplied, it defaults to an uppercase version of the type object's __name__, such that type=int defaults to a metavar of "INT":

>>> opt_p.metavar
'PORT'
>>> opt_d.metavar
'INT'

repeatable

A true/false flag indicating whether the option may appear more than once on the command line. Defaults to True for Add and Append options, and False for Set options and option_handler methods:

>>> opt_d.repeatable
1
>>> opt_L.repeatable
1
>>> opt_q.repeatable
0

sortKey

The sort key is a value used to arrange options in a specified order for help messages. Options with lower sortKey values appear earlier in generated help than options with higher sortKey values. The default sortKey is 0:

>>> opt_x.sortKey
0
>>> opt_L.sortKey
99

Options that have the same sortKey will be sorted in the order in which they were created, so you don't ordinarily need to set this. (Except to insert new options in the display order, ahead of previously-defined options.)

group

Set this to an options.Group() instance, if you want the option to appear under that group's heading in any generated help messages. (See the section on Option Groups for more details.)

Note that an option must have either a type (in which case it accepts an argument), or a value (in which case it does not accept an argument). It must have one or the other, not both.

Note also that more than one option can be specified for a given attribute, although in that case they will usually all be Set(value=someval) options. For example, the For example, the -v and -q options shown above would most likely be used with the same attribute, e.g.:

>>> class Foo:
...     binding.metadata(verbose = [opt_v, opt_q])
...     verbose = False

For the above class, -q will set verbose to False, and -v will set it to True.

>>> class Foo:
...     [options.option_handler('-z', type=int, help="Zapify!")]
...     def zapify(self, parser, optname, optval, remaining_args):
...         """Do something here"""

>>> class Foo:
...     binding.metadata(verbose = [opt_v, opt_q])
...     verbose = False

>>> print options.get_help(Foo())
options:
  -v, --verbose  Be verbose
  -q, --quiet    Be quiet

>>> class Bar(Foo):
...     binding.metadata(libs = opt_L, debug=opt_d)
...
...     [options.option_handler('-z', type=int, help="Zapify!")]
...     def zapify(self, parser, optname, optval, remaining_args):
...         print "Zap!", optval

>>> print options.get_help(Bar())   # doctest: +NORMALIZE_WHITESPACE
options:
  -v, --verbose  Be verbose
  -q, --quiet    Be quiet
  -d INT         Add debug flag
  -z INT         Zapify!
  -L LIBPATH

# Even though it was defined after -d, -L is last because its sortKey is 99

>>> class Baz(Foo):
...     options.reject_inheritance('--quiet','-v')

>>> print options.get_help(Baz())
options:
  --verbose  Be verbose
  -q         Be quiet

>>> class Spam(Foo):
...     options.reject_inheritance()

>>> options.get_help(Spam())
''

Parsing Examples

Let's go ahead and parse some arguments, using options.parse(). This API function takes a target object and a list of input arguments, returning a list of the non-option arguments. Meanwhile, the target object's attributes are modified (or its handler methods are called) according to the options found in the input arguments.

• No options or arguments:

  >>> foo = Foo(); options.parse(foo, [])
  []
  >>> foo.verbose
  0
  

• An option and an argument:

  >>> foo = Foo(); options.parse(foo, ['-v', 'q'])
  ['q']
  >>> foo.verbose
  1
  

• Stop processing options after first argument:

  >>> foo = Foo(); options.parse(foo, ['xyz', '-v'])
  ['xyz', '-v']
  >>> foo.verbose
  0
  

• Unless interspersed arguments are allowed (see Parser Settings below):

  >>> foo = Foo()
  >>> options.parse(foo, ['xyz', '-v'], allow_interspersed_args=True)
  ['xyz']
  >>> foo.verbose
  1
  

• Two options:

  >>> foo = Foo(); options.parse(foo, ['-v', '-q'])
  []
  >>> foo.verbose
  0
  

• Repeating unrepeatable options:

  >>> foo = Foo(); options.parse(foo, ['-v', '-q', '-v'])
  Traceback (most recent call last):
  ...
  InvocationError: -v/--verbose can only be used once
  
  >>> bar = Bar(); options.parse(bar, ['-z','20', '-z', '99'])
  Traceback (most recent call last):
  ...
  InvocationError: -z can only be used once
  

• Using an invalid value for the given type converter:

  >>> bar = Bar(); options.parse(bar, ['-z','foobly'])
  Traceback (most recent call last):
  ...
  InvocationError: -z: 'foobly' is not a valid INT
  

• Option handler called in the middle of parsing:

  >>> bar = Bar(); options.parse(bar, ['-z','20', '-v', 'xyz'])
  Zap! 20
  ['xyz']
  >>> bar.verbose
  1
  

• Append option with multiple values, specified in different ways:

  >>> bar = Bar(); bar.libs = []
  >>> options.parse(bar, ['-Labc','-L', 'xyz', '123'])
  ['123']
  >>> bar.libs
  ['abc', 'xyz']
  

• Add option with multiple values, specified in different ways:

  >>> bar = Bar(); bar.debug = 0
  >>> options.parse(bar, ['-d23','-d', '32', '321'])
  ['321']
  >>> bar.debug
  55
  

• Unrecognized option:

  >>> foo = Foo()
  >>> options.parse(foo, ['--help'])
  Traceback (most recent call last):
  ...
  InvocationError: ... no such option: --help
  

Help/Usage Messages

By default, PEAK doesn't include a --help option in the options for an arbitrary class, so if you want one, you have to create your own. (Unless you're using a commands framework base class, in which case it may be provided for you.) Here's one way to implement such an option:

>>> class Test(Bar):
...     options.reject_inheritance('-L', '-d')
...     [options.option_handler('--help',value=None,help="Show help")]
...     def show_help(self, parser, optname, optval, remaining_args):
...         print parser.format_help().strip()
>>> test = Test()
>>> args = options.parse(test, ['--help'])
options:
  -v, --verbose  Be verbose
  -q, --quiet    Be quiet
  -z INT         Zapify!
  --help         Show help

(Of course, for command objects, the help should actually be sent to the command's standard out or standard error, rather than to sys.stdout as is done in this example.)

>>> args = options.parse(test, ['--help'],
...     usage="%prog [options]", description="Just a test program.",
...     prog="Test",
... )
usage: Test [options]
...
Just a test program.
...
options:
  -v, --verbose  Be verbose
  -q, --quiet    Be quiet
  -z INT         Zapify!
  --help         Show help

>>> silly = options.Group(
...     "Silly Options",
...     "Forward aerial half turn every alternate step",
... )
>>> silly.title
'Silly Options'
>>> silly.description
'Forward aerial half turn every alternate step'
>>> silly.sortKey
0

>>> trendy = options.Group(title="Trendy Options", sortKey=10)
>>> trendy.title
'Trendy Options'
>>> trendy.sortKey
10
>>> trendy.description is None
1

>>> dangerous = options.Group("DANGER")
>>> opt_explode = options.Set('--explode', value="BOOM", help="Explode!",
...     group=dangerous)

>>> opt_r = options.Set('-r', '--request-grant', type=float,
...     help="Request grant from the Ministry", group = silly
... )
>>> opt_r.group
Group('Silly Options', 'Forward aerial half turn every alternate step', 0)

>>> class Groupy(Foo):
...     binding.metadata(money=opt_r, bang=opt_explode)
>>> print options.get_help(Groupy())
options:
  -v, --verbose         Be verbose
  -q, --quiet           Be quiet
...
  Silly Options:
    Forward aerial half turn every alternate step
...
    -r FLOAT, --request-grant=FLOAT
                        Request grant from the Ministry
...
  DANGER:
    --explode           Explode!

>>> class GroupsOnly:
...     binding.metadata(money=opt_r, bang=opt_explode)
>>> print options.get_help(GroupsOnly())
options:
  Silly Options:
    Forward aerial half turn every alternate step
...
    -r FLOAT, --request-grant=FLOAT
                        Request grant from the Ministry
...
  DANGER:
    --explode           Explode!

Using Options with Commands

Now let's take a look at how to integrate options with command objects from peak.running.commands. First, we'll define a command class with a few options:

>>> class MyCommand(commands.AbstractCommand):
...     bang = binding.Make(
...         lambda: NOT_GIVEN,
...         [opt_explode, opt_v, opt_q]    # bind options to this attribute
...     )

And let's see what its help output now looks like:

>>> import sys
>>> from peak.tests import testRoot
>>> exitcode = MyCommand(testRoot(),stderr=sys.stdout).showHelp()
Either this is an abstract command class, or somebody forgot to
define a usage message for their subclass.
...
options:
  -v, --verbose  Be verbose
  -q, --quiet    Be quiet
  --help         Show help
...
  DANGER:
    --explode    Explode!
...

As you can see, the option information is now part of the help output produced by the command. Also, we see that the --help option is automatically defined for us by commands.AbstractCommand. It will invoke the command's showHelp() method, so we can override how it actually works in our subclass if we need to.

To actually parse the options supplied to a command, we need only reference its parsed_args attribute. This will cause argument parsing to occur if it hasn't already, updating any attributes associated with the options, and returning the non-option arguments from the command line:

>>> cmd = MyCommand(testRoot(),argv=['foo','--explode','spam'])
>>> cmd.bang
NOT_GIVEN
>>> cmd.parsed_args
['spam']
>>> cmd.bang
'BOOM'

As you can see, the bang attribute had its default value until we looked at parsed_args, at which point it was set according to the supplied options. (Note: the "foo" in the argv parameter is the "program name", which is why it doesn't appear in cmd.parsed_args.)

Finally, you should be aware that both the help generation and option parsing are done via the option_parser attribute of the command, which is automatically populated using options.make_parser(self). You can therefore access it directly, or override the binding in a subclass to change how the parser gets set up.

>>> cmd.option_parser
<...OptionParser...>

So there you have it. Metadata-driven option parsing, seamlessly integrated with the peak.running.commands framework. The rest of this document deals strictly with implementation details, so you don't need or want to read it unless you're trying to track down a bug in the framework itself, or want to learn more about how it works internally.

options.AbstractOption

The AbstractOption class is a base class used to create command-line options. Instances are created by specifying a series of option names, and optional keyword arguments to control everything else. Here are some examples of correct AbstractOption usage:

>>> opt=options.AbstractOption('-x', value=42, sortKey=100)
>>> opt=options.AbstractOption('-y','--yodel',type=int,repeatable=False)
>>> opt=options.AbstractOption('--trashomatic',type=str,metavar="FILENAME")
>>> opt=options.AbstractOption('--foo', value=None, help="Foo the bar")

A valid option spec must have one or more option names, each of which begins with either '-' or '--'. It may have a type OR a value, but not both. It can also have a help message, and if the type is specified you may specify a metavar used in creating usage output. You may also specify whether the option is repeatable or not. If your input fails any of these conditions, an error will occur:

>>> options.AbstractOption(foo=42)
Traceback (most recent call last):
...
TypeError: ... constructor has no keyword argument foo
>>> options.AbstractOption()
Traceback (most recent call last):
...
TypeError: ... must have at least one option name
>>> options.AbstractOption('x')
Traceback (most recent call last):
...
ValueError: ... option names must begin with '-' or '--'
>>> options.AbstractOption('---x')
Traceback (most recent call last):
...
ValueError: ... option names must begin with '-' or '--'

>>> options.AbstractOption('-x', value=42, type=int)
Traceback (most recent call last):
...
TypeError: ... options must have a value or a type, not both or neither
>>> options.AbstractOption('-x', value=42, metavar="VALUE")
Traceback (most recent call last):
...
TypeError: 'metavar' is meaningless for options without a type

>>> key, xopt = opt_x.makeOption('foo')
>>> xopt.action
'callback'
>>> xopt.nargs
0
>>> xopt.callback == opt_x.callback
1
>>> xopt.callback_args
('foo',)
>>> xopt.metavar is None
1
>>> xopt.help is opt_x.help
1

>>> key, popt = opt_p.makeOption('bar')
>>> popt.nargs
1
>>> popt.callback == opt_p.callback
1
>>> popt.callback_args
('bar',)
>>> popt.metavar
'PORT'

>>> print opt_f.makeOption('baz')[1]
-f/--file
>>> print opt_f.makeOption('baz', {'-f':opt_f})[1]
-f
>>> print opt_f.makeOption('baz', {'--file':opt_f})[1]
--file
>>> print opt_f.makeOption('baz', {'--file':opt_f, '-f':opt_f})[1]
-f/--file

>>> print opt_f.makeOption('baz', {'--foo':opt_f})[1]
Traceback (most recent call last):
...
TypeError: at least one option string must be supplied

options.OptionRegistry

The options.OptionRegistry is a dispatcher that manages option metadata for any class that has command-line options. When an option is declared as metadata for an attribute, it is added to the registry. Here, we check that the 'method' passed to the registry's __setitem__ is correct:

>>> from peak.util.unittrace import History
>>> class Foo: pass
>>> option_x = options.AbstractOption('-x',value=True)
>>> h = History()
>>> h.trace(binding.declareAttribute, Foo, 'bar', option_x)
>>> h.calledOnce(options.OptionRegistry.__setitem__).args.method
(('-x', ('bar', <...AbstractOption...>)),)

The registered value for an option is a tuple of (optname,(attr,option)) tuple structures, one for each option name in the option object. This is then turned into more useful information by the option registry's method combiner:

>>> options.OptionRegistry[Foo(),]
{'-x': ('bar', <...AbstractOption...>)}

Although, it's probably easier to see the usefulness with a more elaborate example:

>>> class Foo:
...     binding.metadata(
...         bar = options.AbstractOption('-x','--exact',value=True),
...         baz = options.AbstractOption('-y','--yodeling',value=False),
...     )
>>> options.OptionRegistry[Foo(),]  # doctest: +NORMALIZE_WHITESPACE
{'--exact': ('bar', <...AbstractOption...>),
'-y': ('baz', <...AbstractOption instance...>),
'-x': ('bar', <...AbstractOption instance...>),
'--yodeling': ('baz', <...AbstractOption instance...>)}