Python’s optparse for human beings

Table of contents

IntroductionBACK TO TOC

This article extends optparse‘s documentation. optparse is a Python’s module that allows your program to easily parse command line options it receives. In addition, it takes care of some of the very common tasks, such as handling -h command line option.

optparse is one of those modules that are an absolutely must have for almost every project. However, because we start new project so seldom, it is difficult to remember all those recipes that we come up with, every time we add a new command line option. This article is an attempt to bring together such recipes.

optparse‘s official documentation lacks some very important information. On the contrary to official documentation, in this document I am trying to give you more hands-on kind of documentation. This article is a cookbook of things that you do with optparse most often.

What do we do with command line optionsBACK TO TOC

Every time we use optparse, we use it to do a number of things. Obviously we want our program to support command line options. What varies is the type of options we want to support. Some options doesn’t have additional arguments (boolean options). Others are mandatory. Some require one or more arguments. Finally, we may want options to depend on each other – i.e. we may want one option to depend on presence of another.

These are most common things that we would like to implement, but there are more. Lets try to build a list of all such functionalities that we may want to implement.

  • Support for a standalone command line options (boolean options).
  • Support for mandatory (required) options.
  • Support for options with one or more arguments.
  • Default values for options.
  • Dependency between options.
  • Grouping of options in help screen.
  • Customizing help screen.

However, before we dig into implementation details of each and every one of these bullets, lets see the basics of using optparse.

optparse basicsBACK TO TOC

No matter what you do with optparse, it all starts with importing optparse and instantiating OptionParser class.

import optparse

parser = optparse.OptionParser()

Adding optionsBACK TO TOC

To add an option, we should use OptionParser‘s method called add_option(). It accepts a large number of parameters. For now, we will see only the most basic. We will see more advanced parameters later in this article. Note that we should call add_option() for every option that we would like our program to support.

Obviously the most important parameter tells what command line option we would like to support. Lets say that for sake of this article we would like our little Python script to support -n command line option. Also, we would like -n to have longer sibling –new. –new and -n would have the same meaning, but one is shorter and the other longer and more verbose.

This is how we add these options.

parser.add_option('-n', '--new')

Note that by all means this is not enough to be able to parse even simplest case of command line options. This is only the basics.

Specifying help string for an optionBACK TO TOC

One more thing that is very common when using optparse is to give a short help string that would tell user what this particular command line option does. optparse will print this string if user runs your program with -h.

To specify such help string pass parameter named help to add_option(). Like this:

parser.add_option('-n', '--new', help='creates a new object')

Telling optparse to parse argumentsBACK TO TOC

Once we’ve added all the options we want our program to support, we should tell optparse that it is time to do actual parsing. This is how we do it.

(opts, args) = parser.parse_args()

parse_args() method returns a tuple of objects. First object, opts, contains all values we’ve received via command line. We will learn how to use this later in the article.

args is a list of words that contain everything that left after options that parser recognizes. This is useful if you want your program to support indefinite number of arguments – like in cp or mv Unix commands, where you specify options first and then a long list of files to copy/move, followed by a destination directory.

Putting it togetherBACK TO TOC

This is how our script looks like so far.

#!/usr/bin/python

import optparse

parser = optparse.OptionParser()
parser.add_option('-n', '--new', help='creates a new object')

(opts, args) = parser.parse_args()

Running it with -h command line option, would produce following result.

alex ~/works/args -> ./args.py -h
Usage: args.py [options]

Options:
  -h, --help         show this help message and exit
  -n NEW, --new=NEW  creates a new object
alex ~/works/args ->

args.py is the name I had given to the script. Take this little nice help screen. Note that we didn’t do a thing to make it appear. This is where power of optparse begins.

optparse cookbookBACK TO TOC

What we saw until now were common things that you do with optparse to operate it. No matter what options you would like your program to support and what are relationships between the options, you have to instantiate OptionParser and have to call parse_args() method.

However, few methods that we’ve seen so far, are useless by themselves. We use them to do more specific things, which we will study in this section of the article.

Support for a standalone command line optionsBACK TO TOC

I call such options boolean options because eventually we want some boolean variable to indicate if optparse saw the option or not.

For the sake of the demonstration, lets say we want our script to support -b option. When this option is specified, we want some variable to be True. This is how we do it.

parser.add_option('-b', help='boolean option', dest='bool', \
    default=False, action='store_true')

(opts, args) = parser.parse_args()

Note three new named parameters that we’re passing to add_option(). dest specifies the name of the variable that will hold True or False value once optparse runs into -b. default specifies default value of the variable, in case optparse doesn’t find -b. Finally action tells optparse what to do when it runs into -b. store_true action tells optparse to place True value into bool once it detects -b.

Additional action that we may want to use called store_false. It tells optparse to set a value of bool to False. Note that when we use store_false, we better change default value to something other than False – otherwise you won’t be able to see the difference between when -b is there and it is not.

Once parse_args() finishes, you can access the variable via opts.bool. You can drop the default value of the variable. In this case, if you don’t specify -b command line option, value of opts.bool will be None.

Support for mandatory (required) options.BACK TO TOC

optparse itself doesn’t support mandatory options. As optparse‘s documentation states, mandatory options are a bad practice and should be avoided. However, I think that at the end, it is your choice to make and modules such as optparse should give you the tools to do things that you want to do. This said, this is what I usually do to have a mandatory option.

For a single variable, just don’t set default value for it and check if opts.<variable name> is None, after opt_parse(). Like this:

parser.add_option('-m', help='mandatory option', dest='man',
    action='store_true')

(opts, args) = parser.parse_args()

if opts.man is None:
    print "A mandatory option is missing\n"
    parser.print_help()
    exit(-1)

Obviously, when a mandatory option is missing, we want to do something about it. This can be anything of course, but most likely you want to tell about the mistake and print a help message. This is exactly what happens in lines 7-9.

But what if you have multiple mandatory options? Well, we can do exactly the same, but for several options. Lets have a look.

parser.add_option('-m', help='mandatory option', dest='man',
    action='store_true')
parser.add_option('-p', help='mandatory option', dest='pan',
    action='store_true')

(opts, args) = parser.parse_args()

# Making sure all mandatory options appeared.
mandatories = ['man', 'pan']
for m in mandatories:
    if not opts.__dict__[m]:
        print "mandatory option is missing\n"
        parser.print_help()
        exit(-1)

Here, we have two mandatory options: -m and -p. In lines 9-14, I build a list of all mandatory options (represented by their destination property name), run through the list and see if any of them is None. If so, it prints an error message, help message and returns.

Support for options with one or more argumentsBACK TO TOC

With optparse we can parse options with an argument and even several arguments. This is how we do it.

parser.add_option('-s', help='arguments', dest='opt_args', \
    action='store')

This example is somewhat different from what we’ve seen before. Here, use an action named store. This action, tells optparse to store option’s argument in specified destination member of opts.

How about having an option with two or three arguments. It is doable as well.

parser.add_option('-M', help='multiple arguments', dest='multi', \
    action='store', nargs=2)

As you can see, you can tell optparse how many arguments you want some option to support. To do this, we pass nargs parameter to add_option(). Its default value is 1. This is why we could omit it when we registered option with a single argument. However, if we need two or more options, nargs is a must.

In this case parse_args() will place all arguments in a tuple. I.e. after we run parse_args(), opt.multi will be a tuple containing all arguments that user has passed to our program.

Option destination typesBACK TO TOC

By default, all members of opts object will be strings. I.e. when you specify a destination for your argument, its type is string, by default. However, it doesn’t have to be this way. You can change the type of the destination variable for the argument by passing add_option() a parameter named type. Note that default type is string. Despite you can always convert string into anything you want, you can ask optparse to do it for you. This is how.

parser.add_option('-s', help='arguments', dest='opt_args', \
    action='store', type='int', default=10)
parser.add_option('-M', help='multiple arguments', dest='multi', \
    action='store', nargs=2, type='string')

Here we’ve specified int type for -s and string type for -M. Note default value of -s. It should be of the same type as option itself.

Also note -M. Although optparse turns two arguments into a tuple, it does not support different types for several arguments. So, when an option has multiple arguments, their type should be the same.

Dependencies between optionsBACK TO TOC

As with mandatory options, optparse does not give you much help setting dependencies between various options. If you still want to have some relationship between options, you have to implement it manually, after parse_args() is over.

Luckily this is not very difficult thing to do. Remember how we’ve implemented mandatory options? We can do the same here.

parser.add_option('-A', help='option A', dest='a',
    action='store_true')
parser.add_option('-B', help='option B', dest='b',
    action='store_true')

(opts, args) = parser.parse_args()

if opts.b and not opts.a:
    print "Option B requires option A\n"
    parser.print_help()
    exit(-1)

Here, option B requires option A. If B is there, but A is missing, script will produce an error message, print help screen and exit.

Help screen customizationsBACK TO TOC

Grouping optionsBACK TO TOC

optparse has a neat feature allowing you to group options. You can create as many groups as you want. Each group will have a short description that would appear on help screen. Also, you can use option groups to group options inside of your program, making it easier to understand and more readable. This is how you do it.

First, we create an option parser as we usually do.

parser = optparse.OptionParser()

Next we create a new options group.

group1 = optparse.OptionGroup(parser, 'Options group 1')
group2 = optparse.OptionGroup(parser, 'Options group 2')

Note that to create an option group, we have to specify parser that we will use and a short description of the options in this option group. The description will later appear in the help screen.

Now we should add the actual options. This time however, we add them to groups instead of adding them to parser.

group1.add_option('-t', help='group 1 option', dest='t', \
    action='store')
group2.add_option('-d', help='group 2 option', dest='d', \
    action='store')

add_option() method of class OptionGroup is exactly the same as add_option() method of class OptionParser.

Finally, we add groups to parsers and call parse_options().

parser.add_option_group(group1)
parser.add_option_group(group2)

(args, opts) = parser.parse_args()

Now, lets have a look at the help screen.

alex ~/works/args -> ./args.py -h
Usage: args.py [options]

Options:
  -h, --help  show this help message and exit

  Option group 1:
    -t T      group 1 option

  Option group 2:
    -d D      group 2 option
alex ~/works/args ->

See how -t and -d standout, each in its own group.

Changing programs usage stringBACK TO TOC

You can specify your own usage string. This is the first line of help screen. You do this by passing usage parameter to OptionParser constructor.

parser = optparse.OptionParser(usage='Usage: %prog <options>')

If we run this code, this is how our help screen would look like.

alex ~/works/args --> ./args.py -h
Usage: args.py <options>
.
.
.

As you can see, you can use %prog mnemonics inside of usage string. It will be substituted with the name of the program.

Telling option’s default valueBACK TO TOC

Another thing that you can do is specify default value of the option in its help screen. To do that you can use %default mnemonics inside of help string of an option. Like this:

parser.add_option('-w', default='hello', dest='t', \
    action='store', help='this one has a default value [%default]')

This will make help line in help screen for this option to look like this:

-w T        this one has a default value [hello]

Better argument descriptionBACK TO TOC

parser.add_option('-s', help='single argument', dest='single', \
    action='store')
parser.add_option('-M', help='multiple arguments', dest='multi', \
    action='store', nargs=2)

Lets have a look at how the help screen would look like for two options above.

  -s SINGLE          single argument
  -M MULTI           multiple arguments

Not a pretty sight. See how optparse described parameters that each option receives. In fact, optparse take this description from name of the destination argument. To generate such description, it converts its name into a string and changes it to upper case.

This may somewhat work for an option that receives one argument. But it certainly will not work for an option that receives mutliple arguments, such as -M. Luckily, there’s a solution for this problem. add_option() receives a parameter named metavar. It tells optparse how to describe option argument in help screen. So, instead of calling add_option() the way we did it for -s and -M, we should call it this way:

parser.add_option('-s', help='single argument', dest='single', \
    action='store', metavar='<ARG>')
parser.add_option('-M', help='multiple arguments', dest='multi', \
    action='store', metavar='<ARG1> <ARG2>', nargs=2)

This makes the help message for these two options look like this:

  -s <ARG>           single argument
  -M <ARG1> <ARG2>   multiple arguments

Now this is much better.

Adding program descriptionBACK TO TOC

You can have a description of what your program does. When having it, it will appear in help screen between usage line and options description. To do that, pass description argument to OptionParser constructor, when creating OptionParser object. Like this:

desc="""This is a description of %prog. optparse uses Python's
textwrap module to format the text, so
don't
bother
adding
new
line
characters, as optparse will prettify your description in
its own way."""

parser = optparse.OptionParser(description=desc)

parser.add_option('-s', help='single argument', dest='single', \
    action='store')

(args, opts) = parser.parse_args()

Running this little program with -h, will produce following help screen.

alex ~/works/args --> ./args.py -h
Usage: args.py [options]

This is a description of args.py. optparse uses Python's  textwrap
module to format the text, so  don't  bother  adding  new  line
characters, as optparse will prettify your description in its own
way.

Options:
  -h, --help  show this help message and exit
  -s SINGLE   single argument
alex ~/works/args -->

Note how optparse reformatted the description string. It uses Python’s textwrap module to format your description, producing nice, 80 character long lines of text.

Also note that as with usage parameter, you can use %prog mnemonics here. It will be substituted with name of your program.

Adding epilogBACK TO TOC

Epilog will appear after options description. To specify epilog, pass epilog parameter to OptionParser‘s constructor. Note that as with description string, optparse will prettify the text you use with textwrap.

Program version stringBACK TO TOC

Like help screen, optparse can also generate version string, responding to –version option. Unlike with help screen where optparse did most of the job, here optparse merely prints what you’ve specified as version string.

To specify version string, pass version parameter to OptionParser‘s constructor. You can use %prog mnemonics inside of version string. Here’s an example.

parser = optparse.OptionParser(version='%prog version 1.0')
(args, opts) = parser.parse_args()

Running this script with –version option, produces following result.

alex ~/works/args --> ./args.py --version
args.py version 1.0
alex ~/works/args -->

optparse produce version string only if you run it with –version option. If you want to your program to print its version with -v option, you will have to add -v option manually and then call print_version() method of OptionParser, to produce the version string.

ConclusionBACK TO TOC

There is a bunch of other things that you can do with optparse. You can have your own actions, your own formatter – an object that formats the help screen.  You can use callbacks that get called when optparse runs into certain option. However, I think I’ve covered 99.9% of what you may need. In case I missed something, send me an email to alex@alexonlinux.com.

Did you know that you can receive periodical updates with the latest articles that I write right into your email box? Alternatively, you subscribe to the RSS feed!

Want to know how? Check out
Subscribe page

41 Comments

  1. dwight says:

    thanks for the article…i’m right in the middle of a project using cmdline options…this really helps above and beyond the std Python docs. (using 2.6.2 from python.org)

  2. Reader says:

    ‘Usage:’ has no i18n.
    ‘-n, –new=NEW’ would be better than ‘-n NEW, –new=NEW’
    max_help_position should be bigger

  3. @Reader
    Usage has indeed no i18n. No ‘-n NEW, –new=NEW’ is better than ‘-n, –new=NEW’ – you do want to tell that -n requires an argument.

  4. @dwight
    I’m glad to hear someone has found this article helpful. Please visit again for more articles :-)

  5. Markus says:

    Thanks a bunch! I was enhancing a Python script that didn’t work under OSX and also missed some much required command line options. Never having used Python before didn’t make it easier… but your cookbook saved my day :-)

  6. @Markus
    You are most welcome. Please visit again :-)

  7. Hairy Seaword says:

    It is dangerous when checking for mandatory arguments to test if the variable is not false because this algorithm will not work when the mandatory argument is set to 0. You should instead test if it is None.

    if opts.man is None:
    print “A mandatory option is missing\n”
    parser.print_help()
    exit(-1)

  8. @Hairy Seaword
    I am not sure that you are right in this particular case because I explicitly asked not to specify default value. However, it is a good rule of thumb. Thanks for pointing it out and I’ll fix the article.

  9. flav says:

    Very clear and helpful article ! Tanks !

    ++

  10. Matthew says:

    Great article in addition to the Python docs.

    One thing I could not find an answer to is how can you disable the automatic help feature? When I use:

    parser.add_option(“-h”, “–help”, action=”store_true”, dest=”showHelp”, default=False)

    I get an error like this:

    optparse.OptionConflictError: option -h/–help: conflicting option string(s): -h, –help

    I would like to implement my own usage/help message.

    Thanks

  11. Martin says:

    I would like to use the opts structure to pass along to other functions in my program. But before doing so, I would like to add some items based on the options the user has already indicated. But I can’t seem to find a way to add items to opts, or (better yet) to iterate over opts and store the result in a list or dict.

    Above you show opts.__dict__[key], that could be used if I hold all the items in another list first and then iterate over that. I tried parser._get_all_options()…

  12. Martin says:

    Basically I would like to do this,

    for i in parser.values:
    print( “%s – %s” % (i[0], i[1]) )

    But parser.values (which is the same as opts) is not a sequence.

  13. Originally Posted By Martin
    I would like to use the opts structure to pass along to other functions in my program. But before doing so, I would like to add some items based on the options the user has already indicated. But I can’t seem to find a way to add items to opts, or (better yet) to iterate over opts and store the result in a list or dict.

    Above you show opts.__dict__[key], that could be used if I hold all the items in another list first and then iterate over that. I tried parser._get_all_options()…

    opts is a simple dictionary.
    You can do:
    opts.new_var = <new_var_value>
    it will create opts member called new_var and assign it a value new_var_value.

  14. Philip says:

    Thanks for the very good and essential instructions on optparse.

    I am currently struggling to print the choices in a %prog mnemonic way. It seems there is only “%prog” and “%default” available, but not “%choices” or the like..?

  15. @Philip
    Yes, as of Python2.6 usage only supports %prog. And I rarely change usage myself because of it. It is better to let optparse to generate help by itself. What you can do is to use metavar to change how it displays things, but this is as much as you can do.

  16. Philip says:

    @Alex

    Alex, you might have gotten me wrong. I was looking for something to display the possible choices of an option within that options help.

    Example:
    parser.add_option(“-m”, “–mesh”, choices=['dt_planar_trimesh','af_planar_trimesh','af_planar_quadmesh'],\
    dest=”mesh”,\
    help=”algorithm and element type for automeshing [%default out of %choices]“)

    However, “%choices” is not translated into (‘dt_planar_trimesh’,’af_planar_trimesh’,’af_planar_quadmesh’), which would be nice…

  17. @Philip
    Oh, I see your point. You are right, this feature isn’t implemented yet. I think you can submit a feature request here: http://bugs.python.org/

  18. anonymous says:

    for m in mandatories:
    if not opts.__dict__[m]:
    print “mandatory option is missing\n”

    Doesn’t work for me (with python 2.6)
    Solution:

    for m in mandatory_options:
    if not options.__dict__[m]:
    if getattr(options, m) == None:
    parser.error(“option –” + m + ” is mandatory.”)

  19. anonymous says:

    Whoops!

    for m in mandatory_options:
    if getattr(options, m) == None:
    parser.error(“option –” + m + ” is mandatory.”)

  20. @anonymous
    Thanks for correcting me. Actually, it is much cleaner the way you proposed.

  21. srikanth says:

    What a coincidence, about to re-read optparse documentation.

    Thanks for the article, very helpful.

  22. Jason C says:

    Have you checked out argparse, http://code.google.com/p/argparse/ ? It handles all of the cases you mentioned.

  23. @srikanth
    You are welcome. Please visit my web-site again ;D

  24. Jerry Rocteur says:

    Nice article.

    Can’t find you on twitter.

  25. gunslinger_ says:

    @Matthew – you don’t need to define -h or –help
    because optparse will added automatically

  26. Sn3akyP3t3 says:

    This article seems clear and descriptive, but isn’t useful as a complete tutorial in my experience. I’m trying to take action on a bool value once set. I’m having difficulty finding the correct function call to verify and use the stored value. This tutorial shows how to set values and display help which is all nice and all, but incomplete for most human beings. Now with version 2.7+ optparse is replaced with argparse as someone commented a link to above. However, for those running older versions such as myself I invite you to add the missing portions to make this completed. Thanks for all done thus-far.

  27. Iain says:

    Thanks Alex. I found this really useful.

  28. Jared Feldman says:

    Thank you so much for the help! I got it now!

  29. cv says:

    Thanks!! This was quite helpful.

  30. Brent says:

    Hi Alex, thanks for that. Off topic a bit, what is the plugin you use to display your code and to get the view source and copy popup. It looks great and have looked around for about an hour with no avail. Love your content also.
    Thanks!

  31. Jim says:

    Hi Alex,

    Thanks a lot for the clear, descriptive, to-the-point tutorial about optparse. I am not a Python programmer. But, I needed to create a script quickly in Python for the project I am working for. Your tutorial was perfect to cover the commandline arguments part of the script.

    Also, quite a catchy url: optparse-for-human-beings. Quickly followed the link from google search results as I was sure that this would be ideal for a novice like me in Python.

    Regards,
    Jim

  32. Zac says:

    “args is a list of words that contain everything that left after options that parser recognizes”

    wtf?

  33. Paul says:

    Your blog is very informative. Please keep posting!
    Paul
    Chicago, IL

  34. Legend Cox says:

    I have a question, If I wanted to make a dedicated application that would act as a command line inside the application much like if one were just to enter python into terminal and it comes up with the >>> then you can type raw commands into python. I’d like to make an app that would bring you to this but rather than entering long strings of commands to get one product, I’d like to be able to type something like “lights on” and activate a gpio pin on my raspberry pi, then have it come back saying lights on. Also having the ability to run other mini scripts with just that one command that could bring data back such as outdoor temperature, alerts from motion sensors, and so forth. If I make any sense please notify me.

  35. Manan says:

    Awesome article ! Cleared some of my doubts around option parser!

  36. godDLL says:

    Thank you. You should mention OptionParser.remove_option() for the benefit of people trying to do separate USAGE (-h) and FULL HELP (–help), like GNU.

Leave a Reply


four − 4 =