2. Writing executable scripts with Argparse

Objectives: learn how to use argparse to write more sophisticated Python scripts.

Intro to argparse

In all of the Python scripts we’ve written so far we have hardcoded various options, such as a file to load. By hardcode, we mean that the user has no input as to what the script should do at runtime; everything is pre-coded before hand. This makes for some rather limited scripts, since you would need to re-write a script everytime you want to change anything.

The only bit of user-input at runtime we’ve seen before is by using the input argument. But this is somewhat clunky, as the program will stop until the user inputs something. It makes batch computing (in which you run the same script a bunch of times just chaning the input) difficult, for example.

In bash, we have seen that it’s possible to provide arguments to commands to get them to change their default behavior. For example, running ls will list the contents of the current directory, whereas running ls folder will list the contents of folder instead. In this case, folder is known as a positional argument. We’ve also seen that it’s possible to provide optional arguments to bash commands to get the to change their behavior. For example, running rm folder will fail because the default rm cannot delete folders, but running rm -r folder will work. The -r tells rm to recursively remove everything in folder followed by folder itself.

It would be wonderful if we could also provide positional and optional arguments to our Python scripts so that the user can change how the script behaves at runtime. Fortunately, we can! Enter argparse. Argparse is an in-built module that provides the ability to add arguments (both required and optional) to your scripts. It allows you to write more fully featured, sophisticated Python scripts. The Python docs have an excellent tutorial on argparse. Here, we’ll just cover some of the basics.

The basic steps to using argparse in your script are:

Import argparse.
Create an argparse.ArgumentParser instance.
Add options to the parser.
Parse the user input. You can then use the user input in your script.

Below we’ll see some examples to get you acquainted with it.

Positional arguments

Here’s a simple example, which we’ll call echo.py:

# echo.py
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('echo')
args = parser.parse_args()

print(args.echo)

This adds a single positional argument to our script called echo. To use it, on the command line we run:

python echo.py hello

You should get back hello. As you can see, whatever the user provides after the name of the script is mapped to the echo argument, which can then be accessed from within the program via args.echo. As with bash commands, spaces determine what counts as different arguments. For example, if you try running:

python echo.py hello world

You get an error:

usage: echo.py [-h] echo
echo.py: error: unrecognized arguments: world

This is because argparse sees “world” as a second argument, but since the script only takes one argument (echo), it doesn’t know what to do with it. To pass space-separated strings as a single argument, you need to encase them in quotes, just as we do with other bash commands. In this case, running:

python echo.py "hello world"

works; you’ll get back hello world.

Alternatively, we could add a second positional argument to our program, call it echo2:

# echo2.py
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('echo1')
parser.add_argument('echo2')
args = parser.parse_args()

print(args.echo1, args.echo2)

Now we can do:

python echo2.py hello world

In this case, “hello” gets mapped to “echo1” and “world” gets mapped to echo2. Running with any more or any less arguments will raise an error. (Try it!) This is because, as we’ve written it, the script expects two and only two input arguments, i.e., two arguments are required.

Multiple args

If we expect to always pass multiple inputs for a single argument, we can add the nargs keyword argument to parse_arg to tell it how many inputs to expect. For example, if we want our echo.py to take in two inputs, but don’t want to create separate arguments for each, we can do:

# echo.py
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('echo', nargs=2)
args = parser.parse_args()

print(args.echo)

Now if we run:

python echo.py hello world

it works, we get:

['hello', 'world']

Notice that now args.echo is a list, with the elements in the list corresponding what was given on the command line. This program will still expect two and only two inputs (try it!). If we instead pass nargs='*' to add_argument, the argument can take zero or more arguments, with no limit. For example:

# echo.py
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('echo', nargs='*')
args = parser.parse_args()

print(args.echo)

Now running:

python echo.py hello world, how are you?

yields:

['hello', 'world,', 'how', 'are', 'you?']

Providing help messages

The add_argument method takes several other keyword arguments aside from nargs. One of the most useful is help. The help keyword allows you to write a message about what an argument does in the program and how to use it. The user can see the help message by running your script with the argument -h or --help. In either case, the script will just print the help message then quit. For example, if we add a help to our echo.py program:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('echo', help='The argument to print.')
args = parser.parse_args()

print(args.echo)

Then running:

python echo.py -h

yields:

usage: echo.py [-h] echo

positional arguments:
  echo        The argument to print.

options:
  -h, --help  show this help message and exit

We can also add a description for the overall program by providing the description keyword argument to ArgumentParser. For example:

import argparse

parser = argparse.ArgumentParser(description="Prints to screen whatever is provided.")
parser.add_argument('echo', help='The argument to print.')
args = parser.parse_args()

print(args.echo)

Then

python echo.py -h

yields:

usage: echo.py [-h] echo

Echoes whatever is provided.

positional arguments:
  echo        The argument to print.

options:
  -h, --help  show this help message and exit

Note that running the script with -h or --help always works, even if we don’t provide a help message for an argument. In that case, the arguments just won’t have any informative message in the help, but the user will still at least see what arguments the script takes. The help feature is one of the nice things about argparse: it comes in-built; you don’t need to add anything extra to get it to work.

Specifying types

By default, arguments are assumed to be strings. You can check that yourself: add print(type(args.echo)) to your echo.py script. What if we want to input another datatype, like a float? You can do that by providing the type keyword to add_argument. For example, say we want our echo argument to be a float:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('echo', help='The argument to print.', type=float)
args = parser.parse_args()

print(args.echo, type(args.echo))

Running, yields:

python echo.py 1
1.0 <class 'float'>

Note that passing in python echo.py hello no longer works in this case, since hello cannot be cast to a float.

Code Challenge 3.2.1

Write a Python script called add.py that will take two positional arguments as input, a and b and print their sum. Add help messages to a and b.

Solution

# add.py

import argparse

parser = argparse.ArgumentParser(description="Prints the sum of two numbers.")
parser.add_argument('a', help='The first value to sum.', type=float)
parser.add_argument('b', help='The second value to sum.', type=float)
args = parser.parse_args()

print(args.a + args.b)

Optional arguments

In addition to positional arguments, argparse allows you to define optional arguments. These are arguments that start with either a - or a --. We’ve already seen one example of an optional argument, the in-built -h/--help.

To add an optional argument, you simply add -- to the start of the argument name. For example, building on the previous challenge problem, here’s a simple math script in which the user can optionally specify what operator to do:

# calculator.py

import argparse

parser = argparse.ArgumentParser(description="Prints the sum of two numbers.")
parser.add_argument('a', help='The first value to sum.', type=float)
parser.add_argument('b', help='The second value to sum.', type=float)
parser.add_argument('--operator', default='add', help='What operation to do. Default is add.')
args = parser.parse_args()

if args.operator == 'add':
    res = args.a + args.b
elif args.operator == 'subtract':
    res = args.a - args.b
elif args.operator == 'multiply':
    res = args.a * args.b
elif args.operator == 'divide':
    res = args.a / args.b
else:
    raise ValueError(f"unrecognized operator {args.operator}")
print(res)

Running:

python calculator.py 2 3
5.0

whereas:

python calculator.py 2 3 --operator multiply
6.0

Specifying choices

In the previous example our operator argument should only take four pre-defined possibilities. We can specify those using the choices keyword argument. Providing that both provides a more helpful help message, and also negates needing to check that the argument is one of the knowns, since argparse will check for you before the script is even executed. Applying to our calculator.py script:

import argparse

parser = argparse.ArgumentParser(description="Prints the sum of two numbers.")
parser.add_argument('a', help='The first value to sum.', type=float)
parser.add_argument('b', help='The second value to sum.', type=float)
parser.add_argument('--operator', default='add', choices=['add', 'multiply', 'divide', 'subtract'],
                    help='What operation to do. Default is add.')
args = parser.parse_args()

if args.operator == 'add':
    res = args.a + args.b
elif args.operator == 'subtract':
    res = args.a - args.b
elif args.operator == 'multiply':
    res = args.a * args.b
elif args.operator == 'divide':
    res = args.a / args.b
print(res)

Short options

We can specify short aliases for optional arguments by providing both in the add_argument. For example:

import argparse

parser = argparse.ArgumentParser(description="Prints the sum of two numbers.")
parser.add_argument('a', help='The first value to sum.', type=float)
parser.add_argument('b', help='The second value to sum.', type=float)
parser.add_argument('-o', '--operator', default='add', choices=['add', 'multiply', 'divide', 'subtract'],
                    help='What operation to do. Default is add.')
args = parser.parse_args()

if args.operator == 'add':
    res = args.a + args.b
elif args.operator == 'subtract':
    res = args.a - args.b
elif args.operator == 'multiply':
    res = args.a * args.b
elif args.operator == 'divide':
    res = args.a / args.b
print(res)

We can now run our script by specifying either -o or --operator for the operator. For example:

python calculator.py 2 3 -o multiply
6.0

Code Challenge 3.2.2

Add the ability to calculator.py to optionally write the result out to a file.

Solution

# calculator.py

import argparse

parser = argparse.ArgumentParser(description="Prints the sum of two numbers.")
parser.add_argument('a', help='The first value to sum.', type=float)
parser.add_argument('b', help='The second value to sum.', type=float)
parser.add_argument('-o', '--operator', default='add', choices=['add', 'multiply', 'divide', 'subtract'],
                    help='What operation to do. Default is add.')
parser.add_argument('-f', '--output-file', help='Write the result to the given file.')
args = parser.parse_args()

if args.operator == 'add':
    res = args.a + args.b
elif args.operator == 'subtract':
    res = args.a - args.b
elif args.operator == 'multiply':
    res = args.a * args.b
elif args.operator == 'divide':
    res = args.a / args.b
if args.output_file:
    with open(args.output_file, 'w') as fp:
        print(res, file=fp)
else:
    print(res)