User Guide

The pidiff command

The pidiff command compares two versions of a Python package and produces a report on API differences.

usage: pidiff
       [--workdir WORKDIR]
       [-e ENABLE]
       [-d DISABLE]
       [--requirement REQUIREMENT]
       [--constraint CONSTRAINT]
       [--index-url INDEX_URL]
       [--extra-index-url EXTRA_INDEX_URL]
       [--pip-args PIP_ARGS]

Positional Arguments


Old package for test; a requirement specifier as accepted by the pip command

The most common forms of specifying a pip-installable package are supported, including:

  • latest version: mypkg

  • a specific version or range: mypkg==1.0.0, mypkg<2

  • a local directory containing a ~/src/mypkg


New package for test


Name of the Python module which serves as the entry point of the API to test. If omitted, the command will attempt to determine this automatically using egg metadata

Named Arguments


Use this working directory

-v, --verbose

Verbose execution


Use fully qualified names in log messages

-r, --recreate

Force recreation of virtual environments


Version generator mode: report is written to stderr; proposed new version number is written to stdout; exit code is 0 unless an error occurs.

-e, --enable

Enable checks by error code or name. Multiple checks can be provided, comma-separated. Option may be provided multiple times.

-d, --disable

Disable checks by error code or name.

pip arguments

These arguments affect the behavior of pip install while pidiff installs the packages to be diffed. See pip install --help for more info on the behavior of these arguments.


Install from the given requirements file.

--constraint, -c

Constrain versions using the given constraints file.


Include pre-release and development versions.

--index-url, -i

Base URL of the Python Package Index.


Extra URLs of package indexes.


Additional arguments to pass to the `pip’ command.

Output format

The pidiff command produces output as in the following example:

$ pidiff more-executors==1.15.0 more-executors==1.16.0
more_executors/ N230 method added: flat_bind
more_executors/ N450 ExceptionRetryPolicy now accepts unlimited keyword arguments
more_executors/ B330 argument in ExceptionRetryPolicy can no longer be passed positionally: max_attempts (was position 0)
more_executors/ N450 RetryExecutor now accepts unlimited keyword arguments
more_executors/ B130 method removed: new_default
more_executors/ N220 function added: flat_bind

Major API changes were found; inappropriate for 1.15.0 => 1.16.0
New version should be equal or greater than 2.0.0

For each change found, a message is produced with:


Approximate location of the added/removed/changed object.

Note that this is the location where the related object is defined, which may not be in the same file where the object is exported as public API.

error code

Each type of error is associated with an error code. Error codes are one of:


New backwards-compatible functionality. New classes, objects, functions or arguments are available to clients. The package minor version must be increased.


Breaking changes. Classes, objects, functions or arguments were removed or changed in such a way that clients written against the old package version may be broken when used against the new version. The package major version must be increased.


A summary explaining why the diff passed or failed.

If the diff fails, a new version number will be suggested for the package, where possible.

Exit codes

The pidiff command uses the following exit codes:


Either no differences were found, or all differences are appropriate for old and new package versions, or --gen-version is in use and no errors occurred.


--gen-version is in use and errors occurred.


Breaking API changes were found, and this is inappropriate for the old and new package versions; the major version should be bumped.


New functionality was found, and this is inappropriate for the old and new package versions; the minor version should be bumped.

other non-zero

An error occurred.

Configuring checks

By default, pidiff enables all checks.

Individual checks may be explicitly disabled or enabled either using the --disable, --enable command-line options, or using a settings file.

pidiff will look for settings in these files, in the current directory and any parent directories:

  • pidiff.ini

  • tox.ini

  • setup.cfg

Settings should be placed under a [pidiff] section. Checks may be enabled or disabled as in the following example:

# list of checks to enable

# list of checks to disable

The enable setting and command-line argument takes precedence over disable.

Checks may be listed using either an error code or a check name. To find the name associated with each error code, see the Error Reference or the Index.

What is “public API”?

Roughly, the tool’s concept of “public API” is: any object reachable from any modules underneath your package’s entry point, with a name not beginning with _.

A more complete description of the method used to enumerate public API follows.

  • First, the module_name given to the pidiff command is imported (or, if omitted, the module to import is detected from the package’s top_level.txt metadata).

  • All submodules of that module are also imported, recursively, ignoring any modules whose name begins with _.

  • All modules imported by the above process are enumerated with dir() to find available objects; those objects themselves are enumerated with dir() to find child objects; and so on, recursively. Processing stops for any objects whose name begins with _ or whose location is not underneath the directory containing the API entry point.

Caveats and limitations

  • Python 2.x is not supported.

  • It must be possible to import the API to be checked from within the same Python interpreter used for the pidiff command.

  • pidiff doesn’t check the return values of functions and methods.

  • pidiff is designed for pure Python modules only and is not expected to work for native extensions.