python

Python’s not the strictest language, so to have any confidence in your code you need to hit it with a barrage of checks to ensure the code at least meets some level of quality.

The tools I use are

  • Black, to ensure code is formatted
  • Pylint, I use this to disallow unused imports, and
  • Mypy for type checking.

A fourth is Pytest (unit test runner), but this can take some time to run so I run it manually, and not automated - the exception being in deployment scripts (we can’t allow deployments if tests are failing).

When taking on a new or existing project, setting up these automated checks should be the first action to take.

Pre-commit hooks

This runs the checks locally on your development machine when you try to commit.

Install pre-commit.

Add the following .pre-commit-config.yaml file to the root of your repository.

fail_fast: true

repos:
  - repo: https://github.com/ambv/black
    rev: 21.6b0
    hooks:
      - id: black
        args: [--diff, --check]

  - repo: https://github.com/pre-commit/mirrors-pylint
    rev: v3.0.0a3
    hooks:
      - id: pylint
        args: [--disable=all, --enable=unused-import]

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v0.902
    hooks:
      - id: mypy
        exclude: ^tests/
        args: [--strict]

Install them as git hooks:

pre-commit install

Github Actions

This will run the checks when a pull request is created, allowing a reviewer to see any potential issues straight up before beginning their review.

Add the following to your repository in .github/workflows/checks.yml.

name: Checks
on: [pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Checks
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v2
      with:
        python-version: 3.x
    - run: |
        pip install --upgrade pip
        pip install black==21.6b0 pylint==v3.0.0a3 mypy==v0.902
        black --diff --check mypackage
        pylint --disable=all --enable=unused-import mypackage
        mypy --strict mypackage

Even though we’re running the same checks as in pre-commit, this can actually pick up errors that weren’t found in pre-commit, e.g. if you’ve created a file but forgotten to add it to the repository.

Deployments

This will run final checks before deploying. It’s the most important place to run the checks because it’s the last line of defence.

Put the commands in your build script, before deploying.

set -e  # Stop on any error (you can also set this in the shebang)
black --diff --check /path/to/code
pylint --disable=all --enable=unused-import /path/to/code
mypy --strict /path/to/code
pytest /path/to/tests

Each of the commands will exit with a non-zero status if they find a problem, which will abort the script and should fail your build.

Notes

  • It’s important to choose a specific Black version and be consistent with it. The formatting often changes between Black versions, so what’s considered “formatted” in one version may not be in another.

  • If you have an existing project with unformatted code, format the entire codebase all at once. Don’t do it gradually.

See also