How to run Black, Pylint, Mypy and Pytest in pre-commit hooks, Github Actions and deployments
Python’s not the strictest language, so to have any confidence in your code you need to hit it with a barrage of checks to ensure the code at least meets some level of quality.
The tools I use are
- Black, to ensure code is formatted
- Pylint, I use this to disallow unused imports, and
- Mypy for type checking.
A fourth is Pytest (unit test runner), but this can take some time to run so I run it manually, and not automated - the exception being in deployment scripts (we can’t allow deployments if tests are failing).
When taking on a new or existing project, setting up these automated checks should be the first action to take.
This runs the checks locally on your development machine when you try to commit.
Add the following
.pre-commit-config.yaml file to the root of your
fail_fast: true repos: - repo: https://github.com/ambv/black rev: 21.6b0 hooks: - id: black args: [--diff, --check] - repo: https://github.com/pre-commit/mirrors-pylint rev: v3.0.0a3 hooks: - id: pylint args: [--disable=all, --enable=unused-import] - repo: https://github.com/pre-commit/mirrors-mypy rev: v0.902 hooks: - id: mypy exclude: ^tests/ args: [--strict]
Install them as git hooks:
This will run the checks when a pull request is created, allowing a reviewer to see any potential issues straight up before beginning their review.
Add the following to your repository in
name: Checks on: [pull_request] jobs: build: runs-on: ubuntu-latest name: Checks steps: - uses: actions/checkout@v2 - uses: actions/setup-python@v2 with: python-version: 3.x - run: | pip install --upgrade pip pip install black==21.6b0 pylint==v3.0.0a3 mypy==v0.902 black --diff --check mypackage pylint --disable=all --enable=unused-import mypackage mypy --strict mypackage
Even though we’re running the same checks as in pre-commit, this can actually pick up errors that weren’t found in pre-commit, e.g. if you’ve created a file but forgotten to add it to the repository.
This will run final checks before deploying. It’s the most important place to run the checks because it’s the last line of defence.
Put the commands in your build script, before deploying.
set -e # Stop on any error (you can also set this in the shebang) black --diff --check /path/to/code pylint --disable=all --enable=unused-import /path/to/code mypy --strict /path/to/code pytest /path/to/tests
Each of the commands will exit with a non-zero status if they find a problem, which will abort the script and should fail your build.
It’s important to choose a specific Black version and be consistent with it. The formatting often changes between Black versions, so what’s considered “formatted” in one version may not be in another.
If you have an existing project with unformatted code, format the entire codebase all at once. Don’t do it gradually.