Skip to content

The anatomy of a test (in Pytest)

Imagine you have in your project:

# In a `tests/test_pricing.py` file

def discount(subtotal: int, code: str) -> float:
    if code == "SPRING":
        return subtotal * 0.9
    return subtotal

def test_discount_applies_to_subtotal():
    assert discount(subtotal=100, code="SPRING") == 90

def test_discount_dont_apply_to_subtotal():
    assert discount(subtotal=100, code="SPRINGS") == 90

The discount function is tested in the test_discount_applies_to_subtotal and test_discount_dont_apply_to_subtotal, and we can verify with Pytest running:

asciinema: pytest -q on a passing test, then on a failing one

Simple pattern, but in complex codebases, where functions call functions, abstractions, dependencies and classes, having these guarantees increases the confidence that the existing code works as expected.

input code output assert == PASS FAIL 1 · deterministic same input, every time 2 · single-valued exactly one right answer 3 · exact equality, not judgement 4 · cheap ≈ 2 ms · $0 run it on every push Four properties. Every one of them is load-bearing.
svg: the four properties, redrawn broken on the next page

What is a test?

Mechanically, a pytest test is a function whose name starts with test_ containing a bare assert. There is no class to subclass and no assertion API to learn: pytest rewrites the assert so a failure reports the actual values, which is why assert 100 == 90 comes back with the call that produced the 100. To run the same body over many inputs, @pytest.mark.parametrize takes the cases as data.

Conceptually it is a bet that four things hold. The input is deterministic. The output is single-valued: one right answer to compare against. The assertion is exact, equality or membership rather than taste. And the whole thing is cheap: milliseconds, no marginal cost, which is the only reason it is sane to run thousands on every commit.

Those four are what let a test collapse into one bit and gate a merge. Hold them in that order. But what happens when we don't know exactly what the code will output?

Go deeper