Pytest Rules

---
name: pytest-rules
description: "Team conventions for Python tests with pytest: suite layout by domain, test names, public interfaces, state over interactions, contract-tested fakes, flaky tests, strict markers, plain assert, expected exceptions, fixture scope, mocking at boundaries, time-machine, testcontainers, pytest-asyncio, Hypothesis, coverage, random order and timeouts. Use when writing, fixing or reviewing a pytest test or test configuration. Background knowledge that agents preload; not a command."
user-invocable: false
---

<!-- generated by knowledge.mjs from 33 note(s) (sha256 41772fa919d9); edit the notes in knowledge/, then run build -->

# pytest rules

Apply these rules to every Python test you write or review. The first section is always in force; open a rule from the table when the task touches it.

## Always apply (critical, high)

### Make every test able to fail when the behaviour it names breaks

_critical · any_

**Rule.** End every test with an assertion that would fail if the behaviour in its name broke. Before you
trust a new test, check that it can fail: break the condition it checks, or assert the opposite,
and see it go red.

**Why.** A test that only renders and clicks, or asserts something that is always true, stays green while
the feature is broken, so the suite reports coverage that does not exist.

### Fix the cause of a flaky test instead of rerunning it until it passes

_critical · any_

**Rule.** Treat a test that fails without a code change as a defect: find its nondeterminism (clock, random data, scheduling, network, test order, shared state) and remove it.
A rerun (`--reruns`, `@flaky`, `retryTimes`, `retry`) is never the fix: a suite that reruns reports each test that passed only on a rerun as flaky, to be fixed.

**Why.** A rerun turns the suite green while the race or leak behind the failure is still in the code, and
once red can mean "just flaky", people stop reading failures and miss the real ones.

### Assert the resulting state, not which calls the code made to its collaborators

_high · any_

**Rule.** Assert on what the code produced: its return value, the error it raised, or the state now held by a
real or fake dependency. Verify a call on a mock only when the call itself is the behaviour, such as
sending an e-mail, charging a card or invoking a callback a developer passed in, and then check only
the arguments that behaviour depends on.

**Why.** A test that asserts `repo.save` was called once with given arguments fails on every harmless
refactor and still passes when the saved data is wrong, because it checks how the result was
reached instead of the result.

### Keep branches and loops out of a test's body

_high · any_

**Rule.** Write each test as straight-line code (arrange, act, assert), with no `if` or loop around the
action or the assertion. Run one behaviour over many inputs with the framework's parametrisation
instead of a loop.

**Why.** A branch in a test means some runs check nothing, and a loop over inputs stops at the first failure
without naming the input that broke.

**Exceptions.** A helper that waits for a condition, or that starts the competing calls of a race test, may loop
inside itself; the test that calls it stays straight-line.

### Keep the unit under test real, and change the test rather than the code

_high · any_

**Rule.** Mock only what the unit under test depends on, never the unit itself. When a new test fails, fix
the test; if the failure shows a real defect in the code, report it instead of editing the code or
weakening the assertion to make the test pass.

**Why.** A mocked or quietly patched unit makes the test check itself: it passes while the shipped code is
still broken, and the bug it found disappears from view.

### Replace a dependency the test cannot run with a fake that passes the real contract tests

_high · any_

**Rule.** When the real implementation is too slow, costly or unavailable for a test, replace it with a fake:
a small working implementation that keeps its state in memory, not a stub per call or a set of call
expectations. Run one contract suite against both the fake and the real implementation, so the fake
behaves like the real one wherever tests rely on it. Whether to replace a dependency at all is
`testing/strategy/principles/mock-only-what-you-cannot-run`.

**Why.** A fake that nobody checks against the real implementation drifts from it, so tests pass against
behaviour production does not have, and stubs repeated in every test leak the implementation and
break on each refactor.

**Exceptions.** Use a stub to force a single return value or error the fake cannot easily produce, such as a
timeout. A third-party service with no runnable implementation (payments, e-mail) checks its fake
against the provider's sandbox or recorded responses, or the test says the gap is unchecked.

### Make every test pass alone and in any order

_high · any_

**Rule.** Write each test so it passes when run alone and in any order. Configure and reset mocks before each
test, and share repeated setup through a small helper function, never through a mutable variable
that tests write to.

**Why.** A test that leans on an earlier one passes in the full run and fails when filtered or reordered,
which shows up as a flaky suite whose failures point at the wrong test.

### Lay out Python tests by domain, with the kind as a marker

_high · python, pytest_

**Decision.** Mirror the package in `tests/`: one directory per domain module, one file per topic (`test_api.py`,
`test_service.py`, `test_domain.py`, `test_concurrency.py`). Say what kind a test is with a
registered marker (`integration` for a real database, `concurrency` for race tests), not with a
folder, so a module's tests stay together and are moved or deleted with it. Open a second top-level
directory only for a suite that needs a run of its own: infrastructure the normal run cannot start
(a deployed environment, a cloud account) or a schedule, with its own make target and CI job.

### Build test Settings with _env_file=None

_high · python, pytest, pydantic-settings_

**Rule.** Create settings in tests with `Settings(_env_file=None, ...)`, ideally through one helper in `conftest.py`, and pass explicitly each value the test depends on. Do not call `Settings()` bare in a test.

**Why.** A bare `Settings()` picks up the developer's `.env`, so the same test passes locally and fails in CI, or the reverse, for a reason no one can see in the test.

**Correct:**

```python
def make_settings(**overrides: Any) -> Settings:
    return Settings(_env_file=None, environment="test", **overrides)

def test_page_size() -> None:
    assert make_settings(default_page_size=5).default_page_size == 5
```

**Exceptions.** `_env_file=None` skips the file only. Exported shell variables still fill any field the test does not pass, so a suite that must be immune to them also clears the relevant ones with `monkeypatch.delenv`.

### Expect an exception with a specific type, match= and one statement in the block

_high · python, pytest_

**Rule.** Use `pytest.raises(SpecificError, match=...)` with exactly one statement inside the block.

**Why.** A broad `pytest.raises(Exception)`, or a block that also runs setup, passes when the setup or a typo raises, so the test stays green while the behaviour is broken; `match=` (a regex searched in the message) rules out a different error of the same type.

**Correct:**

```python
order = Order(total=-1)
with pytest.raises(ValueError, match="amount must be positive"):
    charge(order)
```

### Keep fixtures function-scoped; widen the scope only for resources tests never mutate

_high · python, pytest_

**Rule.** Leave fixtures at the default `function` scope. Use `module` or `session` only for expensive resources that no test mutates, or reset them per test.

**Why.** A shared fixture that one test modifies makes the next test pass or fail depending on the order they run in.

**Correct:**

```python
@pytest.fixture
def cart():
    return Cart()
```

### Mock only system boundaries, and create every mock with autospec

_high · python, pytest_

**Rule.** Mock the edges of the system (HTTP with `respx` for httpx or `responses` for requests, queues), not your own modules; the clock has its own rule. Create each mock with `autospec=True`.

**Why.** A plain mock accepts any attribute and any arguments, so a renamed method keeps the test green while the real call fails.

**Correct:**

```python
with patch.object(Client, "fetch_user", autospec=True, return_value=user):
    ...   # a wrong name or signature raises
```

### Prove a race-guard test by removing the guard once and watching it fail

_high · python, pytest_

**Rule.** After writing a test for a race guard, remove the guard once and confirm the test fails. Write the test with the competing calls released together and assert on the final state, as in the example, not on timing.

**Why.** A race test that also passes without the guard proves nothing: on SQLite a last-copy test stayed green against a mutant that kept `BEGIN IMMEDIATE` and wrote an absolute value, so it guarded nothing.

**Correct:**

```python
def test_only_one_of_eight_gets_the_last_copy(client, book_with_one_copy):
    barrier = threading.Barrier(8, timeout=5)      # a timeout fails a stuck run fast

    def lend(_: int) -> int:
        barrier.wait()
        return client.post("/loans", json={"book_id": book_with_one_copy.id}).status_code

    with ThreadPoolExecutor(8) as pool:            # pool size >= Barrier parties
        statuses = list(pool.map(lend, range(8)))

    assert sorted(statuses) == [201] + [409] * 7   # then remove the guard: this must fail
```

**Exceptions.** A guard that a database constraint already enforces and maps to its own error (`UNIQUE` to 409) needs one sequential test of the error mapping.

Pick a mutant the backend lets fail. A `CHECK` that backs the guard makes removing the `WHERE` clause go red on an error (a 500), not on interleaving. On SQLite, `BEGIN IMMEDIATE` serialises every writer, so a read-check-write mutant that keeps it still passes; the mutant that shows red is a plain `BEGIN`, with `database is locked` 500s. Do not claim the conditional `UPDATE` itself is proven there.

Confirm the mutant was the code that ran: run it from a copy and print the package's `__file__` first, because an editable install imports the original and the test stays green.

### Test each module through the interface its callers use, never through private helpers

_high · python, pytest_

**Rule.** Test a route through the HTTP test client, a service through its public methods with fakes for its
ports, a domain object through its public methods, and an adapter through its port's contract suite.
Never import a name that starts with `_` into a test or assert on an object's private attributes;
cover that code through the public call that uses it.

**Why.** A test of `_compute_fine` or of `loan._state` breaks when the code is reorganised without a change
in behaviour, and still passes while the public call that uses it is broken.

**Correct:**

```python
def test_returning_a_book_three_days_late_charges_a_fine(service: LoanService) -> None:
    loan = service.borrow(BOOK_ID, MEMBER_ID, today=date(2026, 1, 1))

    returned = service.return_book(loan.id, today=loan.due + timedelta(days=3))

    assert returned.fine == Decimal("1.50")
```

**Exceptions.** A helper complex enough to need its own edge cases (a parser, a pricing formula) becomes a public
function of its own module and gets its own tests.

### Mock only what a test cannot run or could only run with great difficulty

_high · any_

**Rule.** Mock something only when the test could not be written without the mock or would be very awkward with
the real thing. In UI tests that means the network, animations, and services with a real cost or effect
such as payments and e-mail. In E2E tests it means almost nothing.

**Why.** Every mock cuts the link between the tested code and what it replaced, so the suite stays green when
that link breaks in production.

### Reproduce a bug with a failing test before fixing it

_high · any_

**Rule.** When a bug earns a test, write the test that reproduces it before touching the code, and see it fail.
Decide whether it earns one the way you prioritise any other work, by what the breakage costs users.

**Why.** A regression test written after the fix may never have failed, so it can pass without covering the bug,
and the bug comes back with the test still green.

**Exceptions.** A bug that costs users little, in a place that is expensive to test, can get a fix with no test.

## Read when the task touches it

| Impact | Rule | Applies to |
| --- | --- | --- |
| medium | [Keep test setup inline until repetition hides what differs between tests](references/rules.md#keep-test-setup-inline-until-repetition-hides-what-differs-between-tests) — A helper takes overrides and returns what the test needs, with no branch per case; a test shows what matters to it. | any |
| medium | [Derive the expected value from the test's own input, not a repeated literal](references/rules.md#derive-the-expected-value-from-the-tests-own-input-not-a-repeated-literal) — Assert against the data the test passed in (items.length, the prefilled value), never a hard-coded copy of it. | any |
| medium | [Name each test after the behaviour it checks, the situation and the expected outcome](references/rules.md#name-each-test-after-the-behaviour-it-checks-the-situation-and-the-expected-outcome) — test_returning_an_overdue_book_charges_a_fine, not test_return_book or test_2. | any |
| medium | [Add new tests to the project's existing suite, in the file for their topic](references/rules.md#add-new-tests-to-the-projects-existing-suite-in-the-file-for-their-topic) — A new test goes in the file for its topic; do not write a case twice or open a parallel suite. | python, pytest |
| medium | [Control the clock with time-machine, not hand-patched datetime](references/rules.md#control-the-clock-with-time-machine-not-hand-patched-datetime) — time-machine moves the clock for the whole process, including names already imported. | python, pytest, time-machine |
| medium | [Mirror the package in tests/ by domain module, never by test kind](references/rules.md#mirror-the-package-in-tests-by-domain-module-never-by-test-kind) — tests/orders/ holds every kind of orders test; a marker, not a unit/ or integration/ folder, says the kind. | python, pytest |
| medium | [Install pytest-randomly and pytest-timeout so hidden coupling and hangs fail loudly](references/rules.md#install-pytest-randomly-and-pytest-timeout-so-hidden-coupling-and-hangs-fail-loudly) — Random order exposes order dependence and a global timeout stops a hung test from stalling CI. | python, pytest |
| medium | [Register every marker and run pytest with --strict-markers](references/rules.md#register-every-marker-and-run-pytest-with---strict-markers) — A misspelled marker is only a warning, so -m "not slow" still runs the test you meant to exclude. | python, pytest |
| medium | [Run one contract suite against the database adapter and the in-memory fake](references/rules.md#run-one-contract-suite-against-the-database-adapter-and-the-in-memory-fake) — One suite and a repo fixture per adapter, so the in-memory fake and the SQL adapter pass the same tests. | python, pytest |
| medium | [Run integration tests against the production database engine with testcontainers](references/rules.md#run-integration-tests-against-the-production-database-engine-with-testcontainers) — A SQLite stand-in accepts SQL and types the real engine rejects; a throwaway container does not. | python, pytest, testcontainers |
| medium | [Pin every counterexample Hypothesis finds as an @example](references/rules.md#pin-every-counterexample-hypothesis-finds-as-an-example) — A failure found by generated inputs becomes a fixed regression case that runs on every build. | python, pytest, hypothesis |
| medium | [Measure branch coverage and never lower an existing fail_under](references/rules.md#measure-branch-coverage-and-never-lower-an-existing-fail_under) — branch = true catches the else that never ran; lowering the threshold to go green stops guarding. | python, pytest |
| medium | [Use pytest's built-in fixtures instead of hand-rolled setup](references/rules.md#use-pytests-built-in-fixtures-instead-of-hand-rolled-setup) — tmp_path, monkeypatch, capsys and caplog create and undo their state per test; reach for them before writing your own. | python, pytest |
| medium | [Keep pytest-asyncio in strict mode and mark every async test and fixture](references/rules.md#keep-pytest-asyncio-in-strict-mode-and-mark-every-async-test-and-fixture) — Strict mode runs only marked coroutine tests, so it never claims tests meant for another async framework. | python, pytest, pytest-asyncio |
| medium | [Use plain assert and test one behaviour at a time](references/rules.md#use-plain-assert-and-test-one-behaviour-at-a-time) — A test with one reason to fail has a name that says what broke; several asserts for that behaviour are fine. | python, pytest |
| medium | [Choose the next test by what would hurt most if it broke](references/rules.md#choose-the-next-test-by-what-would-hurt-most-if-it-broke) — Start from the worst breakage in untested code, narrow to its units and users, write the manual steps, automate them. | any |
| low | [Give every parametrized pytest case a readable id](references/rules.md#give-every-parametrized-pytest-case-a-readable-id) — Name each case with pytest.param(id=) or ids=, so a failure says which input broke. | python, pytest |