Testforge

---
name: testforge
description: Generates runnable pytest test suites from Python source and reports coverage gaps with concrete fill-in suggestions. Use when the user asks to 生成测试, 写单测, testforge, 补测试, generate tests, 增加覆盖率, or 给这个函数写测试; best for AI-generated code that ships without tests or with low coverage.
---

# testforge

A skill for the Novera-AI-skills suite that turns untested Python source into a
runnable pytest suite and then tells you exactly where the coverage is thin.

The problem it solves: AI-generated code often ships with no tests or with
tests that only assert `is not None`. `testforge` scaffolds the structure so
the agent spends its effort on *meaningful* assertions instead of boilerplate.

Both scripts are pure standard library - no `pip install` needed to run them.

---

## When to use

Trigger this skill when the user says any of:

- 生成测试 / 写单测 / 补测试 / 给这个函数写测试 / 增加覆盖率
- testforge / generate tests / add coverage

Also use it proactively after producing a Python module that has no tests.

---

## Workflow

Follow these steps in order. Each step names the script and its job.

### 1. Generate stubs

Run `gen_pytest.py` against the source file:

```
python scripts/gen_pytest.py <source.py>
```

It parses the file with the `ast` module and writes `test_<source>_gen.py`
next to it, with one skeleton test per top-level function and per class method.
Each stub imports the callable, calls it with `None` placeholders, and leaves a
single `# TODO` placeholder assertion. The script prints the number of stubs
and the output path.

What it covers:

- Top-level functions -> `def test_<name>()`.
- Class methods -> `def test_<Class>_<method>()` (instance, `@staticmethod`,
  and `@classmethod` are each handled; `__init__` is skipped).

### 2. Fill the assertions

Open the generated file and replace every `# TODO` with a real check. Use
`references/test-patterns.md` for copy-paste templates:

- Normal path: assert the return value for valid input.
- Boundary: empty / zero / None / negative / very large.
- Exception: `with pytest.raises(<ExpectedError>):` for bad input.
- Isolation: patch slow or external calls with `unittest.mock`.
- Prefer `@pytest.mark.parametrize` over copy-pasted tests.

A filled stub looks like this:

```python
def test_divide():
    assert divide(10, 2) == 5


def test_divide_by_zero():
    import pytest
    with pytest.raises(ValueError):
        divide(10, 0)
```

### 3. Run pytest

```
python -m pytest test_<source>_gen.py
```

`gen_pytest.py` does not require pytest, but running the suite does. Install it
with `pip install pytest` if missing.

### 4. Find the gaps

Point `coverage_hint.py` at the test directory:

```
python scripts/coverage_hint.py <test_dir>
```

It counts test files and test functions, flags any test that is still a stub or
asserts nothing, and prints concrete next steps (fill placeholders, add
exception tests, add boundary cases, use parametrize). If `pytest` is on the
PATH it also prints the exact command to run the suite.

Loop steps 2-4 until `coverage_hint` reports `0 stub(s)` and the suite is green.

---

## Scripts

### scripts/gen_pytest.py

- Input: a single `.py` source file.
- Output: `test_<source>_gen.py` in the same directory.
- Pure stdlib (`ast`). Deterministic. Errors are reported on stderr with a
  non-zero exit code (2 for bad args / missing file, 1 for parse failure).

```
python scripts/gen_pytest.py path/to/module.py
# -> generated 5 test stub(s) -> path/to/test_module_gen.py
```

### scripts/coverage_hint.py

- Input: a directory containing test files (`test_*.py` or `*_test.py`).
- Output: a scan summary plus suggestions. Does not need pytest installed.
- Pure stdlib. Detects the `pytest` executable and prints the run command when
  present.

```
python scripts/coverage_hint.py path/to/tests
```

---

## Reference material

- `references/test-patterns.md` - normal-path / boundary / exception / mock
  patterns with templates to paste into the generated stubs.

---

## Limitations

- `gen_pytest.py` generates structure, not intelligence. Placeholder arguments
  are `None`; the agent must supply real inputs or the call may raise at
  runtime. That failure is the signal to fill the stub.
- It does not compute line coverage percentages (that needs `pytest-cov`). It
  reports structural gaps: missing assertions, unfilled stubs, and missing
  exception/boundary/parametrize coverage.
- Nested functions and module-level lambdas are not scaffolded; only
  top-level functions and class methods are.