Generate Tests

---
name: generate-tests
description: Generate EvalView test cases — either from a SKILL.md file using LLM-powered generation, or by capturing real agent interactions through a proxy.
---

# Generate Tests

Use this skill when the user wants to create test cases for their AI agent or skill without writing YAML by hand.

## Four approaches

### 1. Generate tests from a SKILL.md file

Use the `generate_skill_tests` MCP tool to auto-generate a test suite from a skill definition. This reads the SKILL.md and produces YAML test cases covering explicit triggers, implicit triggers, contextual triggers, and negative cases.

**Steps:**
1. Ask the user which SKILL.md to generate tests for (or detect it from context).
2. Call `generate_skill_tests` with:
   - `skill_path`: path to the SKILL.md file
   - `output_path` (optional): where to save the generated YAML
   - `count` (optional): number of test cases (default: 10)
3. After generation, offer to run the tests with `run_skill_test`.

**CLI equivalent:**
```
evalview skill generate-tests .claude/skills/my-skill/SKILL.md --auto
evalview skill generate-tests .claude/skills/my-skill/SKILL.md -c 20 -o tests/my-skill-tests.yaml
```

### 2. Create individual test cases manually

Use the `create_test` MCP tool to create a single test YAML file from a description.

**Steps:**
1. Gather from the user: test name, query, expected tools, forbidden tools, expected output keywords, and minimum score.
2. Call `create_test` with the parameters.
3. After creating the test, call `run_snapshot` to establish the golden baseline.

### 3. Capture real interactions

Use the CLI `evalview capture` command to proxy real agent traffic and save interactions as test YAMLs automatically. This records the query, output, and tool calls from live usage.

**CLI equivalent:**
```
evalview capture --agent http://localhost:8080/execute --output-dir tests/test-cases
evalview capture --multi-turn  # saves all turns as one multi-turn conversation test
```

### 4. Validate a skill before testing

Use `validate_skill` to check a SKILL.md for correct structure and completeness before generating tests from it.

## Running generated tests

After generating tests, execute them with `run_skill_test`:
- `test_file`: path to the generated YAML
- `no_rubric: true` for fast deterministic-only checks (no LLM cost)
- `verbose: true` for detailed output on all tests

**CLI equivalent:**
```
evalview skill test tests/my-skill-tests.yaml
evalview skill test tests/my-skill-tests.yaml --no-rubric  # fast, $0
evalview skill test tests/my-skill-tests.yaml --verbose --model claude-sonnet-4-20250514
```