User Test Concept

# User Test Concept

Step 8.5 of the PM Pipeline. Generates a concept test plan for validating solution wireframes with real users.

## Purpose

Bridge the gap between PM-reviewed wireframes and real user feedback. A 15-minute concept test with 3-5 users catches usability issues and validates the core value proposition before investing in high-fidelity design and development.

## When to use

- After step 8 (sketch-solution) when wireframes exist
- When PM wants to validate the solution direction with real users before Problem Research Report
- When the solution involves novel interaction patterns that need validation
- When PM has access to 3-5 target users

## When NOT to use

- No access to target users
- Extremely tight timeline (concept test adds 3-5 days)
- Solution is a minor iteration on existing patterns (low risk)
- Solution has already been validated through other means

## Inputs

- `output/solution-sketch.md` — solution description with screens
- `output/screens/` — wireframe images (if available)
- `output/hypotheses.md` — original problem hypotheses
- `CONTEXT.md` — user segment definition

## Process

### 1. Generate test scenario

Create a realistic scenario that puts the user in the right context:

```markdown
## Scenario
"Imagine you're [context from CONTEXT.md]. You want to [job-to-be-done from solution hypothesis].
You open [product] and see this screen..."
```

Rules:
- Scenario must be specific and concrete
- Don't lead the user toward the "right" answer
- Use language the user would use, not PM jargon
- Include a realistic trigger (why are they here?)

### 2. Generate questions per screen

For each screen in the solution sketch:

**First impression** (5 seconds):
- "What is this screen about?"
- "What would you do first?"

**Comprehension**:
- "How would you [key action for this screen]?"
- "What do you expect to happen when you tap [element]?"

**Value**:
- "Is this useful to you? Why / why not?"
- "How does this compare to how you do this today?"

**Friction**:
- "Is anything confusing or unclear?"
- "Is anything missing?"

Rules:
- Open-ended questions only (no yes/no)
- Don't explain the UI — let the user interpret
- 3-5 questions per screen maximum
- Total test duration: 15 minutes max

### 3. Define success/fail criteria

For each solution hypothesis being tested:

```markdown
## Success criteria
- [ ] 4/5 users understand the main screen purpose within 5 seconds
- [ ] 3/5 users can complete the primary task without hints
- [ ] 3/5 users say this is better than their current approach
- [ ] No user encounters a "dead end" (confusion with no recovery)

## Fail criteria (stop/rethink)
- [ ] 2+ users fundamentally misunderstand the core concept
- [ ] 3+ users can't find the primary CTA
- [ ] 3+ users say they wouldn't use this
```

### 4. Create test protocol

```markdown
## Concept Test Protocol

**Duration**: 15 minutes
**Participants**: 3-5 users from [segment]
**Format**: 1-on-1, screen share or in-person
**Recording**: with consent

### Setup (2 min)
- Introduce: "We're testing a design idea, not testing you"
- "Think aloud — say what you're thinking as you look at each screen"
- "There are no wrong answers"

### Screens (10 min)
[Screen-by-screen walkthrough with questions]

### Wrap-up (3 min)
- "Overall, how useful would this be to you? (1-5)"
- "What would you change?"
- "Anything else?"
```

## Output

Write `research/concept-test-results.md` with:

1. **Test plan** (generated by AI — sections above)
2. **Results** (filled by PM after conducting test):
   - Per-user notes
   - Success criteria checklist
   - Key quotes
   - Patterns across users
3. **Recommendations**:
   - What to keep
   - What to change
   - What to investigate further

## After the test

When PM provides results:
1. Update solution hypothesis evidence: `REAL, confidence 0.8-1.0`
2. If fail criteria triggered — discuss with PM whether to iterate on sketch (back to step 8) or proceed with caveats
3. Update `output/solution-sketch.md` changelog with test findings
4. Log in `output/decisions.md`