Ab Testing

---
name: ab-testing
description: Use when designing experiments for subject lines, offers, cadences, or
  journeys.
---

# Experimentation & A/B Testing Skill

## When to Use
- Validating new subject lines or creative.
- Testing segmentation hypotheses (persona vs behavior).
- Optimizing cadence, timing, or automation triggers.

## Framework
1. **Hypothesis** – define expected uplift + rationale.
2. **Metric Selection** – primary (open/click/conv) + guardrails (unsubs, spam).
3. **Sample Sizing** – ensure stat significance (min 500 recipients per variant or use power calculator).
4. **Execution** – randomize, keep variants isolated, limit simultaneous tests.
5. **Analysis** – use z-test or Bayesian uplift; document learnings.

## Templates
- Experiment brief (hypothesis, segments, KPI, risk guardrails).
- Variant table (control vs test inputs, creative asset links, owner).
- Calculator sheet for minimum detectable effect + sample size.
- Post-test debrief doc capturing learnings + rollout plan.

## Experiment Ideas
- Subject line vs preview text combos.
- CTA placement (hero vs footer).
- Personalization depth (basic vs dynamic modules).
- Wait times between touches.

## Tips
- Run no more than two tests per journey simultaneously.
- Recycle learnings into playbooks + automation templates.
- Segment results by persona to catch hidden signals.

---