Long Term Impact Evaluation

---
name: long-term-impact-evaluation
description: "Choose methods for measuring long-term product impact after or beyond an A/B test. Use when comparing long-term holdbacks, post-period analysis, continuous monitoring, CLV models, delayed effects, short-term versus long-term metric tradeoffs, or lower-cost alternatives to long-term holdbacks."
category: data-science-and-ml
source_repo: hashgraph-online/awesome-codex-plugins
source_path: "plugins/LVTD-LLC/skills/skills/long-term-impact-evaluation/SKILL.md"
source_url: https://github.com/hashgraph-online/awesome-codex-plugins/blob/HEAD/plugins/LVTD-LLC/skills/skills/long-term-impact-evaluation/SKILL.md
---


# Long-Term Impact Evaluation

Use this skill to choose a practical method for measuring product impact beyond
the initial experiment window. It compares long-term holdbacks, post-period
analysis, continuous monitoring, and customer lifetime value models.

## Source Traceability

Primary source: *Next-Level A/B Testing* by Leemay Nassery. Guidance is
transformed and paraphrased from Chapter 8 on long-term impact, short-term and
long-term metric relationships, holdbacks, post-period analysis, continuous
monitoring, and CLV models.

Related skills:

- `holdback-experiment-design` for detailed holdback planning.
- `trustworthy-experiment-insights` for result credibility and false positives.
- `experimentation-strategy-roadmap` for cost, quality, and complexity tradeoffs.

## Reference Routing

| Need | Read |
|------|------|
| Long-term evaluation concepts | `references/core/knowledge.md` |
| Method selection and risk rules | `references/core/rules.md` |
| Scenario examples | `references/core/examples.md` |
| Step-by-step method selection | `workflows/choose-long-term-evaluation.md` |

## Workflow

1. State why short-term experiment metrics are insufficient.
2. Identify the expected time horizon and delayed mechanisms.
3. Compare holdback, post-period analysis, continuous monitoring, and CLV model
   options.
4. Weigh accuracy against user cost, business cost, complexity, and confounding.
5. Choose the simplest method that answers the long-term question.
6. Define metric cadence, interpretation limits, and follow-up decisions.

## Output Format

```markdown
# Long-Term Impact Evaluation Plan

## Long-Term Question
[What delayed or sustained effect must be measured.]

## Recommended Method
[Holdback | Post-period analysis | Continuous monitoring | CLV model | Hybrid]

## Tradeoffs
| Method | Benefit | Cost/Risk | Fit |
|--------|---------|-----------|-----|

## Measurement Plan
- Short-term metric:
- Long-term metric:
- Time horizon:
- Confounders:

## Decision Rules
- Continue measuring if:
- End or revise if:
- Escalate if:
```

## Quality Bar

- Do not create a long-term holdback when a lower-cost method can answer the
  decision well enough.
- Do not use post-period analysis without accounting for external factors such
  as seasonality or campaigns.
- Do not rely on CLV models without naming model drift and behavior-change risk.
- Do not confuse continuous monitoring with causal long-term measurement.

---

**Source:** [`hashgraph-online/awesome-codex-plugins`](https://github.com/hashgraph-online/awesome-codex-plugins) → `plugins/LVTD-LLC/skills/skills/long-term-impact-evaluation/SKILL.md`