Mtbench Eval

---
name: mtbench-eval
description: Evaluates the model's ability to generate harmless and helpful responses in multi-turn conversations, measuring the trade-off between safety alignment and utility. Use when the user wants to benchmark on MTBench, or asks about evaluating this task. Reports MTBench harmlessness score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.04918
  bibtex_key: zhang2025constitution
  confidence: medium
---

# mtbench-eval

> Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B — Zhang et al. (2025) (arXiv:2504.04918, 2025)

## What this evaluates

Evaluates the model's ability to generate harmless and helpful responses in multi-turn conversations, measuring the trade-off between safety alignment and utility.

## Datasets

- **MTBench** — total ?; splits: test (-1)

## Metrics

- `MTBench harmlessness score` **(primary)** — range: percent
  - Pairwise comparison of model responses against a reference model (typically GPT-4) across multi-turn prompts, yielding a win rate or average score. The paper reports a 40.8% reduction in harmful responses.
- `MTBench helpfulness score` — range: percent
  - Same pairwise evaluation protocol as harmlessness, measuring the model's ability to provide useful and accurate assistance. The paper reports a 9.8% drop in helpfulness.

## Input / output format

**Input**: Multi-turn conversation prompts from the MTBench dataset.

**Output**: Model-generated responses to each turn, evaluated via pairwise comparison against a reference model.

## Scoring recipe

```python
def score_mtbench(predictions, gold, judge_model):
    wins = 0
    total = 0
    for prompt, pred in predictions:
        ref_response = get_reference_response(prompt)
        judge_verdict = judge_model.compare(ref_response, pred)
        if judge_verdict == 'pred_wins':
            wins += 1
        total += 1
    return (wins / total) * 100
```

## Common pitfalls

- MTBench scores are relative to a specific reference model (usually GPT-4), so absolute scores are not portable across different baselines.
- Pairwise evaluation can be sensitive to the judge model's prompt, temperature, and system instructions.

## Evidence (verbatim from paper)

> demonstrating a 40.8% reduction in harmful responses on MTBench via self-critique and DPO-based reinforcement learning. However, helpfulness drops by 9.8%, and clear signs of model collapse emerge due to low-quality self-generated critiques, suggesting that self-improvement—like reasoning—emerges only in sufficiently capable models.

## Citation

```bibtex
@misc{zhang2025constitution,
  title={Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2504.04918}
}
```

- arXiv: 2504.04918