Mt Proxy Correlation

---
name: mt-proxy-correlation
description: Evaluates whether machine translation quality serves as a scalable proxy for multilingual model performance on downstream tasks. It measures the alignment between MT metric scores and actual benchmark success across languages and model sizes. Use when the user has predictions and gold and needs to compute Pearson r.
metadata:
  skill_kind: metric
  source_arxiv: 2601.11778
  bibtex_key: [REDACTED]
  confidence: high
---

# mt-proxy-correlation

> Translation as a Scalable Proxy for Multilingual Evaluation — Issaka et al. (2026) (arXiv:2601.11778, 2026)

## What this evaluates

Evaluates whether machine translation quality serves as a scalable proxy for multilingual model performance on downstream tasks. It measures the alignment between MT metric scores and actual benchmark success across languages and model sizes.

## Datasets

- **FLORES-200** — total ?; splits: test (-1); HF `flores200`
- **Multilingual Benchmarks (Belebele, HellaSwag, AfriMMLU, Global MMLU, TruthfulQA, INCLUDE, MGSM, MLQA, AfriXNLI)** — total ?; splits: test (-1)

## Metrics

- `Pearson r` **(primary)** — range: [-1, 1]
  - Measures linear correlation between MT scores and benchmark scores across languages. Computed per model-metric-benchmark triplet.
- `Spearman ρ` — range: [-1, 1]
  - Measures rank correlation between MT scores and benchmark scores across languages. Computed per model-metric-benchmark triplet.

## Input / output format

**Input**: Per language, a model's MT metric score and its corresponding score on a downstream multilingual benchmark.

**Output**: Correlation coefficient (Pearson r or Spearman ρ) computed across languages.

## Scoring recipe

```python
def compute_correlation(mt_scores, bench_scores):
    # mt_scores: list of MT metric scores per language
    # bench_scores: list of benchmark scores per language
    r = pearsonr(mt_scores, bench_scores).statistic
    rho = spearmanr(mt_scores, bench_scores).statistic
    return {"pearson_r": r, "spearman_rho": rho}
```

## Common pitfalls

- Correlation is computed across languages, not models or instances, so it reflects cross-lingual alignment rather than per-instance accuracy.
- Metric sensitivity varies significantly by translation dataset (FLORES-200 vs. WMT24++ vs. NTREX) and task category (semantic vs. specialized reasoning).
- Median correlations can mask high variance on specific benchmark-metric pairs, requiring careful inspection of individual coefficients.

## Evidence (verbatim from paper)

> We analyze the relationship between MT quality and downstream multilingual task performance by computing correlations between 7 MT metrics and 9 multilingual benchmarks across 14 LLMs. ... For most metrics, the median correlation exceeds $r=0.80$. Neural metrics are the most consistently correlated: xCOMET achieves a median $r=0.91$, MetricX a median $r=0.89$, and SSA-COMET a median $r=0.87$.

## Citation

```bibtex
@misc{issaka2026translation,
  title={Translation as a Scalable Proxy for Multilingual Evaluation},
  author={Issaka et al. (2026)},
  year={2026},
  note={arXiv:2601.11778}
}
```

- arXiv: 2601.11778