Fuximt Xxzh Eval

---
name: fuximt-xxzh-eval
description: Evaluates multilingual machine translation capability for Chinese-involved pairs (xx-zh), measuring translation quality across varying levels of parallel data availability. It probes how well models leverage cross-lingual knowledge transfer and handle data scarcity in low-resource settings. Use when the user wants to benchmark on xx-zh translation pairs, or asks about evaluating this task. Reports BLEU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.14256
  bibtex_key: zhu2025fuximt
  confidence: high
---

# fuximt-xxzh-eval

> FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation — Shaolin Zhu et al. (2025) (arXiv:2505.14256, 2025)

## What this evaluates

Evaluates multilingual machine translation capability for Chinese-involved pairs (xx-zh), measuring translation quality across varying levels of parallel data availability. It probes how well models leverage cross-lingual knowledge transfer and handle data scarcity in low-resource settings.

## Datasets

- **xx-zh translation pairs** — total ?; splits: (unstated)

## Metrics

- `BLEU` **(primary)** — range: percent
  - Standard sentence-level BLEU score (n-gram precision with brevity penalty) computed on xx-zh translation pairs.

## Input / output format

**Input**: Source sentence in a target language (xx)

**Output**: Translated sentence in Chinese (zh)

## Scoring recipe

```python
def compute_bleu(predictions, references):
    # predictions and references are lists of strings
    # Compute sentence-level BLEU (typically BLEU-4 with brevity penalty)
    # Average BLEU scores across all test instances
    return average_bleu(predictions, references)
```

## Common pitfalls

- BLEU is reported exclusively for the xx-zh direction; bidirectional or zh-xx scores are not provided.
- Resource categories (High/Medium/Low/Very Low) are defined by the volume of available parallel data, not by language family or typology.
- Ablation results report a single 'Average BLEU' across pairs, while main results are stratified by resource level.

## Evidence (verbatim from paper)

> Table [1] presents the performance of FuxiMT and the baselines across different resource levels of languages, categorized as High, Medium, Low, and Very Low resource in terms of the amount of parallel data available (as described in Appendix Table [5]).

Table 1: BLEU scores for different resource languages xx-zh.

## Citation

```bibtex
@misc{zhu2025fuximt,
  title={FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation},
  author={Shaolin Zhu et al. (2025)},
  year={2025},
  note={arXiv:2505.14256}
}
```

- arXiv: 2505.14256