Yufeng Xguard Eval

---
name: yufeng-xguard-eval
description: Evaluates the safety classification capabilities of guardrail models across multiple dimensions, including prompt/response safety detection, multilingual robustness, adversarial jailbreak resilience, and safe content completion. It also tests the model's ability to dynamically adapt to new moderation policies without retraining. Use when the user wants to benchmark on Aegis / Aegis2.0, WildGuard, StrongReject, SEval2.0, E-commerce Benchmark, Adaptive Policy Scope Benchmark, or asks about evaluating this task. Reports F1 score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.15588
  bibtex_key: [REDACTED]
  confidence: high
---

# yufeng-xguard-eval

> YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models — Lin et al. (2026) (arXiv:2601.15588, 2026)

## What this evaluates

Evaluates the safety classification capabilities of guardrail models across multiple dimensions, including prompt/response safety detection, multilingual robustness, adversarial jailbreak resilience, and safe content completion. It also tests the model's ability to dynamically adapt to new moderation policies without retraining.

## Datasets

- **Aegis / Aegis2.0** — total ?; splits: test (-1)
- **WildGuard** — total ?; splits: test (-1)
- **StrongReject** — total ?; splits: test (-1)
- **SEval2.0** — total ?; splits: test (-1)
- **E-commerce Benchmark** — total ?; splits: test (-1)
- **Adaptive Policy Scope Benchmark** — total ?; splits: test (-1)

## Metrics

- `F1 score` **(primary)** — range: percent
  - Harmonic mean of precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall). Reported as a percentage across benchmarks.

## Input / output format

**Input**: User prompts and/or model-generated responses.

**Output**: Binary safety classification label (safe/unsafe), risk category, confidence score, and natural-language explanation.

## Scoring recipe

```python
def compute_f1(predictions, gold):
    tp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 1)
    fp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return f1 * 100
```

## Common pitfalls

- Using a uniform confidence threshold (0.5 for prompts, 0.8 for responses) without tuning can significantly skew F1 scores across different benchmarks.
- Safe completion benchmarks like SEval2.0 are highly sensitive to 'over-blocking', where benign content is incorrectly flagged, drastically lowering F1 if not carefully calibrated.
- Multilingual performance averages can mask severe drops in specific low-resource or typologically distant languages.

## Evidence (verbatim from paper)

> To assess foundational safety detection, we evaluated models on a wide range of generic benchmarks. As shown in Table [2], YuFeng-XGuard-8B achieves the highest average F1 score on prompt classification.

## Citation

```bibtex
@misc{lin2026yufengxguard,
  title={YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models},
  author={Lin et al. (2026)},
  year={2026},
  note={arXiv:2601.15588}
}
```

- arXiv: 2601.15588