Cvc Value Alignment Eval

---
name: cvc-value-alignment-eval
description: Evaluates how well large language models align with culturally grounded Chinese value rules compared to Western benchmarks. It probes moral reasoning, preference alignment, and boundary separation across six sensitive themes like drugs, firearms, politics, and suicide. Use when the user wants to benchmark on CVC, or asks about evaluating this task. Reports preference.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.01495
  bibtex_key: wu2025cvarc
  confidence: high
---

# cvc-value-alignment-eval

> C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models — Wu et al. (2025) (arXiv:2506.01495, 2025)

## What this evaluates

Evaluates how well large language models align with culturally grounded Chinese value rules compared to Western benchmarks. It probes moral reasoning, preference alignment, and boundary separation across six sensitive themes like drugs, firearms, politics, and suicide.

## Datasets

- **CVC** — total 170; splits: test (170); repo https://github.com/Beijing-AISI/CVC

## Metrics

- `preference` **(primary)** — range: percent
  - Percentage of test instances where the model's selected option matches the target value rule or human annotator consensus.

## Input / output format

**Input**: A scenario description paired with a set of action options, along with an associated value rule.

**Output**: Selection of the single most appropriate course of action from the provided options.

## Scoring recipe

```python
def compute_preference(predictions, gold_options):
    correct = sum(1 for p, g in zip(predictions, gold_options) if p == g)
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- Annotators were given minimal instruction to avoid value priming, so results reflect intrinsic model alignment rather than prompt-guided compliance.
- The evaluation uses paired rules from CVC and Western benchmarks (SC101/MIC) matched by theme, which may introduce bias if rule phrasing or cultural context differs significantly.

## Evidence (verbatim from paper)

> outperforming Western benchmarks (SC101, MIC) in value alignment—70.5% preference by LLMs and 87.5% agreement with Chinese human annotators—while demonstrating superior semantic clarity, diversity, and boundary separation in value-driven scenarios.

## Citation

```bibtex
@misc{wu2025cvarc,
  title={C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models},
  author={Wu et al. (2025)},
  year={2025},
  note={arXiv:2506.01495}
}
```

- arXiv: 2506.01495