Installs into .claude/skills of the current project.
Are you the author of Evaluation Reproduction?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/vectorspacelab-evaluation-reproduction-arex-skill)
---
name: evaluation-reproduction
description: "Route Qwen benchmark reproduction and evaluation-script selection
for C-Eval, MMLU, CMMLU, GSM8K, HumanEval, and plugin workflows."
disable-model-invocation: true
metadata:
disco-role: operating
license: Apache 2.0
---
# Qwen Evaluation and Reproduction
Use this sub-skill when the user wants to reproduce Qwen benchmarks, select an evaluation script, prepare data directories, or understand the differences between base and chat evaluation paths.
## Safe start
- Use `scripts/qwen_eval_command_builder.py` to print a benchmark command plan without downloading datasets or loading a model.
- Treat HumanEval execution as sandbox-sensitive. Do not run untrusted generated code without explicit approval.
- Do not assume the chat evaluation scripts reproduce the same protocol as external benchmark harnesses; the repository documentation explicitly notes 0-shot chat results in some places where 5-shot results came from another system.
## Routes
| User request | Read |
| --- | --- |
| C-Eval, MMLU, CMMLU, GSM8K, HumanEval, or plugin benchmark overview and file layout | `references/benchmark-reproduction.md` |
| Script flags, model/checkpoint selection, dataset arguments, and base-vs-chat command differences | `references/evaluation-script-reference.md` |
| Missing dataset, wrong result path, sandbox concern, or benchmark dependency issue | `references/troubleshooting.md` |
## Boundaries
- For inference of the underlying checkpoint, use `../inference-model-loading/SKILL.md`.
- For prompt/tool-use behavior, use `../prompting-tool-use-tokenization/SKILL.md`.
- For serving or deployment, use `../serving-deployment/SKILL.md`.
- For finetuning or quantization, use `../finetuning-quantization/SKILL.md`.
## Operating rules
- Benchmark commands are only useful when the dataset layout, checkpoint family, and seed are explicit.
- Never present a score without the benchmark dataset and result file locations the repo expects.
- Keep the repository's warning about HumanEval untrusted code execution visible.
