Evaluates Chinese NLP models on few-shot learning across nine tasks, including single-sentence classification, sentence-pair classification, and machine reading comprehension. It tests the ability of pre-trained language models and few-shot prompting/fine-tuning methods to generalize with limited labeled data. Use when the user wants to benchmark on FewCLUE, or asks about evaluating this task. Reports accuracy.
Installs into .claude/skills of the current project.
Are you the author of Fewclue Eval?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/qhjqhj00-fewclue-eval)
---
name: fewclue-eval
description: Evaluates Chinese NLP models on few-shot learning across nine tasks, including single-sentence classification, sentence-pair classification, and machine reading comprehension. It tests the ability of pre-trained language models and few-shot prompting/fine-tuning methods to generalize with limited labeled data. Use when the user wants to benchmark on FewCLUE, or asks about evaluating this task. Reports accuracy.
metadata:
skill_kind: dataset_eval
source_arxiv: 2107.07498
bibtex_key: xu2021fewclue
confidence: medium
---
# fewclue-eval
> FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark — Xu et al. (2021) (arXiv:2107.07498, 2021)
## What this evaluates
Evaluates Chinese NLP models on few-shot learning across nine tasks, including single-sentence classification, sentence-pair classification, and machine reading comprehension. It tests the ability of pre-trained language models and few-shot prompting/fine-tuning methods to generalize with limited labeled data.
## Datasets
- **FewCLUE** — total ?; splits: train (-1), test (-1); repo https://github.com/CLUEbenchmark/FewCLUE
## Metrics
- `accuracy` **(primary)** — range: [0, 1]
- Proportion of correctly predicted labels out of total instances. Calculated as correct predictions divided by total test instances.
## Input / output format
**Input**: Chinese text sentences or sentence pairs, optionally reformatted into cloze-style prompts with [MASK] tokens and candidate label words for few-shot methods.
**Output**: Predicted label token or probability distribution over candidate label tokens.
## Scoring recipe
```python
correct = 0
for pred, gold in zip(predictions, gold_labels):
if pred == gold:
correct += 1
return correct / len(gold_labels)
```
## Common pitfalls
- Template and label-word selection significantly impact few-shot performance in PET/ADAPET.
- Small labeled subsets cause high variance; results should be averaged over multiple random seeds/splits.
- Cloze-style reformulation may alter semantic meaning, affecting model predictions.
## Evidence (verbatim from paper)
> Finally, supervised training is performed on the dataset by using cross-entropy loss.
## Citation
```bibtex
@misc{xu2021fewclue,
title={FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark},
author={Xu et al. (2021)},
year={2021},
note={arXiv:2107.07498}
}
```
- arXiv: 2107.07498
