Results And Analysis

---
name: results-and-analysis
description: "Interpret OpenCompass summaries, result tables, summary_groups,
  subjective/LLM-judge outputs, result-station reuse, repeat analysis, and
  post-run diagnostics."
disable-model-invocation: true
metadata:
  disco-role: operating
license: Apache 2.0
---

# OpenCompass Results and Analysis

Use this sub-skill when an OpenCompass task is about reading, explaining, or post-processing outputs after inference/evaluation has produced `predictions/`, `results/`, or `summary/` files.

## Route Here For

- Explaining summary table columns: `dataset`, `version`, `metric`, `mode`, model score columns, and `-` placeholders.
- Debugging missing aggregate rows from `summarizer.dataset_abbrs` or `summarizer.summary_groups`.
- Choosing or interpreting objective metrics such as accuracy, exact match, BLEU, ROUGE, toxicity, pass rates, F1, AUC, and dataset-specific metrics.
- Configuring or diagnosing `GenericLLMEvaluator`, `CascadeEvaluator`, judge environment variables, and subjective evaluation summaries.
- Reading `--mode viz`, `--reuse`, result-station read/write behavior, case analysis outputs, and repeat-analysis reports.

## Route Elsewhere

- Dataset creation, custom dataset readers, `abbr`, `reader_cfg`, and config import structure belong in `../configuration-and-datasets/SKILL.md`.
- Launch commands, runner setup, `--mode all/infer/eval/viz`, `--debug`, and `--dry-run` belong in `../evaluation-workflows/SKILL.md`.
- Prompt templates, retrievers, `GenInferencer`/`PPLInferencer`, and answer masking belong in `../prompt-and-inference/SKILL.md`.
- Model backend installation, acceleration, HuggingFace/API adapters, and real inference environment issues belong in `../model-backends/SKILL.md`.

## Start Here

1. Locate the active run directory, then inspect `summary/summary_*.txt`, `summary/summary_*.csv`, `results/<model>/<dataset>.json`, and `predictions/<model>/<dataset>*.json`.
2. If a summary cell is `-`, first check whether the dataset or metric exists in `results/`; then check `dataset_abbrs`, `summary_groups`, metric names, and model abbreviations.
3. If a group row is absent or has `-`, verify that every listed subset has a parsed numeric metric for every model; one missing subset prevents that model's group aggregate.
4. For LLM-judge or subjective runs, inspect the judge-specific result directories and keep API keys in environment variables or a private runtime secret source, not in reusable public configs.
5. For repeated or suspicious generations, use `scripts/analyze_repeat_results.py` on exported prediction JSON/JSONL/CSV fixtures before escalating to full native repeat analysis.

## References

- `references/summarizers-and-results.md`: result table semantics, metric selection, `summary_groups`, result files, reuse/viz interpretation, case and multi-model analysis.
- `references/llm-judge-and-subjective.md`: `GenericLLMEvaluator`, cascade judging, subjective compare/score modes, judge credentials, and output locations.
- `references/troubleshooting.md`: missing cells, incompatible versions, mixed modes, judge env vars, postprocessor failures, and result-station overwrite/read flags.
- `scripts/analyze_repeat_results.py`: self-contained repeat-pattern analyzer for tiny JSON, JSONL, or CSV prediction fixtures.

## Safety Notes

- Do not claim real HuggingFace model execution was verified merely because result files or CLI help are inspectable.
- Treat `version` hashes as comparability guards: mixed prompt/eval settings can make two numeric scores non-comparable even when dataset names match.
- Keep judge credentials out of runtime skill content, committed configs, logs, summary artifacts, and copied examples.