Installs into .claude/skills of the current project.
Are you the author of Benchmarking?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/vectorspacelab-benchmarking-arex-skill)
---
name: benchmarking
description: "Guides Logparser benchmark runs, dataset evaluation, and
accuracy/F1 comparison workflows."
disable-model-invocation: true
metadata:
disco-role: operating
license: NOASSERTION
---
# Benchmarking
Use this sub-skill for any workflow that compares parser output against ground
truth or runs a Loghub-style benchmark harness.
## Include here
- Evaluating `*_structured.csv` against ground truth.
- Interpreting `F1_measure`, `Accuracy`, parsing accuracy, grouping accuracy,
and template accuracy.
- Adapting the parser-specific `benchmark.py` files.
- Reasoning about `data/loghub_2k/` and `data/loghub_2k_corrected/` layouts.
## Exclude from here
- Choosing or tuning a parser for a raw log file; use `../parsing/SKILL.md`.
- Import shims, compilers, APIs, and other special parser issues; use
`../specialized-parsers/SKILL.md`.
## Read these references
- `references/benchmark-workflows.md` for the benchmark flow and dataset notes.
- `references/evaluator-reference.md` for the metric helper and output meaning.
- `references/troubleshooting.md` for result-path, dataset, and metric issues.
## Run this script
- `scripts/evaluate_csvs.py` — quick metric helper for a ground-truth / parsed
CSV pair.
## When to route here
Choose this sub-skill when the request says things like:
- "benchmark Drain"
- "compare parsers"
- "compute F1/accuracy"
- "evaluate the structured CSV"
- "run the Loghub benchmark"
- "inspect the benchmark output"
## Working notes
- The benchmark flow usually reuses the same parser configuration as the parsing
flow, so read the parsing sub-skill if you need to adjust the parser first.
- The benchmark scripts sometimes assume a dataset directory layout; confirm the
input paths before running a large job.
