Claude Skills by ByteStack-Labs

Detects and quantifies confidently-wrong behavior: where a model, classifier, agent, or LLM judge is highly confident and incorrect, and where the coupling between confidence and correctness breaks down under distribution shift. Measures calibration on the in-distribution set and again on the production distribution, builds reliability diagrams, reports expected calibration error and the high-confidence error rate, and produces a selective-prediction or escalation rule you can wire into monit...

Start here. Audits a deployed ML or LLM or agent system that scores well on evaluation but fails, regresses, or behaves unexpectedly in production. Runs a reproducible root-cause "autopsy": frames the eval-to-deployment gap, reproduces the production failure, quantifies it by slice, tests confidence calibration under distribution shift, isolates root cause by ablation, and produces a prioritized diagnostic report with next steps. It also triages across reliability areas and hands off to the d...

Verifies the tools an agent or orchestrator depends on, by re-deriving a tool evaluation's real accuracy instead of trusting a single pass/fail score. Separates a formatting miss (a correct value scored wrong) from a real failure (a wrong value scored right), recomputes the expected answer from the raw inputs rather than a stored ground truth, and surfaces the silent-wrong cases a naive exact-match scorer rubber-stamps. Produces a reproducible tool-reliability receipt and keeps any model's qu...

Evaluates whether an agent or a multi-step or multi-agent system actually completes the task, not just whether each step or the final answer looks right. Captures full trajectories (every step, tool call, and intermediate state), measures per-step success against end-to-end success, exposes the compounding-error curve where high per-step reliability still collapses over a long trajectory, attributes failures to the step that introduces them, and tests whether composing or chaining agents help...