Osbad Eval

---
name: osbad-eval
description: Evaluates the ability of statistical and machine learning models to detect anomalies in battery discharge capacity profiles across different chemistries. It probes cross-chemistry generalization and model robustness on imbalanced, rare-anomaly datasets typical of electrochemical systems. Use when the user wants to benchmark on MIT/Stanford (Severson), Tohoku, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.01745
  bibtex_key: pang2025osbad
  confidence: high
---

# osbad-eval

> An Open-Access Benchmark of Statistical and Machine-Learning Anomaly Detection Methods for Battery Applications — Mei-Chin Pang et al. (2025) (arXiv:2511.01745, 2025)

## What this evaluates

Evaluates the ability of statistical and machine learning models to detect anomalies in battery discharge capacity profiles across different chemistries. It probes cross-chemistry generalization and model robustness on imbalanced, rare-anomaly datasets typical of electrochemical systems.

## Datasets

- **MIT/Stanford (Severson)** — total 46; splits: train (23), test (23); repo https://github.com/meichinpang/osbad
- **Tohoku** — total 10; splits: train (4), test (4); repo https://github.com/meichinpang/osbad

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across all classification thresholds.

## Input / output format

**Input**: Discharge cycle profiles (voltage, current, capacity) for each battery cell.

**Output**: Binary anomaly label per cycle (0 = inlier/normal, 1 = outlier/anomalous).

## Scoring recipe

```python
def compute_auroc(y_true, y_scores):
    from sklearn.metrics import roc_auc_score
    return roc_auc_score(y_true, y_scores)
```

## Common pitfalls

- Data leakage if test cells are used during hyperparameter tuning or baseline assessment
- AUROC can be misleading on highly imbalanced datasets with rare anomalies
- Cross-chemistry generalization requires evaluating models trained on liquid electrolyte cells against solid electrolyte test cells

## Evidence (verbatim from paper)

> The labeled test set is kept completely separate until the final evaluation to prevent data leakage and inflated performance estimates. The framework highlights the limitations of traditional metrics (e.g., AUROC) in imbalanced, rare-anomaly settings.

## Citation

```bibtex
@misc{pang2025osbad,
  title={An Open-Access Benchmark of Statistical and Machine-Learning Anomaly Detection Methods for Battery Applications},
  author={Mei-Chin Pang et al. (2025)},
  year={2025},
  note={arXiv:2511.01745}
}
```

- arXiv: 2511.01745