Enginead Eval

---
name: enginead-eval
description: Probes the ability of one-class anomaly detection algorithms to identify incipient engine faults in real-world, multivariate vehicle sensor telemetry. It specifically evaluates cross-vehicle generalization and robustness to distributional shifts in normal operating conditions across a commercial fleet. Use when the user wants to benchmark on EngineAD, or asks about evaluating this task. Reports F1-score (anomaly class).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.25955
  bibtex_key: hojjati2026enginead
  confidence: high
---

# enginead-eval

> EngineAD: A Real-World Vehicle Engine Anomaly Detection Dataset — Hojjati et al. (2026) (arXiv:2603.25955, 2026)

## What this evaluates

Probes the ability of one-class anomaly detection algorithms to identify incipient engine faults in real-world, multivariate vehicle sensor telemetry. It specifically evaluates cross-vehicle generalization and robustness to distributional shifts in normal operating conditions across a commercial fleet.

## Datasets

- **EngineAD** — total ?; splits: train (-1), test (-1); repo https://github.com/Armanfard-Lab/EngineAD

## Metrics

- `F1-score (anomaly class)` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall computed exclusively on the anomaly class (positive class). Designed to balance false positives and false negatives in highly imbalanced fault detection scenarios.

## Input / output format

**Input**: Flattened 1D feature vector of shape (2400,) representing a 300-timestep multivariate time-series segment across 8 principal components, standardized using a scaler fitted solely on training data.

**Output**: Binary classification label (0=normal, 1=anomaly) derived from model anomaly scores, typically thresholded at the 95th percentile of training normal scores for scoring-based methods.

## Scoring recipe

```python
def compute_f1_anomaly(y_true, y_pred):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return f1
```

## Common pitfalls

- The test set is not a simple random split; it concatenates 30% of normal segments with all available anomaly segments, creating an artificial class balance that differs from the raw data distribution.
- Thresholds for distance/scoring methods are fixed at the 95th percentile of the training normal data scores, not tuned on the test set, which heavily constrains the precision/recall trade-off.
- Input segments are flattened into 1D vectors, discarding temporal structure, which may unfairly disadvantage sequence-aware or recurrent models compared to classical point-wise detectors.

## Evidence (verbatim from paper)

> Performance was measured using the F1-score calculated specifically for the anomaly class, as this metric provides a robust assessment of a detector’s efficacy in highly imbalanced and critical tasks.

## Citation

```bibtex
@misc{hojjati2026enginead,
  title={EngineAD: A Real-World Vehicle Engine Anomaly Detection Dataset},
  author={Hojjati et al. (2026)},
  year={2026},
  note={arXiv:2603.25955}
}
```

- arXiv: 2603.25955