Hephaestus Minicubes Eval

---
name: hephaestus-minicubes-eval
description: Evaluates models on detecting volcanic ground deformation using multi-modal InSAR data. It probes the ability to classify deformation presence and segment deformation areas from spatiotemporal interferometric time-series, while handling atmospheric noise and class imbalance. Use when the user wants to benchmark on Hephaestus Minicubes, or asks about evaluating this task. Reports F1-score, IoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.17782
  bibtex_key: papadopoulos2025hephaestus
  confidence: high
---

# hephaestus-minicubes-eval

> Hephaestus Minicubes: A Global, Multi-Modal Dataset for Volcanic Unrest Monitoring — Papadopoulos et al. (2025) (arXiv:2505.17782, 2025)

## What this evaluates

Evaluates models on detecting volcanic ground deformation using multi-modal InSAR data. It probes the ability to classify deformation presence and segment deformation areas from spatiotemporal interferometric time-series, while handling atmospheric noise and class imbalance.

## Datasets

- **Hephaestus Minicubes** — total ?; splits: train (9840), val (2570), test (6501); repo https://github.com/Orion-AI-Lab/Hephaestus-minicubes

## Metrics

- `F1-score` **(primary)** — range: percent
  - Harmonic mean of precision and recall: 2 * (Prec * Rec) / (Prec + Rec).
- `IoU` **(primary)** — range: percent
  - Intersection over Union: size of intersection between predicted and ground truth masks divided by size of their union.
- `Precision` — range: percent
  - Ratio of true positive predictions to all positive predictions.
- `Recall` — range: percent
  - Ratio of true positive predictions to all actual positives.
- `AUROC` — range: [0, 1]
  - Area Under the Receiver Operating Characteristic curve, measuring classification performance across all thresholds.

## Input / output format

**Input**: Multi-channel InSAR datacubes (phase, coherence, DEM, atmospheric variables) cropped to 512x512 pixels. For time-series tasks, inputs are sequences of 3 interferograms sharing the same primary acquisition date but different secondary dates, ordered chronologically by secondary date.

**Output**: For classification: a binary label (deformation vs. no deformation). For segmentation: a binary mask representing the union of deformation areas across the input sequence.

## Scoring recipe

```python
# Classification F1
tp = sum((pred == 1) & (gold == 1))
fp = sum((pred == 1) & (gold == 0))
fn = sum((pred == 0) & (gold == 1))
prec = tp / (tp + fp) if (tp + fp) > 0 else 0
rec = tp / (tp + fn) if (tp + fn) > 0 else 0
f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0

# Segmentation IoU
intersection = sum((pred_mask == 1) & (gold_mask == 1))
union = sum((pred_mask == 1) | (gold_mask == 1))
iou = intersection / union if union > 0 else 0
```

## Common pitfalls

- Class imbalance is addressed via undersampling negatives to match positives during training, which may skew evaluation if not accounted for.
- Temporal split is used (2014-2019 train, 2019 val, 2020-2021 test) rather than random splitting, making results non-transferable to other time periods.
- Time-series labels are aggregated: a sequence is positive if any product shows deformation, and the segmentation mask is the union of all sequence masks.
- Models are evaluated with and without auxiliary atmospheric variables, significantly impacting performance.

## Evidence (verbatim from paper)

> To enable a fair comparison of future methods for InSAR based volcanic unrest detection, we provide the first benchmark on Hephaestus Minicubes. This benchmark is designed to serve as a strong baseline across two fundamental tasks: binary ground deformation classification and semantic segmentation. ... In Tabs.[3] and[4], we present the classification and segmentation results, respectively, reporting Precision, Recall, F1-score, and Area Under the Receiver Operating Characteristic curve (AUROC) for the classification task, and Precision, Recall, F1-score, and Intersection over Union (IoU) for the segmentation task.

## Citation

```bibtex
@misc{papadopoulos2025hephaestus,
  title={Hephaestus Minicubes: A Global, Multi-Modal Dataset for Volcanic Unrest Monitoring},
  author={Papadopoulos et al. (2025)},
  year={2025},
  note={arXiv:2505.17782}
}
```

- arXiv: 2505.17782