Seismic Segmentation Al Eval

---
name: seismic-segmentation-al-eval
description: This evaluation probes a model's ability to perform semantic segmentation on 3D seismic data under an active learning regime. It measures how effectively a model generalizes to unseen geological volumes when trained on a sequentially selected subset of annotated sections, rather than a fixed passive dataset. Use when the user wants to benchmark on F3 benchmark, Parihaka, or asks about evaluating this task. Reports mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2406.05149
  bibtex_key: benkert2024atlas
  confidence: high
---

# seismic-segmentation-al-eval

> Effective Data Selection for Seismic Interpretation through Disagreement — Benkert et al. (2024) (arXiv:2406.05149, 2024)

## What this evaluates

This evaluation probes a model's ability to perform semantic segmentation on 3D seismic data under an active learning regime. It measures how effectively a model generalizes to unseen geological volumes when trained on a sequentially selected subset of annotated sections, rather than a fixed passive dataset.

## Datasets

- **F3 benchmark** — total ?; splits: train (-1), test (-1)
- **Parihaka** — total ?; splits: train (-1), test (-1)

## Metrics

- `mIoU` **(primary)** — range: [0, 1]
  - Mean Intersection-over-Union across all 6 facies classes. Calculated as the average of IoU (intersection over union) for each class.
- `class accuracy` — range: [0, 1]
  - Per-class pixel accuracy averaged across the 6 facies categories.

## Input / output format

**Input**: 3D seismic data sections/volumes (clipped to [-1500, 1500], depth sub-sampled by factor 3). Each instance is a spatial slice or volume section fed to a DeepLabV3-ResNet18 model.

**Output**: Pixel-wise segmentation mask assigning one of six facies categories to each voxel/pixel in the input section.

## Scoring recipe

```python
def compute_miou(predictions, ground_truth, num_classes=6):
    ious = []
    for c in range(num_classes):
        pred_c = (predictions == c)
        gt_c = (ground_truth == c)
        intersection = np.logical_and(pred_c, gt_c).sum()
        union = np.logical_or(pred_c, gt_c).sum()
        if union == 0:
            ious.append(0.0)
        else:
            ious.append(intersection / union)
    return np.mean(ious)
```

## Common pitfalls

- The Parihaka test set is not the original held-out test volume; it is a synthetic split of the original training volume because original test labels are unavailable.
- Training is halted dynamically when training mIoU reaches 0.9, rather than using a fixed number of epochs, which may bias convergence comparisons across acquisition functions.
- Active learning batch size is fixed at 2 sections per round, which is unusually small and may not reflect standard active learning protocols.

## Evidence (verbatim from paper)

> During each active learning round, we train the model until it achieves an at least an mean-intersection-over-union (mIoU) performance of 0.9 and log performance on the test volume in overall mIoU, as well as class accuracy.

## Citation

```bibtex
@misc{benkert2024atlas,
  title={Effective Data Selection for Seismic Interpretation through Disagreement},
  author={Benkert et al. (2024)},
  year={2024},
  note={arXiv:2406.05149}
}
```

- arXiv: 2406.05149