Facies Classification Eval

---
name: facies-classification-eval
description: This benchmark evaluates machine learning models on 3D seismic facies classification, a task critical for geological interpretation. It probes a model's ability to accurately segment and label distinct geological strata from 3D seismic data using both local patch-based and global section-based contextual information. Use when the user wants to benchmark on F3 Block, or asks about evaluating this task. Reports MCA.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1901.07659
  bibtex_key: alaudah2019machine
  confidence: high
---

# facies-classification-eval

> A Machine Learning Benchmark for Facies Classification — Alaudah et al. (2019) (arXiv:1901.07659, 2019)

## What this evaluates

This benchmark evaluates machine learning models on 3D seismic facies classification, a task critical for geological interpretation. It probes a model's ability to accurately segment and label distinct geological strata from 3D seismic data using both local patch-based and global section-based contextual information.

## Datasets

- **F3 Block** — total ?; splits: train (-1), test #1 (-1), test #2 (-1)

## Metrics

- `PA` — range: [0, 1]
  - Pixel Accuracy: the overall fraction of correctly classified pixels across the entire test set.
- `Class Accuracy` — range: [0, 1]
  - Per-class pixel accuracy: the fraction of correctly classified pixels for each of the six geological facies classes individually.
- `MCA` **(primary)** — range: [0, 1]
  - Macro Class Accuracy: the unweighted mean of the per-class accuracies across all six facies classes.
- `FWIU` — range: [0, 1]
  - Fuzzy Weighted Intersection over Union: a segmentation overlap metric that measures the intersection between predicted and ground truth masks, weighted by class frequency.

## Input / output format

**Input**: 3D seismic data represented as 2D inline/crossline sections or extracted spatial patches, optionally with depth context.

**Output**: A pixel-wise classification label for each input pixel, assigning one of six geological facies classes (Zechstein, Scruff, Rijnland/Chalk, Lower North Sea, Middle North Sea, Upper North Sea).

## Scoring recipe

```python
def compute_metrics(pred, gt):
    pa = (pred == gt).mean()
    class_accs = []
    for c in range(6):
        mask = (gt == c)
        if mask.sum() > 0:
            class_accs.append((pred[mask] == gt[mask]).mean())
        else:
            class_accs.append(0.0)
    mca = np.mean(class_accs)
    # FWIU computed as weighted IoU; exact fuzzy weighting formula not specified in text
    fwiu = compute_fuzzy_weighted_iou(pred, gt)
    return {'PA': pa, 'Class Accuracy': class_accs, 'MCA': mca, 'FWIU': fwiu}
```

## Common pitfalls

- High overall PA can mask poor performance on minority classes (e.g., Zechstein, Scruff) due to severe class imbalance in the dataset.
- Patch-based models often misclassify classes that appear at different depths because they lack the broader spatial and contextual information that section-based models capture.

## Evidence (verbatim from paper)

> Table 2 summarizes the objective results for all the models that we have tested on both test sets, while Figure 9 shows inline 200 of test set #1 labeled using the six different models we have tested. The MCA score shows a 15% improvement of the section-based baseline model vs. the patch-based model.

## Citation

```bibtex
@misc{alaudah2019machine,
  title={A Machine Learning Benchmark for Facies Classification},
  author={Alaudah et al. (2019)},
  year={2019},
  note={arXiv:1901.07659}
}
```

- arXiv: 1901.07659