Synthetic Geology Eval

---
name: synthetic-geology-eval
description: Evaluates a flow matching generative model's ability to produce realistic 3D subsurface geological models, both unconditionally and conditioned on sparse borehole data. It probes the model's capacity for geological interpolation, structural feature reconstruction, and probabilistic uncertainty estimation. Use when the user wants to benchmark on Synthetic Geology / StructuralGeo Dataset, or asks about evaluating this task. Reports probabilistic confidence intervals.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.11164
  bibtex_key: [REDACTED]
  confidence: medium
---

# synthetic-geology-eval

> Synthetic Geology -- Structural Geology Meets Deep Learning — Ghyselincks et al. (2025) (arXiv:2506.11164, 2025)

## What this evaluates

Evaluates a flow matching generative model's ability to produce realistic 3D subsurface geological models, both unconditionally and conditioned on sparse borehole data. It probes the model's capacity for geological interpolation, structural feature reconstruction, and probabilistic uncertainty estimation.

## Datasets

- **Synthetic Geology / StructuralGeo Dataset** — total ?; splits: train (-1), test (-1); repo https://github.com/eldadHaber/StructuralGeo

## Metrics

- `probabilistic confidence intervals` **(primary)** — range: percent
  - Voxel-wise probability estimate derived from an ensemble of 100 conditional reconstructions. Evaluated by plotting contour shapes of probabilistic confidence intervals (e.g., 5%, 33%, 62%, 90%) to assess extrapolation range and feature reconstruction accuracy.

## Input / output format

**Input**: Unconditional: Gaussian noise. Conditional: Sparse surface data and irregularly sampled borehole lithology measurements.

**Output**: 3D categorical voxel grid (e.g., 64x64x64 resolution) representing lithological classes, decoded from continuous flow space.

## Scoring recipe

```python
predictions = []
for seed in range(100):
    noise = sample_gaussian_noise()
    model_output = flow_matching_model(condition=surface_data + boreholes, seed=seed)
    predictions.append(model_output)
prob_map = mean(predictions, axis=0)
reconstructions = {threshold: prob_map > threshold for threshold in [0.05, 0.33, 0.62, 0.90]}
return reconstructions
```

## Common pitfalls

- Relies entirely on qualitative visual inspection rather than standard quantitative benchmark metrics.
- Conditional generation evaluation uses a single unseen 'Geomodel' as ground truth, limiting statistical generalizability.
- Probabilistic confidence intervals are assessed visually via contour plots without numerical validation against independent test sets.

## Evidence (verbatim from paper)

> A large ensemble of samples can be used as a probability estimate for the likelihood of any individual voxel being part of the larger dike structure. A pattern emerges when plotting the extent of the planar dike based on the contour shape of probabilistic confidence intervals. Shown in Figure[9], a lower confidence interval corresponds to a larger and more aggressive extrapolation range from the existing boreholes.

## Citation

```bibtex
@misc{ghyselincks2025syntheticgeology,
  title={Synthetic Geology -- Structural Geology Meets Deep Learning},
  author={Ghyselincks et al. (2025)},
  year={2025},
  note={arXiv:2506.11164}
}
```

- arXiv: 2506.11164