Openfwi Fwi Eval

---
name: openfwi-fwi-eval
description: Evaluates deep learning models for seismic full-waveform inversion (FWI) by predicting subsurface velocity models from seismic wavefield data. It probes the model's ability to generalize across varying geological complexities and out-of-distribution scenarios using parameter-efficient fine-tuning. Use when the user wants to benchmark on OpenFWI, or asks about evaluating this task. Reports SSIM.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2412.19510
  bibtex_key: ghosal2024parameter
  confidence: high
---

# openfwi-fwi-eval

> Parameter Efficient Fine-Tuning for Deep Learning-Based Full-Waveform Inversion — Ghosal et al. (2024) (arXiv:2412.19510, 2024)

## What this evaluates

Evaluates deep learning models for seismic full-waveform inversion (FWI) by predicting subsurface velocity models from seismic wavefield data. It probes the model's ability to generalize across varying geological complexities and out-of-distribution scenarios using parameter-efficient fine-tuning.

## Datasets

- **OpenFWI** — total ?; splits: train (-1), test (-1); repo https://github.com/Kaustav546/FWI-PEFT.git

## Metrics

- `MAE` — range: other
  - Mean Absolute Error: average of absolute differences between predicted and ground truth velocity values.
- `RMSE` — range: other
  - Root Mean Squared Error: square root of the average of squared differences between predicted and ground truth velocity values.
- `SSIM` **(primary)** — range: [-1, 1]
  - Structural Similarity Index Measure: evaluates perceived change in structural information between predicted and reference velocity maps, ranging from -1 to 1 where 1 indicates perfect similarity.

## Input / output format

**Input**: Seismic wavefield recordings (shot gathers or time-series data)

**Output**: Predicted 2D subsurface velocity model/map

## Scoring recipe

```python
def compute_metrics(pred, gt):
    mae = np.mean(np.abs(pred - gt))
    rmse = np.sqrt(np.mean((pred - gt) ** 2))
    ssim = compute_ssim(pred, gt)  # Standard SSIM implementation
    return {'MAE': mae, 'RMSE': rmse, 'SSIM': ssim}
```

## Common pitfalls

- SSIM measures structural similarity rather than pixel-wise accuracy, so a high SSIM does not guarantee low MAE or RMSE.
- Performance heavily depends on dataset complexity (spatial information/gradient magnitude); models may underperform on simple geological structures while excelling on complex ones.
- Out-of-distribution (OOD) evaluation requires strict train/test splits where the test dataset has a significant statistical distribution shift from the training data.

## Evidence (verbatim from paper)

> Three evaluations used for this study are: MAE, RMSE, and SSIM.

## Citation

```bibtex
@misc{ghosal2024parameter,
  title={Parameter Efficient Fine-Tuning for Deep Learning-Based Full-Waveform Inversion},
  author={Ghosal et al. (2024)},
  year={2024},
  note={arXiv:2412.19510}
}
```

- arXiv: 2412.19510