Deeptoponet Eval

---
name: deeptoponet-eval
description: Evaluates a deep learning model's ability to reconstruct subglacial bed topography by fusing sparse radar ice thickness measurements with high-resolution surface elevation and ice dynamics data. It probes spatial interpolation accuracy, structural preservation of terrain features, and robustness in data-scarce glacial regions. Use when the user wants to benchmark on Upernavik Isstrøm, or asks about evaluating this task. Reports MAE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.23980
  bibtex_key: tama2025deeptoponet
  confidence: high
---

# deeptoponet-eval

> DeepTopoNet: A Framework for Subglacial Topography Estimation on the Greenland Ice Sheets — Bayu Adhi Tama et al. (2025) (arXiv:2505.23980, 2025)

## What this evaluates

Evaluates a deep learning model's ability to reconstruct subglacial bed topography by fusing sparse radar ice thickness measurements with high-resolution surface elevation and ice dynamics data. It probes spatial interpolation accuracy, structural preservation of terrain features, and robustness in data-scarce glacial regions.

## Datasets

- **Upernavik Isstrøm** — total ?; splits: train (-1), val (-1); repo https://github.com/bayuat/DeepTopoNet

## Metrics

- `MAE` **(primary)** — range: other
  - Mean absolute error between predicted and reference bed elevation values across all grid cells. Lower is better.
- `RMSE` — range: other
  - Root mean square error of elevation predictions. Lower is better.
- `R²` — range: other
  - Coefficient of determination measuring the proportion of variance in bed elevation explained by the model. Higher is better.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index Measure evaluating luminance, contrast, and structural correlation between predicted and reference grids. Higher is better.
- `PSNR` — range: other
  - Peak Signal-to-Noise Ratio in decibels, derived from the mean squared error of the elevation grids. Higher is better.
- `TRI Relative Difference` — range: percent
  - Relative Difference (%) = |TRI_predicted - TRI_BedMachine| / TRI_predicted × 100, where TRI is the root mean square of elevation differences between a cell and its 8 neighbors. Lower is better.

## Input / output format

**Input**: Overlapping 16×16 grid patches containing four channels: surface elevation, x/y ice velocity components, surface thickening/thinning rate, and surface mass balance.

**Output**: A 2D grid map of predicted subglacial bed topography (elevation values in meters) matching the spatial resolution of the input patches.

## Scoring recipe

```python
def compute_metrics(pred, gold):
    mae = np.mean(np.abs(pred - gold))
    rmse = np.sqrt(np.mean((pred - gold)**2))
    ssim = compute_ssim(pred, gold)
    psnr = 10 * np.log10(max_val**2 / np.mean((pred - gold)**2))
    tri_pred = np.sqrt(np.mean((pred[1:]-pred[:-1])**2 + (pred[:,1:]-pred[:,:-1])**2))
    tri_gold = np.sqrt(np.mean((gold[1:]-gold[:-1])**2 + (gold[:,1:]-gold[:,:-1])**2))
    tri_rel_diff = np.abs(tri_pred - tri_gold) / tri_pred * 100
    return mae, rmse, ssim, psnr, tri_rel_diff
```

## Common pitfalls

- The 80/20 train/val split is not explicitly stated as spatial, risking data leakage given the overlapping 16×16 patch training strategy and spatially correlated ice dynamics data.
- The TRI relative difference formula divides by TRI_predicted, which can produce unstable or misleading percentages when predicted ruggedness approaches zero.
- R² scores can be negative (as shown in Table 2 for baselines), indicating the model performs worse than a horizontal mean baseline, which is often misinterpreted as a standard bounded accuracy metric.

## Evidence (verbatim from paper)

> For the evaluation of full grid predictions, several standard metrics are employed to comprehensively measure the model’s accuracy and structural fidelity such as MAE, RMSE, Structural Similarity Index Measure (SSIM)*(Wang et al., [2004])*, and Peak Signal-to-Noise Ratio (PSNR)*(Hore and Ziou, [2010])*. Moreover, a domain-specific metric, the Terrain Ruggedness Index (TRI) *(Reily Shawn et al., [1999])*, is used as a quantitative measure to evaluate local variations in terrain elevation by capturing the differences between a cell and its neighboring elevation values.

## Citation

```bibtex
@misc{tama2025deeptoponet,
  title={DeepTopoNet: A Framework for Subglacial Topography Estimation on the Greenland Ice Sheets},
  author={Bayu Adhi Tama et al. (2025)},
  year={2025},
  note={arXiv:2505.23980}
}
```

- arXiv: 2505.23980