Ocean Workbench Eval

---
name: ocean-workbench-eval
description: Evaluates SAR foundation models on a suite of ocean observation tasks, including geophysical pattern classification, continuous regression for wave height and wind parameters, and iceberg object detection. It tests both zero-shot feature transferability and fine-tuning adaptability across diverse geophysical benchmarks. Use when the user wants to benchmark on Ocean Workbench, or asks about evaluating this task. Reports TenGeoP accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.07392
  bibtex_key: [REDACTED]
  confidence: high
---

# ocean-workbench-eval

> OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation — Tuel et al. (2026) (arXiv:2601.07392, 2026)

## What this evaluates

Evaluates SAR foundation models on a suite of ocean observation tasks, including geophysical pattern classification, continuous regression for wave height and wind parameters, and iceberg object detection. It tests both zero-shot feature transferability and fine-tuning adaptability across diverse geophysical benchmarks.

## Datasets

- **Ocean Workbench** — total ?; splits: test (-1)

## Metrics

- `TenGeoP accuracy` **(primary)** — range: percent
  - Percentage of correctly classified geophysical patterns. Computed as correct predictions divided by total samples, reported in percent.
- `SWH regression error` — range: other
  - Regression error for significant wave height prediction. Lower values indicate better performance, reported in meters.
- `Wspd regression error` — range: other
  - Regression error for wind speed estimation. Lower values indicate better performance, reported in meters per second.
- `Wdir regression error` — range: other
  - Regression error for wind direction estimation. Lower values indicate better performance, reported in degrees.
- `F1-score` — range: [0, 1]
  - Harmonic mean of precision and recall for iceberg detection, computed at an IoU threshold of 0.1 and a confidence score threshold of 0.5.

## Input / output format

**Input**: Sentinel-1 Wave Mode SAR imagery with calibrated σ⁰ backscatter. Models receive raw image patches or full scenes depending on the backbone architecture.

**Output**: Zero-shot: image-level embeddings (384-2048 dimensions). Fine-tuning: class labels for TenGeoP, scalar regression values for SWH/Wspd/Wdir, and bounding boxes with confidence scores for iceberg detection.

## Scoring recipe

```python
def evaluate(predictions, gold, mode='zero-shot', task=None):
    if mode == 'zero-shot':
        return kNN_classify_or_regress(predictions, gold, k=5)
    elif mode == 'fine-tune':
        if task == 'TenGeoP':
            return accuracy(predictions, gold)
        elif task in ['SWH', 'Wspd', 'Wdir']:
            return rmse(gold, predictions)
        elif task == 'YOLOIB':
            return compute_f1(predictions, gold, iou_thresh=0.1, score_thresh=0.5)
```

## Common pitfalls

- TerraMind outputs feature maps rather than image-level embeddings; global max averaging must be applied to the final feature map to match reported performance.
- Zero-shot kNN evaluation is explicitly excluded for wind direction and iceberg detection benchmarks due to architectural incompatibility.
- Fine-tuning stability varies significantly across models; some baselines (e.g., TerraMind on YOLOIB) fail to converge or lack available code, yielding unreliable results marked with asterisks.

## Evidence (verbatim from paper)

> We record the F1-score on predicted boxes, using an IoU threshold of 0.1 and score threshold of 0.5. In zero-shot mode, we apply kNN classification and regression to the image features directly produced by the models (excluding the wind direction and iceberg benchmarks, for which this is not possible).

## Citation

```bibtex
@misc{tuel2026oceansar2,
  title={OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation},
  author={Tuel et al. (2026)},
  year={2026},
  note={arXiv:2601.07392}
}
```

- arXiv: 2601.07392