Orsi Sod Eval

---
name: orsi-sod-eval
description: This benchmark evaluates optical remote sensing salient object detection models by measuring their ability to accurately segment prominent objects from complex, cluttered backgrounds. It probes structural consistency, boundary precision, and error magnitude across varying object scales and scene complexities. Use when the user wants to benchmark on ORSSD, EORSSD, ORSI-4199, or asks about evaluating this task. Reports maximum F-measure ($F_{\beta}^{max}$).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.28584
  bibtex_key: chen2026orsiflow
  confidence: high
---

# orsi-sod-eval

> ORSIFlow: Saliency-Guided Rectified Flow for Optical Remote Sensing Salient Object Detection — Haojing Chen et al. (arXiv:2603.28584, 2026)

## What this evaluates

This benchmark evaluates optical remote sensing salient object detection models by measuring their ability to accurately segment prominent objects from complex, cluttered backgrounds. It probes structural consistency, boundary precision, and error magnitude across varying object scales and scene complexities.

## Datasets

- **ORSSD** — total 800; splits: train (600), test (200)
- **EORSSD** — total 2000; splits: train (1400), test (600)
- **ORSI-4199** — total 4199; splits: train (2000), test (2199)

## Metrics

- `S-measure ($S_{\alpha}$)` — range: [0, 1]
  - Combines region-level and object-level similarity to evaluate structural consistency between prediction and ground truth.
- `maximum F-measure ($F_{\beta}^{max}$)` **(primary)** — range: [0, 1]
  - Computes the harmonic mean of precision and recall at the optimal threshold across a range of values, emphasizing the precision-recall tradeoff.
- `maximum E-measure ($E_{\xi}^{max}$)` — range: [0, 1]
  - Aligns local pixel-level and global image-level features to evaluate enhanced structural similarity.
- `Mean Absolute Error (MAE)` — range: [0, 1]
  - Calculates the mean of absolute pixel-wise differences between the predicted saliency map and the ground truth mask.

## Input / output format

**Input**: Optical remote sensing images resized to 352×352 pixels.

**Output**: Pixel-wise saliency masks (soft or binary maps) matching the input resolution.

## Scoring recipe

```python
def compute_metrics(pred, gt):
    # pred, gt: float tensors [H, W] in [0, 1]
    mae = np.mean(np.abs(pred - gt))
    thresholds = np.linspace(0, 1, 255)
    f_scores = []
    for t in thresholds:
        p = (pred > t).astype(float)
        prec = np.sum(p * gt) / (np.sum(p) + 1e-8)
        rec = np.sum(p * gt) / (np.sum(gt) + 1e-8)
        f = (1.5 * prec * rec) / (0.5 * prec + rec + 1e-8)
        f_scores.append(f)
    f_max = max(f_scores)
    return mae, f_max
```

## Common pitfalls

- Failing to resize inputs to the fixed 352×352 resolution used in the paper, which significantly alters metric values.
- Using a single fixed threshold for F-measure instead of computing the maximum over a range of thresholds ($F_{\beta}^{max}$).
- Not using the official open-source code and recommended settings for baseline comparisons, leading to unfair performance gaps.

## Evidence (verbatim from paper)

> We employed four widely used metrics to quantitatively evaluate the performance of all the methods, including S-measure ($S_{a}$), Mean Absolute Error (MAE), maximum E-measure ($E_{\xi}^{max}$), and maximum F-measure ($F_{\beta}^{max}$).

## Citation

```bibtex
@misc{chen2026orsiflow,
  title={ORSIFlow: Saliency-Guided Rectified Flow for Optical Remote Sensing Salient Object Detection},
  author={Haojing Chen et al.},
  year={2026},
  note={arXiv:2603.28584}
}
```

- arXiv: 2603.28584