Texture Sam Eval

---
name: texture-sam-eval
description: Evaluates a model's ability to perform texture-aware segmentation by measuring how well it segments regions based on repeating texture patterns rather than semantic shape cues. It tests generalization on both synthetic texture-only images and natural images, while also checking for catastrophic forgetting on standard semantic benchmarks. Use when the user wants to benchmark on RWTD, STMD, ADE20K, or asks about evaluating this task. Reports mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.16540
  bibtex_key: cohen2025texturesam
  confidence: high
---

# texture-sam-eval

> TextureSAM: Towards a Texture Aware Foundation Model for Segmentation — Cohen et al. (2025) (arXiv:2505.16540, 2025)

## What this evaluates

Evaluates a model's ability to perform texture-aware segmentation by measuring how well it segments regions based on repeating texture patterns rather than semantic shape cues. It tests generalization on both synthetic texture-only images and natural images, while also checking for catastrophic forgetting on standard semantic benchmarks.

## Datasets

- **RWTD** — total ?; splits: test (-1)
- **STMD** — total ?; splits: test (-1)
- **ADE20K** — total ?; splits: val (-1)

## Metrics

- `mIoU` **(primary)** — range: [0, 1]
  - Mean Intersection over Union across all classes. Computed as the average of IoU (intersection area divided by union area) for each semantic class.
- `ARI` — range: [0, 1]
  - Adjusted Rand Index, a clustering similarity metric that measures the agreement between predicted segmentation labels and ground truth labels, adjusted for chance.
- `mIoU, Aggr.` — range: [0, 1]
  - Aggregated mIoU. Predicted masks are first grouped based on their overlap with ground truth regions to consolidate fragments, then mIoU is computed on the aggregated masks.

## Input / output format

**Input**: RGB image (natural or synthetic texture pattern)

**Output**: Set of predicted segmentation masks (pixel-wise labels or instance masks)

## Scoring recipe

```python
def compute_metrics(pred_masks, gt_masks):
    ious = []
    for class_id in unique_classes:
        pred = (pred_masks == class_id)
        gt = (gt_masks == class_id)
        inter = np.sum(pred & gt)
        union = np.sum(pred | gt)
        ious.append(inter / union if union > 0 else 1.0)
    miou = np.mean(ious)
    ari = adjusted_rand_score(gt_masks.flatten(), pred_masks.flatten())
    agg_pred = aggregate_masks(pred_masks, gt_masks)
    agg_ious = []
    for class_id in unique_classes:
        pred = (agg_pred == class_id)
        gt = (gt_masks == class_id)
        inter = np.sum(pred & gt)
        union = np.sum(pred | gt)
        agg_ious.append(inter / union if union > 0 else 1.0)
    miou_aggr = np.mean(agg_ious)
    return miou, ari, miou_aggr
```

## Common pitfalls

- Over-segmentation by shape-biased models (e.g., SAM-2) artificially lowers mIoU/ARI on texture-only data unless mask aggregation is applied.
- Inference parameters drastically affect results on texture-dominant datasets; using default parameters on SAM-2 yields poor performance compared to modified parameters.
- Aggregation step is essential for fair comparison on texture-defined regions; omitting it penalizes models that correctly identify regions but fragment them into multiple masks.

## Evidence (verbatim from paper)

> Evaluation is performed using two primary metrics: mean Intersection over Union (mIoU) and Adjusted Rand Index (ARI). To evaluate the overall segmentation quality of TextureSAM vs. the original SAM2 model, we apply mask aggregation, where predicted segmentation masks are grouped based on their overlap with ground truth regions.

## Citation

```bibtex
@misc{cohen2025texturesam,
  title={TextureSAM: Towards a Texture Aware Foundation Model for Segmentation},
  author={Cohen et al. (2025)},
  year={2025},
  note={arXiv:2505.16540}
}
```

- arXiv: 2505.16540