Dacl10k Eval

---
name: dacl10k-eval
description: Evaluates the ability of computer vision models to perform multi-label semantic segmentation for identifying and localizing various reinforced concrete defects on bridge infrastructure. It probes pixel-level classification accuracy across diverse, real-world damage types and structural components. Use when the user wants to benchmark on dacl10k, or asks about evaluating this task. Reports mean IoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2309.00460
  bibtex_key: flotzinger2023dacl10k
  confidence: high
---

# dacl10k-eval

> dacl10k: Benchmark for Semantic Bridge Damage Segmentation — Flotzinger et al. (2023) (arXiv:2309.00460, 2023)

## What this evaluates

Evaluates the ability of computer vision models to perform multi-label semantic segmentation for identifying and localizing various reinforced concrete defects on bridge infrastructure. It probes pixel-level classification accuracy across diverse, real-world damage types and structural components.

## Datasets

- **dacl10k** — total 9920; splits: test (-1)

## Metrics

- `mean IoU` **(primary)** — range: [0, 1]
  - Mean Intersection over Union calculated across all damage classes. For each class, IoU is the intersection of predicted and ground truth masks divided by their union, averaged over the dataset.
- `F1 Score` — range: [0, 1]
  - Harmonic mean of precision and recall computed per class and averaged. Precision is the ratio of true positive pixels to all predicted positive pixels, while recall is the ratio of true positive pixels to all actual positive pixels.

## Input / output format

**Input**: RGB images of bridge surfaces captured via UAV or smartphone, containing one or more concrete defects.

**Output**: Multi-label pixel-wise segmentation masks assigning each pixel to one of the defined damage classes or background.

## Scoring recipe

```python
def compute_miou(pred_masks, gt_masks, num_classes):
    ious = []
    for c in range(num_classes):
        pred_c = (pred_masks == c)
        gt_c = (gt_masks == c)
        intersection = np.logical_and(pred_c, gt_c).sum()
        union = np.logical_or(pred_c, gt_c).sum()
        if union > 0:
            ious.append(intersection / union)
        else:
            ious.append(0.0)
    return np.mean(ious)
```

## Common pitfalls

- Hollowareas are not visually recognizable defects but are annotated as yellow/red/blue chalk markings made during hands-on inspections, requiring models to learn annotation artifacts rather than physical damage.
- Significant class imbalance exists, with some classes like WConcor having only 33 images while others like Spalling have over 1000, skewing average metrics if not handled properly.
- Multi-label nature means multiple defects can overlap or co-occur in a single image, complicating pixel-wise boundary delineation.

## Evidence (verbatim from paper)

> Baseline models using deep semantic segmentation architectures achieve a mean IoU of 0.42 on the test set, demonstrating the dataset's utility for evaluating automated damage recognition in civil infrastructure. In Table 6 we display additional metrics from the auxiliary head of the best model on the test split.

## Citation

```bibtex
@misc{flotzinger2023dacl10k,
  title={dacl10k: Benchmark for Semantic Bridge Damage Segmentation},
  author={Flotzinger et al. (2023)},
  year={2023},
  note={arXiv:2309.00460}
}
```

- arXiv: 2309.00460