Cracknex Eval

---
name: cracknex-eval
description: Evaluates few-shot crack segmentation performance under low-light conditions using illumination-invariant features. It tests the model's ability to generalize from well-illuminated support images to unseen low-light query images in both synthetic and real-world scenarios. Use when the user wants to benchmark on ll_CrackSeg9k, LCSD, or asks about evaluating this task. Reports mIOU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2403.03063
  bibtex_key: yao2024cracknex
  confidence: high
---

# cracknex-eval

> CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections — Yao et al. (2024) (arXiv:2403.03063, 2024)

## What this evaluates

Evaluates few-shot crack segmentation performance under low-light conditions using illumination-invariant features. It tests the model's ability to generalize from well-illuminated support images to unseen low-light query images in both synthetic and real-world scenarios.

## Datasets

- **ll_CrackSeg9k** — total 10500; splits: train (9000), test (1500)
- **LCSD** — total 143; splits: train (102), test (41)

## Metrics

- `mIOU` **(primary)** — range: [0, 1]
  - Mean Intersection over Union: average IoU across all test images, where IoU = intersection of predicted and ground truth masks / union of predicted and ground truth masks.

## Input / output format

**Input**: Support set of K well-illuminated crack images with binary masks, and a query image (synthetic or real low-light) to be segmented.

**Output**: Binary pixel-wise segmentation mask for the query image.

## Scoring recipe

```python
def compute_miou(preds, gts):
    ious = []
    for pred, gt in zip(preds, gts):
        intersection = np.logical_and(pred, gt).sum()
        union = np.logical_or(pred, gt).sum()
        if union == 0:
            ious.append(1.0)
        else:
            ious.append(intersection / union)
    return np.mean(ious)
```

## Common pitfalls

- The ll_CrackSeg9k test set uses synthetic low-light images generated via Restormer, not real low-light photos, which may overestimate real-world performance.
- Few-shot evaluation requires reporting metrics separately for 1-shot and 5-shot settings; averaging them without distinction misrepresents performance.
- Training uses random horizontal flipping augmentation, but evaluation must be performed on original unaugmented images to match the reported protocol.

## Evidence (verbatim from paper)

> TABLE I: Baseline comparisons on the ll_CrackSeg9k and LCSD dataset in terms of mIOU↑ ... We select 9000 crack images from CrackSeg9k as our training set and another 1500 crack images as our test set. ... LCSD, with 102 well-illuminated crack images as the training set and 41 low-light crack images as the test set ... We further annotate each crack image pixel-wise and generate a binary label.

## Citation

```bibtex
@misc{yao2024cracknex,
  title={CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections},
  author={Yao et al. (2024)},
  year={2024},
  note={arXiv:2403.03063}
}
```

- arXiv: 2403.03063