Nr Iqa Eval

---
name: nr-iqa-eval
description: This benchmark evaluates no-reference image quality assessment (NR-IQA) models on their ability to predict human-perceived image quality without a pristine reference. It probes how well a model captures diverse authentic and synthetic distortions (e.g., blur, noise, exposure, haze) and maintains monotonic and linear correlation with crowd-sourced Mean Opinion Scores (MOS). Use when the user wants to benchmark on KonIQ-10k, LIVE Challenge, KADID-10k, TID2013, BIQ2021, IP102-IQA, or asks about evaluating this task. Reports SRCC, PLCC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.13112
  bibtex_key: aglin2026mmiqa
  confidence: high
---

# nr-iqa-eval

> A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging — Aglin et al. (2026) (arXiv:2604.13112, 2026)

## What this evaluates

This benchmark evaluates no-reference image quality assessment (NR-IQA) models on their ability to predict human-perceived image quality without a pristine reference. It probes how well a model captures diverse authentic and synthetic distortions (e.g., blur, noise, exposure, haze) and maintains monotonic and linear correlation with crowd-sourced Mean Opinion Scores (MOS).

## Datasets

- **KonIQ-10k** — total 10073; splits: test (10073)
- **LIVE Challenge** — total 1162; splits: test (1162)
- **KADID-10k** — total 10125; splits: test (10125)
- **TID2013** — total 3000; splits: test (3000)
- **BIQ2021** — total 12000; splits: test (12000)
- **IP102-IQA** — total 35000; splits: test (35000)

## Metrics

- `SRCC` **(primary)** — range: [-1, 1]
  - Spearman rank correlation coefficient measuring monotonicity between predicted scores and human MOS labels. Computed as 1 - (6 * sum(d_i^2)) / (n * (n^2 - 1)), where d_i is the rank difference between prediction and label.
- `PLCC` **(primary)** — range: [-1, 1]
  - Pearson linear correlation coefficient measuring linear agreement after predictions are mapped to the MOS scale. Requires fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before computing correlation.
- `Accuracy / Precision / Recall / F1` — range: [0, 1]
  - Classification metrics used for diagnostic validation on IP102-IQA. Accuracy is the fraction of correctly classified images. Precision, Recall, and F1 are computed per distortion class, with macro-averaged and weighted F1 variants to handle class imbalance.

## Input / output format

**Input**: Single RGB image (no reference image provided).

**Output**: A single continuous quality score representing predicted perceptual quality.

## Scoring recipe

```python
def compute_iqa_metrics(predictions, mos_labels, heldout_indices):
    # SRCC: Spearman rank correlation
    srcc = spearman_rank_correlation(predictions, mos_labels)
    
    # PLCC: Pearson correlation after 5-param logistic mapping
    heldout_preds = predictions[heldout_indices]
    heldout_mos = mos_labels[heldout_indices]
    logistic_params = fit_5param_logistic(heldout_preds, heldout_mos)
    mapped_preds = apply_logistic_mapping(predictions, logistic_params)
    plcc = pearson_correlation(mapped_preds, mos_labels)
    
    # Bootstrap resampling for 95% CI (100 iterations)
    bootstrap_means = []
    for _ in range(100):
        sample_idx = np.random.choice(len(predictions), replace=True)
        bootstrap_means.append(spearman_rank_correlation(predictions[sample_idx], mos_labels[sample_idx]))
    return srcc, plcc, np.mean(bootstrap_means)
```

## Common pitfalls

- PLCC must be computed after fitting a standard five-parameter logistic mapping on a held-out set; using a simple linear fit or skipping the held-out split will yield incorrect values.
- Reported SRCC/PLCC medians across standardized train/test partitions are used to reduce split randomness; evaluating on a single arbitrary split may produce non-comparable results.
- IP102-IQA lacks subjective MOS labels; its classification metrics are strictly for diagnostic validation of distortion cues, not for final quality ranking.

## Evidence (verbatim from paper)

> For the five public datasets, MM-IQA was evaluated using standard NR-IQA metrics. SRCC was computed between the model predictions and the provided subjective scores to assess monotonicity, and PLCC was computed after fitting a standard five-parameter logistic mapping on a held-out set to correct scale and bias before accuracy reporting.

## Citation

```bibtex
@misc{aglin2026mmiqa,
  title={A Lightweight Multi-Metric No-Reference Image Quality Assessment Framework for UAV Imaging},
  author={Aglin et al. (2026)},
  year={2026},
  note={arXiv:2604.13112}
}
```

- arXiv: 2604.13112