Miqa Eval

---
name: miqa-eval
description: Evaluates how image degradations impact machine vision system (MVS) performance rather than human perception. It measures the correlation between predicted image quality scores and ground-truth machine task metrics (accuracy and consistency) across classification, detection, and segmentation tasks. Use when the user wants to benchmark on MIQD-2.5M, or asks about evaluating this task. Reports SRCC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.19850
  bibtex_key: wang2025miqa
  confidence: high
---

# miqa-eval

> Image Quality Assessment for Machines: Paradigm, Large-scale Database, and Models — Wang et al. (2025) (arXiv:2508.19850, 2025)

## What this evaluates

Evaluates how image degradations impact machine vision system (MVS) performance rather than human perception. It measures the correlation between predicted image quality scores and ground-truth machine task metrics (accuracy and consistency) across classification, detection, and segmentation tasks.

## Datasets

- **MIQD-2.5M** — total 2500000; splits: train (-1), val (-1); repo https://github.com/XiaoqiWang/MIQA

## Metrics

- `SRCC` **(primary)** — range: [0, 1]
  - Spearman rank correlation coefficient measuring monotonic relationship between predicted scores and ground-truth machine performance scores.
- `PLCC` — range: [0, 1]
  - Pearson linear correlation coefficient measuring linear relationship between predicted scores and ground-truth machine performance scores.
- `KRCC` — range: [0, 1]
  - Kendall’s rank correlation coefficient measuring ordinal association between predicted scores and ground-truth machine performance scores.
- `RMSE` — range: other
  - Root mean square error quantifying the average magnitude of prediction errors relative to ground-truth scores.

## Input / output format

**Input**: Degraded image passed through the MIQA model.

**Output**: A single continuous predicted quality score.

## Scoring recipe

```python
def compute_metrics(pred_scores, gold_scores):
    # Align predictions to ground-truth via 5-parameter logistic regression on validation set
    aligned_pred = nonlinear_regression_align(pred_scores, gold_scores)
    srcc = spearmanr(aligned_pred, gold_scores).correlation
    plcc = pearsonr(aligned_pred, gold_scores).correlation
    krcc = kendallr(aligned_pred, gold_scores).correlation
    rmse = sqrt(mean_squared_error(gold_scores, aligned_pred))
    return {'SRCC': srcc, 'PLCC': plcc, 'KRCC': krcc, 'RMSE': rmse}
```

## Common pitfalls

- Applying HVS-based metrics (PSNR, SSIM) directly without the required nonlinear regression alignment to ground-truth machine performance scores.
- Failing to use stratified sampling for class imbalance, leading to content leakage or skewed validation distributions.
- Confusing 'consistency scores' (robustness to degradation) with 'accuracy scores' (task performance drop) when interpreting results.

## Evidence (verbatim from paper)

> The performance was evaluated using multiple correlation metrics: SRCC, PLCC, and Kendall’s Rank Correlation Coefficient (KRCC), all ranging from 0 to 1 with higher values indicating better performance, alongside RMSE to quantify prediction accuracy across diverse measurement methods.

## Citation

```bibtex
@misc{wang2025miqa,
  title={Image Quality Assessment for Machines: Paradigm, Large-scale Database, and Models},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2508.19850}
}
```

- arXiv: 2508.19850