Qata Cov19 Eval

---
name: qata-cov19-eval
description: Evaluates deep learning models for COVID-19 infected region segmentation and binary detection on chest X-ray images. It probes the model's ability to localize pathological regions at the pixel level and classify whole images as positive or negative for infection. Use when the user wants to benchmark on QaTa-COV19, or asks about evaluating this task. Reports F1-Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2009.12698
  bibtex_key: degerli2020covid19infectionmap
  confidence: high
---

# qata-cov19-eval

> COVID-19 Infection Map Generation and Detection from Chest X-Ray Images — Degerli et al. (2020) (arXiv:2009.12698, 2020)

## What this evaluates

Evaluates deep learning models for COVID-19 infected region segmentation and binary detection on chest X-ray images. It probes the model's ability to localize pathological regions at the pixel level and classify whole images as positive or negative for infection.

## Datasets

- **QaTa-COV19** — total ?; splits: Group-I (15495), Group-II (-1)

## Metrics

- `F1-Score` **(primary)** — range: percent
  - Harmonic mean of Precision and Sensitivity: F1 = 2 * (Precision * Sensitivity) / (Precision + Sensitivity).
- `Sensitivity` — range: percent
  - Recall: TP / (TP + FN).
- `Specificity` — range: percent
  - TN / (TN + FP).
- `Precision` — range: percent
  - TP / (TP + FP).
- `Accuracy` — range: percent
  - (TP + TN) / (TP + TN + FP + FN).
- `F2-Score` — range: percent
  - Weighted F-score emphasizing FN minimization: F2 = 5 * (Precision * Sensitivity) / (4 * Precision + Sensitivity).

## Input / output format

**Input**: Chest X-ray (CXR) images resized to 224x224 pixels.

**Output**: For segmentation: pixel-level binary mask (infected region vs background). For detection: binary class label (COVID-19 positive vs control negative).

## Scoring recipe

```python
def compute_metrics(tp, tn, fp, fn):
    sensitivity = tp / (tp + fn)
    specificity = tn / (tn + fp)
    precision = tp / (tp + fp)
    accuracy = (tp + tn) / (tp + tn + fp + fn)
    f1 = 2 * precision * sensitivity / (precision + sensitivity)
    f2 = 5 * precision * sensitivity / (4 * precision + sensitivity)
    return sensitivity, specificity, precision, accuracy, f1, f2
```

## Common pitfalls

- Significant class imbalance between COVID-19 and control samples requires explicit data augmentation (shifting, rotation) to balance training sets.
- Performance is reported as mean ± 95% confidence interval over 5-fold cross-validation, not single-run accuracy.
- Encoder layers may be frozen or unfrozen during training, which drastically changes the reported metric values.

## Evidence (verbatim from paper)

> The standard performance evaluation metrics are defined as follows: Sensitivity = TP/(TP+FN), Specificity = TN/(TN+FP), Precision = TP/(TP+FP), Accuracy = (TP+TN)/(TP+TN+FP+FN), F(β) = (1+β²)(Precision×Sensitivity)/(β²×Precision+Sensitivity). The F1-Score is calculated with β=1, which is the harmonic average of precision and sensitivity. The F2-score is calculated with β=2, which emphasizes FN minimization over FPs.

## Citation

```bibtex
@misc{degerli2020covid19infectionmap,
  title={COVID-19 Infection Map Generation and Detection from Chest X-Ray Images},
  author={Degerli et al. (2020)},
  year={2020},
  note={arXiv:2009.12698}
}
```

- arXiv: 2009.12698