Covid19 Xray Classification Eval

---
name: covid19-xray-classification-eval
description: This evaluation probes a model's ability to classify chest X-rays as COVID-19 positive or negative using a cross-modal distillation setup where CT images are only used during training. It specifically tests the robustness of transfer learning under extremely small, patient-level paired cohorts and prevalence-heavy validation splits. Use when the user wants to benchmark on COVID-19 Image Data Collection, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.29167
  bibtex_key: ma2026cttoxraydistillation
  confidence: high
---

# covid19-xray-classification-eval

> CT-to-X-ray Distillation Under Tiny Paired Cohorts: An Evidence-Bounded Reproducible Pilot Study — Bo Ma et al. (2026) (arXiv:2603.29167, 2026)

## What this evaluates

This evaluation probes a model's ability to classify chest X-rays as COVID-19 positive or negative using a cross-modal distillation setup where CT images are only used during training. It specifically tests the robustness of transfer learning under extremely small, patient-level paired cohorts and prevalence-heavy validation splits.

## Datasets

- **COVID-19 Image Data Collection** — total 783; splits: train (628), val (155)

## Metrics

- `Accuracy` **(primary)** — range: [0, 1]
  - Fraction of correctly classified instances out of the total validation set. Computed as (TP+TN)/(TP+TN+FP+FN).
- `Macro-F1` — range: [0, 1]
  - Unweighted mean of recall or precision for each class, averaged across the two classes (COVID-19 and non-COVID).
- `Balanced Accuracy` — range: [0, 1]
  - Average of recall obtained on each class, computed as (TPR+TNR)/2, which mitigates class imbalance effects.
- `Specificity` — range: [0, 1]
  - True negative rate, calculated as TN/(TN+FP). Measures the proportion of actual negatives correctly identified.
- `MCC` — range: [-1, 1]
  - Matthews Correlation Coefficient, a correlation coefficient between observed and predicted binary classifications. Ranges from -1 to +1, with +1 representing perfect prediction.
- `PR-AUC` — range: [0, 1]
  - Area under the Precision-Recall curve, computed by integrating precision over recall thresholds. Particularly informative for imbalanced datasets.

## Input / output format

**Input**: 128×128 grayscale chest X-ray images. For cross-modal experiments, CT images are provided only during teacher training, not at inference.

**Output**: Binary class label (COVID-19 vs. non-COVID) or probability scores for the positive class.

## Scoring recipe

```python
import numpy as np
from sklearn.metrics import accuracy_score, f1_score, balanced_accuracy_score, specificity_score, matthews_corrcoef, precision_recall_curve, auc

def compute_metrics(y_true, y_pred, y_prob=None):
    acc = accuracy_score(y_true, y_pred)
    macro_f1 = f1_score(y_true, y_pred, average='macro')
    bal_acc = balanced_accuracy_score(y_true, y_pred)
    spec = specificity_score(y_true, y_pred)
    mcc = matthews_corrcoef(y_true, y_pred)
    if y_prob is not None:
        prec, rec, _ = precision_recall_curve(y_true, y_prob)
        pr_auc = auc(rec, prec)
    else:
        pr_auc = np.nan
    return {'Accuracy': acc, 'Macro-F1': macro_f1, 'Balanced Accuracy': bal_acc,
            'Specificity': spec, 'MCC': mcc, 'PR-AUC': pr_auc}
```

## Common pitfalls

- Validation splits are extremely small (4 images in the fixed split, 5–10 per Monte Carlo resample), making bootstrap confidence intervals and paired significance tests numerically unreliable.
- Splits are heavily prevalence-heavy with only one negative validation patient per resample, which artificially inflates accuracy and necessitates balanced accuracy, specificity, and MCC for fair assessment.
- Results are highly unstable across random seeds and resampling runs; single-run rankings or mean differences should not be interpreted as definitive performance claims.

## Evidence (verbatim from paper)

> The headline table intentionally keeps one non-paired reference row and then separates the shared paired fixed split below it. That paired fixed split contains only four validation patients and four X-ray images (three positive, one negative), so we keep the main-text table focused on accuracy, macro-F1, and balanced accuracy rather than on threshold-sensitive secondary metrics.

## Citation

```bibtex
@misc{ma2026cttoxraydistillation,
  title={CT-to-X-ray Distillation Under Tiny Paired Cohorts: An Evidence-Bounded Reproducible Pilot Study},
  author={Bo Ma et al. (2026)},
  year={2026},
  note={arXiv:2603.29167}
}
```

- arXiv: 2603.29167