Icu Readmission Prediction Eval

---
name: icu-readmission-prediction-eval
description: Predicts the risk of a patient being readmitted to the ICU within 30 days of discharge using longitudinal electronic medical record (EMR) data. The task evaluates how well different deep learning architectures can model time-varying clinical events (diagnoses, procedures, medications, vital signs) alongside static demographic covariates to capture complex patient trajectories and risk factors. Use when the user wants to benchmark on MIMIC-III, or asks about evaluating this task. Reports AUROC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1905.08547
  bibtex_key: barbieri2019icubenchmark
  confidence: high
---

# icu-readmission-prediction-eval

> Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Risk — Barbieri et al. (2019) (arXiv:1905.08547, 2019)

## What this evaluates

Predicts the risk of a patient being readmitted to the ICU within 30 days of discharge using longitudinal electronic medical record (EMR) data. The task evaluates how well different deep learning architectures can model time-varying clinical events (diagnoses, procedures, medications, vital signs) alongside static demographic covariates to capture complex patient trajectories and risk factors.

## Datasets

- **MIMIC-III** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/sebbarb/timeAwareattention

## Metrics

- `AUROC` **(primary)** — range: [0, 1]
  - Area under the receiver operating characteristic curve. Measures the model's ability to discriminate between readmission and non-readmission across all possible classification thresholds.
- `Average Precision` — range: [0, 1]
  - Area under the precision-recall curve. Summarizes the trade-off between precision and recall for the positive class (readmission), particularly informative under class imbalance.
- `F1-Score` — range: [0, 1]
  - Harmonic mean of precision and recall calculated at a fixed classification threshold.
- `Sensitivity` — range: [0, 1]
  - True positive rate: proportion of actual readmissions correctly identified by the model.
- `Specificity` — range: [0, 1]
  - True negative rate: proportion of non-readmissions correctly identified by the model.

## Input / output format

**Input**: Per patient: a sequence of time-stamped clinical events (ICD-9 diagnosis codes, procedure codes, medications, vital signs) aggregated per ICU stay, alongside static demographic and admission covariates (e.g., age, gender, insurance, admission location).

**Output**: Binary label indicating whether the patient was readmitted to the ICU within 30 days of discharge (1 = readmitted, 0 = not readmitted).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred_prob, threshold=0.5):
    y_pred = (y_pred_prob >= threshold).astype(int)
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))
    tn = np.sum((y_true == 0) & (y_pred == 0))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    sensitivity = tp / (tp + fn)
    specificity = tn / (tn + fp)
    auroc = roc_auc_score(y_true, y_pred_prob)
    ap = average_precision_score(y_true, y_pred_prob)
    return {'AUROC': auroc, 'Average Precision': ap, 'F1-Score': f1, 'Sensitivity': sensitivity, 'Specificity': specificity}
```

## Common pitfalls

- High class imbalance leads to low Average Precision (~0.33) despite decent AUROC (~0.74), making AP a more informative metric than accuracy or AUROC alone for this clinical task.
- F1, Sensitivity, and Specificity are threshold-dependent; the paper reports point estimates but does not explicitly state the operating threshold used to derive them, which can vary across implementations.
- Time-varying codes are capped per stay (max 552 diagnosis/procedure codes, 392 meds/vitals), potentially truncating long ICU stays or losing fine-grained temporal ordering if not modeled with appropriate sequence architectures.

## Evidence (verbatim from paper)

> Average precision, AUROC, $F_{1}$ -score, sensitivity, and specificity for the considered deep learning architectures and the logistic regression model are reported in Table 1.

## Citation

```bibtex
@misc{barbieri2019icubenchmark,
  title={Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Risk},
  author={Barbieri et al. (2019)},
  year={2019},
  note={arXiv:1905.08547}
}
```

- arXiv: 1905.08547