Ehr Clinical Outcome Prediction Eval

---
name: ehr-clinical-outcome-prediction-eval
description: This benchmark evaluates clinical outcome prediction models across three distinct EHR data representations (multivariate time-series, event streams, and textual event streams). It probes how well different architectures handle sparse, irregular longitudinal patient data and varying feature missingness rates in both acute ICU and long-term care settings. Use when the user wants to benchmark on MIMIC-IV, EHRSHOT, or asks about evaluating this task. Reports F1 score, AUROC, AUPRC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2510.09159
  bibtex_key: chen2025crossrepresentationbenchmarking
  confidence: high
---

# ehr-clinical-outcome-prediction-eval

> Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction — Chen et al. (2025) (arXiv:2510.09159, 2025)

## What this evaluates

This benchmark evaluates clinical outcome prediction models across three distinct EHR data representations (multivariate time-series, event streams, and textual event streams). It probes how well different architectures handle sparse, irregular longitudinal patient data and varying feature missingness rates in both acute ICU and long-term care settings.

## Datasets

- **MIMIC-IV** — total ?; splits: train (-1), val (-1), test (-1)
- **EHRSHOT** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `F1 score, AUROC, AUPRC` **(primary)** — range: [0, 1]
  - For binary tasks, F1, AUROC, and AUPRC are computed using a 0.5 probability threshold. For multi-label phenotyping, macro-averaged versions (macro-AUROC, macro-AUPRC, macro-F1) are reported.

## Input / output format

**Input**: Structured EHR data formatted as either a multivariate time-series matrix, an OMOP-coded event stream, or a textual event stream converted for LLM ingestion.

**Output**: Binary or multi-label clinical outcome predictions (e.g., ICU mortality, ICU phenotyping, 30-day readmission, 1-year pancreatic cancer risk).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred_proba, task_type='binary'):
    if task_type == 'binary':
        y_pred = (y_pred_proba >= 0.5).astype(int)
        f1 = f1_score(y_true, y_pred)
        auroc = roc_auc_score(y_true, y_pred_proba)
        auprc = average_precision_score(y_true, y_pred_proba)
        return {'F1': f1, 'AUROC': auroc, 'AUPRC': auprc}
    else:
        y_pred = (y_pred_proba >= 0.5).astype(int)
        return {
            'macro-AUROC': roc_auc_score(y_true, y_pred_proba, average='macro'),
            'macro-AUPRC': average_precision_score(y_true, y_pred_proba, average='macro'),
            'macro-F1': f1_score(y_true, y_pred, average='macro')
        }
```

## Common pitfalls

- Assuming feature pruning always improves performance; missingness-based pruning helps ICU models but degrades longitudinal care prediction where sparse features carry important long-term information.
- Assuming pretrained foundation models always outperform simple baselines; count-based models (XGBoost on event counts) match or exceed pretrained models when sufficient training data is available.
- Ignoring representation-specific feature selection; time-series models require task-specific feature selection to handle sparsity, whereas event-stream models leverage all available features without optimization.

## Evidence (verbatim from paper)

> For binary tasks, we report AUROC, AUPRC, and F1 using a 0.5 threshold. For multi-label phenotyping, we report macro-AUROC, macro-AUPRC, and macro-F1.

## Citation

```bibtex
@misc{chen2025crossrepresentationbenchmarking,
  title={Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction},
  author={Chen et al. (2025)},
  year={2025},
  note={arXiv:2510.09159}
}
```

- arXiv: 2510.09159