Mimic Sepsis Eval

---
name: mimic-sepsis-eval
description: Evaluates the ability of models to predict clinical outcomes (mortality, length of stay, shock onset) from time-aligned ICU patient trajectories and treatment dynamics. Use when the user wants to benchmark on MIMIC-Sepsis, or asks about evaluating this task. Reports performance.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2510.24500
  bibtex_key: huang2025mimicsepsis
  confidence: medium
---

# mimic-sepsis-eval

> MIMIC-Sepsis: A Curated Benchmark for Modeling and Learning from Sepsis Trajectories in the ICU — Huang et al. (2025) (arXiv:2510.24500, 2025)

## What this evaluates

Evaluates the ability of models to predict clinical outcomes (mortality, length of stay, shock onset) from time-aligned ICU patient trajectories and treatment dynamics.

## Datasets

- **MIMIC-Sepsis** — total 35239; splits: train (-1), val (-1), test (-1); repo https://github.com/yongh7/MIMIC-sepsis

## Metrics

- `performance` **(primary)** — range: [0, 1]
  - General predictive performance measured via accuracy for binary classification tasks (mortality, shock onset) and MAE for continuous regression (length of stay).

## Input / output format

**Input**: Time-aligned clinical trajectories resampled to 4-hour intervals from 24 hours before to 72 hours after suspected infection onset, including static demographics, longitudinal vitals/labs, and cumulative treatment interventions (vasopressors, fluids, antibiotics, ventilation).

**Output**: Predictions for three tasks: binary mortality status, continuous length of stay, and binary shock onset status.

## Scoring recipe

```python
def score(predictions, gold):
    if task in ['mortality', 'shock']:
        return sum(p == g for p, g in zip(predictions, gold)) / len(gold)
    elif task == 'los':
        return sum(abs(p - g) for p, g in zip(predictions, gold)) / len(gold)
    return None
```

## Common pitfalls

- Imputation strategy varies by missingness level (<5% linear, <80% KNN, >80% excluded), which can bias results if not replicated.
- Temporal alignment is strictly relative to suspected infection onset (Sepsis-3 criteria), not ICU admission.
- Treatment variables are cumulative per 4-hour interval, requiring careful handling of dosing conversions (e.g., norepinephrine-equivalent).

## Evidence (verbatim from paper)

> Empirical results show that integrating treatment dynamics significantly improves performance of Transformer-based models, especially in early mortality prediction, length-of-stay estimation, and shock onset classification—demonstrating the critical role of treatment-aware modeling in critical care prediction.

## Citation

```bibtex
@misc{huang2025mimicsepsis,
  title={MIMIC-Sepsis: A Curated Benchmark for Modeling and Learning from Sepsis Trajectories in the ICU},
  author={Huang et al. (2025)},
  year={2025},
  note={arXiv:2510.24500}
}
```

- arXiv: 2510.24500