Cnsight Eval

---
name: cnsight-eval
description: Evaluates the capability of models to detect section boundaries in clinical notes at the token level. It probes how well different architectures handle structured sentence-level segmentation versus unstructured freetext narrative variability in medical records. Use when the user wants to benchmark on MIMIC-IV Clinical Notes, or asks about evaluating this task. Reports Token-level F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.22795
  bibtex_key: surana2025cnsight
  confidence: high
---

# cnsight-eval

> CNSight: Evaluation of Clinical Note Segmentation Tools — Surana et al. (2025) (arXiv:2512.22795, 2025)

## What this evaluates

Evaluates the capability of models to detect section boundaries in clinical notes at the token level. It probes how well different architectures handle structured sentence-level segmentation versus unstructured freetext narrative variability in medical records.

## Datasets

- **MIMIC-IV Clinical Notes** — total ?; splits: MIMIC Hospital Sentences (-1), MIMIC Hospital Freetext (-1)

## Metrics

- `Token-level F1` **(primary)** — range: percent
  - F1 = 2 * (Precision * Recall) / (Precision + Recall). Precision = TP / (TP + FP), Recall = TP / (TP + FN). TP is a predicted boundary token exactly matching a gold boundary; FP is a spurious split; FN is a missed gold boundary. Weighted F1 averages per-class performance proportional to class frequency, while micro-averaged F1 aggregates decisions across all boundaries.

## Input / output format

**Input**: Tokenized clinical note text where each token is evaluated for boundary status.

**Output**: Binary token-level prediction sequence indicating whether each token is a section boundary (1) or not (0).

## Scoring recipe

```python
tp = sum(p == 1 and g == 1 for p, g in zip(predictions, gold))
fp = sum(p == 1 and g == 0 for p, g in zip(predictions, gold))
fn = sum(p == 0 and g == 1 for p, g in zip(predictions, gold))
precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
```

## Common pitfalls

- Applying macro-averaged F1 instead of the paper's specified weighted F1 for sentences or micro-averaged F1 for freetext, which misrepresents performance on imbalanced or highly variable section types.
- Evaluating at the span or sentence level rather than the token level, which changes the definition of true/false positives and invalidates the reported Precision/Recall/F1 scores.

## Evidence (verbatim from paper)

> Model performance is assessed using token-level Precision, Recall, and F1. For clinical note segmentation, we treat each predicted section boundary token as a classification decision. In this setup, a True Positive (TP) is a predicted boundary token that exactly matches a gold-standard boundary token, a False Positive (FP) is a predicted boundary token that does not correspond to any gold-standard boundary (i.e., a spurious split), and a False Negative (FN) is a gold-standard boundary token that the model fails to predict (i.e., a missed split).

## Citation

```bibtex
@misc{surana2025cnsight,
  title={CNSight: Evaluation of Clinical Note Segmentation Tools},
  author={Surana et al. (2025)},
  year={2025},
  note={arXiv:2512.22795}
}
```

- arXiv: 2512.22795