Graphood Drugood Eval

---
name: graphood-drugood-eval
description: Evaluates graph neural networks' out-of-distribution (OOD) generalization across synthetic, image, molecular, and text graph datasets. It measures how well models maintain performance when tested on domain-shifted splits (e.g., different graph sizes or molecular scaffolds) compared to in-distribution data. Use when the user wants to benchmark on GraphOOD & DrugOOD, or asks about evaluating this task. Reports ROC-AUC, Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.08855
  bibtex_key: qiu2026rethinking
  confidence: high
---

# graphood-drugood-eval

> Rethinking Graph Generalization through the Lens of Sharpness-Aware Minimization — Qiu et al. (2026) (arXiv:2602.08855, 2026)

## What this evaluates

Evaluates graph neural networks' out-of-distribution (OOD) generalization across synthetic, image, molecular, and text graph datasets. It measures how well models maintain performance when tested on domain-shifted splits (e.g., different graph sizes or molecular scaffolds) compared to in-distribution data.

## Datasets

- **GraphOOD & DrugOOD** — total ?; splits: domain-based (ID/OOD) (-1)

## Metrics

- `ROC-AUC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, used for binary classification tasks.
- `Accuracy` **(primary)** — range: [0, 1]
  - Standard classification accuracy, calculated as the proportion of correctly predicted labels out of total samples.

## Input / output format

**Input**: Graph data consisting of node features, adjacency matrices, and class labels, partitioned into in-distribution (ID) and out-of-distribution (OOD) domains based on attributes like graph size, scaffold, or color.

**Output**: Predicted class labels for each graph in the test set.

## Scoring recipe

```python
def evaluate(predictions, labels, is_binary):
    if is_binary:
        return roc_auc_score(labels, predictions)
    else:
        return accuracy_score(labels, predictions)
```

## Common pitfalls

- Evaluating ID and OOD splits separately rather than reporting domain-specific results, as performance often drops significantly on OOD.
- Using Accuracy for binary classification datasets instead of ROC-AUC, which the paper explicitly mandates.
- Assuming environment labels are available for all datasets; EC50 and IC50 explicitly lack them.

## Evidence (verbatim from paper)

> ROC-AUC metric is used for the binary classification dataset and Accuracy for the others. ... Each dataset contains one or more domains such as graph size and scaffolds and is divided into domain-based splits, thereby introducing distribution shifts.

## Citation

```bibtex
@misc{qiu2026rethinking,
  title={Rethinking Graph Generalization through the Lens of Sharpness-Aware Minimization},
  author={Qiu et al. (2026)},
  year={2026},
  note={arXiv:2602.08855}
}
```

- arXiv: 2602.08855