Gnn Ak Eval

---
name: gnn-ak-eval
description: Evaluates the expressiveness and practical performance of GNN-AK, a framework that replaces star-shaped neighbor aggregation with subgraph-based encoding in Message Passing Neural Networks. It probes the model's ability to distinguish complex graph structures (e.g., strongly regular graphs, substructures) and predict graph-level properties on standard benchmarks. Use when the user wants to benchmark on ZINC-12K, CIFAR10, PATTERN, MolHIV, MolPCBA, EXP, SR25, or asks about evaluating this task. Reports accuracy (ACC), mean absolute error (MAE).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2110.03753
  bibtex_key: zhao2021gnnak
  confidence: high
---

# gnn-ak-eval

> From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness — Zhao et al. (2021) (arXiv:2110.03753, 2021)

## What this evaluates

Evaluates the expressiveness and practical performance of GNN-AK, a framework that replaces star-shaped neighbor aggregation with subgraph-based encoding in Message Passing Neural Networks. It probes the model's ability to distinguish complex graph structures (e.g., strongly regular graphs, substructures) and predict graph-level properties on standard benchmarks.

## Datasets

- **ZINC-12K** — total 12000; splits: train/val/test (-1)
- **CIFAR10** — total ?; splits: train/val/test (-1)
- **PATTERN** — total ?; splits: train/val/test (-1)
- **MolHIV** — total ?; splits: train/val/test (-1)
- **MolPCBA** — total ?; splits: train/val/test (-1)
- **EXP** — total 600; splits: train/val/test (-1)
- **SR25** — total 15; splits: train/val/test (-1)

## Metrics

- `accuracy (ACC)` **(primary)** — range: [0, 1]
  - Fraction of correctly classified graphs out of the total number of graphs in the dataset.
- `mean absolute error (MAE)` **(primary)** — range: [0, inf)
  - Average of absolute differences between predicted and true graph-level regression values. For graph property tasks, log10(MAE) is reported.
- `ROC-AUC` — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive and false positive rates for binary classification.
- `Average Precision (AP)` — range: [0, 1]
  - Area under the Precision-Recall curve, summarizing the precision-recall trade-off for multi-label classification.

## Input / output format

**Input**: Graph-structured data consisting of nodes, edges, and node/edge features. Tasks include graph classification and graph property regression.

**Output**: For classification: a discrete class label. For regression: a continuous scalar value. Metrics are computed over the entire dataset.

## Scoring recipe

```python
def compute_metrics(predictions, labels, task_type):
    if task_type == 'classification':
        return sum(1 for p, l in zip(predictions, labels) if p == l) / len(labels)
    elif task_type == 'regression':
        return sum(abs(p - l) for p, l in zip(predictions, labels)) / len(labels)
    elif task_type == 'binary':
        return roc_auc_score(labels, predictions)
    elif task_type == 'multilabel':
        return average_precision_score(labels, predictions)
```

## Common pitfalls

- Graph property regression metrics are reported as log10(MAE), not raw MAE.
- PPGN-AK+ frequently runs out of memory (OOM) on larger real-world datasets due to quadratic complexity.
- SubgraphDrop sampling (R parameter) trades off performance for runtime/memory, requiring careful selection of R for fair comparison.

## Evidence (verbatim from paper)

> Table 1: Simulation dataset performance:  $GNN-AK^{(+)}$  boosts base GNN across tasks, empirically verifying expressiveness lift. (ACC: accuracy, MAE: mean absolute error, OOM: out of memory)  

Table 3: Real-world dataset performance:  $GNN-AK^{+}$  achieves SOTA performance for ZINC-12K, CIFAR10, and PATTERN. (OOM: out of memory, -: missing values from literature)

## Citation

```bibtex
@misc{zhao2021gnnak,
  title={From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness},
  author={Zhao et al. (2021)},
  year={2021},
  note={arXiv:2110.03753}
}
```

- arXiv: 2110.03753