Rna 3d Scoring Eval

---
name: rna-3d-scoring-eval
description: Evaluates a model's ability to score and rank candidate 3D RNA structural models by predicting their deviation from the true native structure. It probes the model's capacity to distinguish accurate conformations from decoys using only atomic coordinates and types. Use when the user wants to benchmark on RNA-Puzzles & FARFAR2 Decoys, or asks about evaluating this task. Reports RMSD.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2210.16392
  bibtex_key: zhang2022physics
  confidence: high
---

# rna-3d-scoring-eval

> Physics-aware Graph Neural Network for Accurate RNA 3D Structure Prediction — Zhang et al. (2022) (arXiv:2210.16392, 2022)

## What this evaluates

Evaluates a model's ability to score and rank candidate 3D RNA structural models by predicting their deviation from the true native structure. It probes the model's capacity to distinguish accurate conformations from decoys using only atomic coordinates and types.

## Datasets

- **RNA-Puzzles & FARFAR2 Decoys** — total ?; splits: train (-1), val (-1), benchmark1 (-1), benchmark2 (-1)

## Metrics

- `RMSD` **(primary)** — range: other
  - Root Mean Square Deviation between predicted and native atomic coordinates. Lower values indicate higher structural accuracy and are used to rank candidate models.

## Input / output format

**Input**: Atomic coordinates and atom types for a set of candidate 3D RNA structural models generated for a given RNA sequence.

**Output**: A scalar predicted RMSD value for each candidate structural model.

## Scoring recipe

```python
pred_rmsds = model.predict(candidates)
ranked = argsort(pred_rmsds, ascending=True)
best_rank = find_rank(ranked, threshold=2.0)
top10_acc = count_within_threshold(ranked[:10], threshold=2.0)
```

## Common pitfalls

- The benchmark evaluates a scoring/ranking function, not direct structure generation. Models must output RMSD scores to rank decoys.
- Benchmark 2 explicitly contains no near-native models (<2Å RMSD), making it a distinct and harder evaluation scenario than Benchmark 1.
- Researchers must use the updated Benchmark 2 version from the ARES paper, as the original version contained errors that skewed baseline comparisons.

## Evidence (verbatim from paper)

> In practice, PaxNet predicts the root mean square deviation (RMSD) from the unknown true structure for each structural model. A lower RMSD would suggest a more accurate structural model predicted by PaxNet. ... benchmark 1 consists of the first 21 RNAs in the RNA-Puzzles structure prediction challenge... benchmark 2 contains no near-native models, and includes 16 RNAs that are all substantially different from any of those used in the aforementioned datasets

## Citation

```bibtex
@misc{zhang2022physics,
  title={Physics-aware Graph Neural Network for Accurate RNA 3D Structure Prediction},
  author={Zhang et al. (2022)},
  year={2022},
  note={arXiv:2210.16392}
}
```

- arXiv: 2210.16392