Molecular Structure Ground Truth Matching And Accuracy Evaluation

---
name: molecular-structure-ground-truth-matching-and-accuracy-evaluation
description: Use when after an end-to-end neural model (CNN + transformer) has generated
  predicted molecular structures (formula and connectivity) from 1D NMR spectra.
license: CC-BY-4.0
metadata:
  edam_operation: http://edamontology.org/operation_3802
  edam_topics:
  - http://edamontology.org/topic_0154
  - http://edamontology.org/topic_3172
  - http://edamontology.org/topic_3577
  tools:
  - convolutional neural network
  - transformer architecture
  techniques:
  - NMR
  license_tier: open
  provenance_tier: literature
derived_from:
- doi: 10.1021/acscentsci.4c01132
  title: NMR2Struct
evidence_spans: []
claims: []
provenance:
  collection: https://w3id.org/holobiomicslab/asb-skill/collection/metabolomics/v2
  assembled_by: scripts/collect_metabolomics_collection.py
  sources:
  - build: coll_nmr2struct
    doi: 10.1021/acscentsci.4c01132
    title: NMR2Struct
  dedup_kept_from: coll_nmr2struct
schema_version: 0.2.0
attribution:
  generator: AgenticScienceBuilder
  original_doi: 10.1021/acscentsci.4c01132
  all_source_dois:
  - 10.1021/acscentsci.4c01132
  zenodo_doi: 10.5281/zenodo.20794027
  curators: []
  promoter: Louis-Félix Nothias
  sponsor: CNRS & Université Côte d'Azur
---

# molecular-structure-ground-truth-matching-and-accuracy-evaluation

## Summary

Compares predicted molecular structures (formula and connectivity) against experimentally known ground-truth structures to compute exact recovery rates and other fidelity metrics. This skill is essential for validating end-to-end NMR-to-structure models on molecules up to 19 heavy atoms.

## When to use

After an end-to-end neural model (CNN + transformer) has generated predicted molecular structures (formula and connectivity) from 1D NMR spectra. Use this skill to quantify how many predictions exactly match the experimentally determined ground-truth structures in a held-out test set, yielding top-k exact structure recovery rates that reflect real-world accuracy.

## When NOT to use

- When predicted structures have not yet been generated by the model; this skill is a validation step, not a prediction step.
- When comparing spectra or spectral features directly without intermediate structure prediction—use spectral similarity metrics instead.
- When ground-truth structures are not available or are unreliable; accuracy evaluation requires experimentally confirmed reference structures.

## Inputs

- predicted molecular structures (SMILES strings or connectivity graphs) from the end-to-end model
- ground-truth molecular structures (canonical SMILES or adjacency matrices) for test molecules
- test set metadata (molecule identifiers, heavy atom counts ≤19)
- model confidence scores or ranking for top-k evaluation

## Outputs

- top-k exact structure recovery rates (e.g., top-1, top-5, top-10 accuracy percentages)
- count of exact matches and mismatches per test molecule
- accuracy metrics stratified by molecular size (heavy atom counts)
- detailed match/mismatch report for error analysis

## How to apply

Load the ground-truth molecular structures (canonical SMILES or connectivity graphs) for the test set molecules (≤19 heavy atoms). For each predicted structure output by the model, perform a canonical comparison (e.g., SMILES normalization or graph isomorphism) against its corresponding ground-truth structure. Count exact matches (both molecular formula and connectivity must match) and compute top-k metrics by ranking predictions by model confidence. Report the fraction of test molecules where the top-1, top-5, or top-10 prediction contains the correct structure. Cross-validate that the ground-truth set is disjoint from training and validation sets to avoid inflated metrics.

## Related tools

- **convolutional neural network** (encodes raw 1D NMR spectra into learned feature representations upstream of structure prediction)
- **transformer architecture** (assembles spectral features and molecular fragments into connectivity predictions that are then matched against ground truth)

## Evaluation signals

- Top-k accuracy metrics (top-1, top-5, top-10) are computed and match or exceed the paper's reported baseline performance on the same test set.
- Exact matches are verified by canonical SMILES comparison or graph isomorphism to exclude false positives from different but equivalent structural representations.
- Ground-truth test set is confirmed to be disjoint from training and validation splits (no data leakage).
- Accuracy is stratified by molecular size (heavy atom bins) and shows consistent recovery rates across the ≤19 heavy atom range reported in the paper.
- Mismatch cases are inspectable: predicted vs. ground-truth structures can be visually inspected or dumped to a report for error classification (e.g., wrong formula, correct formula but wrong connectivity).

## Limitations

- The skill is limited to test molecules with ≤19 heavy atoms, as the model was trained and validated only in this regime; extrapolation to larger molecules is not supported by the paper.
- Accuracy depends on the quality and completeness of the ground-truth reference structures; experimental errors or missing reference data will bias metrics.
- Top-k metrics assume the model outputs a ranked list of candidate structures; if only a single prediction is returned, only top-1 accuracy can be computed.
- The skill does not account for stereoisomerism or 3D conformation; it evaluates connectivity and formula only, so molecules differing only in stereochemistry will be treated as different structures.

## Evidence

- [intro] end-to-end structure recovery accuracy metrics: "What are the exact structure recovery accuracy metrics achieved by the end-to-end NMR2Struct model on test molecules containing up to 19 heavy atoms?"
- [intro] multitask prediction generates formula and connectivity: "Perform multitask prediction to generate both molecular formula and connectivity output."
- [intro] top-k comparison against ground truth: "Compute top-k exact structure recovery rates by comparing predicted structures against ground-truth molecular structures."
- [intro] model outputs structure from spectra: "predicts the molecular structure (formula and connectivity) of an unknown compound solely based on its 1D 1H and/or 13C NMR spectra"
- [intro] scope: molecules up to 19 heavy atoms: "We demonstrate the effectiveness of this framework on molecules with up to 19 heavy (non-hydrogen) atoms"