Chemical Annotation Matrix Construction

---
name: chemical-annotation-matrix-construction
description: Use when you have selected a subset of public tandem MS files from ReDU/MassIVE that have been processed through GNPS spectral library matching, and you need to organize their chemical annotations into a matrix (rows = files, columns = annotated features) before performing PCA or other multivariate.
license: CC-BY-4.0
metadata:
  edam_operation: http://edamontology.org/operation_3443
  edam_topics:
  - http://edamontology.org/topic_3520
  - http://edamontology.org/topic_0121
  tools:
  - ReDU
  - MassIVE
  - GNPS
  - scikit-learn
  techniques:
  - CE-MS
derived_from:
- doi: 10.1038/s41592-020-0916-7
  title: ReDU
- doi: 10.1186/2047-217x-2-16
  title: ''
evidence_spans:
- Validation of the ReDU sample information template using the drag-and-drop validator
- ReDU only interacts with MassIVE
- data uploaded to MassIVE as a public dataset
claims: []
provenance:
  collection: https://w3id.org/holobiomicslab/asb-skill/collection/metabolomics/v2
  assembled_by: scripts/collect_metabolomics_collection.py
  sources:
  - build: coll_redu_cq
    doi: 10.1038/s41592-020-0916-7
    title: ReDU
  dedup_kept_from: coll_redu_cq
schema_version: 0.2.0
attribution:
  generator: AgenticScienceBuilder
  original_doi: 10.1038/s41592-020-0916-7
  all_source_dois:
  - 10.1038/s41592-020-0916-7
  - 10.1186/2047-217x-2-16
  zenodo_doi: 10.5281/zenodo.20794027
  curators: []
  promoter: Louis-Félix Nothias
  sponsor: CNRS & Université Côte d'Azur
---

# chemical-annotation-matrix-construction

## Summary

Construct a feature-by-sample matrix from GNPS chemical annotations (m/z, retention time, spectral library matches) retrieved from a subset of ReDU/MassIVE files, handling missing values by imputation or removal to prepare data for unsupervised dimensionality reduction and visualization.

## When to use

You have selected a subset of public tandem MS files from ReDU/MassIVE that have been processed through GNPS spectral library matching, and you need to organize their chemical annotations into a matrix (rows = files, columns = annotated features) before performing PCA or other multivariate analysis to explore sample relationships and chemical annotation patterns across the cohort.

## When NOT to use

- Input files have not been processed through GNPS spectral library matching (raw MS2 spectra only, no annotations).
- Only unannotated m/z features are available (use raw feature tables from mass spectrometry preprocessing instead).
- You need to include multiple levels of annotation confidence (e.g., level 2 and level 3 GNPS annotations separately); this skill merges all available annotations without stratifying by confidence.

## Inputs

- Selected MassIVE file identifiers (MSV IDs)
- GNPS spectral library search results with compound identifications
- ReDU chemical annotation database query (per-file feature lists with m/z, RT, annotation ID)

## Outputs

- Feature-by-sample matrix (tab-separated text file or CSV: rows = files, columns = annotated chemical features)
- Metadata file mapping file identifiers to MassIVE IDs and sample attributes
- Matrix statistics (feature count, file count, sparsity, missing value report)

## How to apply

Query the ReDU database to retrieve GNPS chemical annotation results (spectral library matches with m/z, retention time, and compound identities) for each selected MassIVE file. Organize the annotations into a matrix where rows represent individual files and columns represent unique annotated chemical features. Handle missing values—where a file lacks an annotation for a feature—by either removing features below a frequency threshold or imputing with zero abundance. Validate matrix dimensions and consistency (e.g., no negative values, correct data types). The resulting matrix is then compatible with scikit-learn PCA and Emperor visualization import, enabling unsupervised exploration of chemical annotation structure across samples.

## Related tools

- **ReDU** (Query and retrieve GNPS-annotated chemical features from public MassIVE files for selected cohort) — https://github.com/mwang87/ReDU-MS2-GNPS
- **GNPS** (Perform spectral library matching to generate chemical annotations (m/z, retention time, compound identities) for each file's MS2 spectra) — https://github.com/CCMS-UCSD/GNPSDocumentation
- **MassIVE** (Public repository storing raw and processed tandem MS data files that serve as input to ReDU and GNPS)
- **scikit-learn** (Apply PCA to the constructed feature-by-sample matrix to compute principal component loadings and scores)

## Evaluation signals

- Matrix dimensions (rows = number of selected files, columns = number of unique annotated features) are consistent with input file count and annotation inventory.
- No negative or invalid values in the matrix; all entries are non-negative abundances or presence/absence flags.
- Sparsity metric (percentage of zero/missing entries) is documented and reasonable for the annotation scope.
- Metadata file row count matches matrix row count; all file identifiers are traceable to MassIVE IDs.
- Matrix imports successfully into scikit-learn and Emperor without format errors; PCA computes and produces valid PC scores.

## Limitations

- GNPS annotations are level 2 or 3 by the 2007 metabolomics standard initiative (putative or putatively characterized, not confirmed identifications), limiting biological interpretability.
- The same chemical may have multiple GNPS annotations due to slight variation in MS2 spectra (m/z or abundance differences), leading to feature duplication; deduplication strategy must be specified.
- Missing annotations for a file–feature pair can introduce bias if imputation method is not documented or if removal threshold is too aggressive (loss of rare features).
- Matrix construction does not account for batch effects, instrumental drift, or sample heterogeneity across files from different laboratories or collection dates.

## Evidence

- [methods] Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features (m/z, retention time, GNPS annotations).: "Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features (m/z, retention time, GNPS annotations)."
- [methods] Construct a feature-by-sample matrix from the annotation data, handling missing values by imputation or removal.: "Construct a feature-by-sample matrix from the annotation data, handling missing values by imputation or removal."
- [abstract] The same chemical can have multiple GNPS annotations. Slight variation in the MS2 spectra (*m/z* or abundance) cause the pattern to match different reference MS2 spectra for the same chemical: "The same chemical can have multiple GNPS annotations. Slight variation in the MS2 spectra cause patterns to match different reference spectra"
- [abstract] GNPS annotations via spectral reference matching are considered level 2 (putative annotation based on spectral library similarity) or level 3 (putatively characterized compound class): "GNPS annotations via spectral reference matching are considered level 2 (putative annotation based on spectral library similarity) or level 3"
- [readme] Download batch template for GNPS at ```https://redu.ucsd.edu/metabatchdump``` Run Batch Workflow for Spectral Library Search: "Download batch template for GNPS at https://redu.ucsd.edu/metabatchdump Run Batch Workflow for Spectral Library Search"