Metadata Normalization And Reconciliation

---
name: metadata-normalization-and-reconciliation
description: Use when you have multiple CSV feature lists from different acquisition methods (e.g., LC-MS vs LC-IMS-MS) or processing software, each using different naming conventions, retention time scales, or m/z precision;
license: CC-BY-4.0
metadata:
  edam_operation: http://edamontology.org/operation_3281
  edam_topics:
  - http://edamontology.org/topic_3520
  - http://edamontology.org/topic_0121
  tools:
  - IonToolPack
  - Comparador
  techniques:
  - LC-MS
  - ion-mobility-MS
derived_from:
- doi: 10.1021/jasms.4c00146
  title: PeakQC
evidence_spans:
- IonToolPack is a software suite housing tools for mass spectrometry data
- IonToolPack is a software suite housing tools for mass spectrometry data.
- 'Comparador: Tool to compare lists of features (CSV files) from different acquisition methods or processing software'
claims: []
provenance:
  collection: https://w3id.org/holobiomicslab/asb-skill/collection/metabolomics/v2
  assembled_by: scripts/collect_metabolomics_collection.py
  sources:
  - build: coll_peakqc_cq
    doi: 10.1021/jasms.4c00146
    title: PeakQC
  dedup_kept_from: coll_peakqc_cq
schema_version: 0.2.0
attribution:
  generator: AgenticScienceBuilder
  original_doi: 10.1021/jasms.4c00146
  all_source_dois:
  - 10.1021/jasms.4c00146
  zenodo_doi: 10.5281/zenodo.20794027
  curators: []
  promoter: Louis-Félix Nothias
  sponsor: CNRS & Université Côte d'Azur
---

# metadata-normalization-and-reconciliation

## Summary

Harmonize and reconcile feature metadata (identifiers, retention time, m/z, and other fields) from multiple mass spectrometry feature lists generated by different acquisition methods or processing software, then perform cross-list comparison to identify overlapping and unique features. This skill is essential when integrating heterogeneous MS datasets that must be unified before comparative analysis.

## When to use

You have multiple CSV feature lists from different acquisition methods (e.g., LC-MS vs LC-IMS-MS) or processing software, each using different naming conventions, retention time scales, or m/z precision; you need to identify which features are the same across lists and which are unique to each method before performing downstream analysis or meta-analysis.

## When NOT to use

- Input data are already from a single acquisition method or processing pipeline (no cross-method reconciliation needed)
- Feature lists lack common measurable fields such as m/z or retention time (comparison cannot be performed reliably)
- The goal is quality control of a single feature list rather than integration of multiple lists

## Inputs

- CSV files containing feature lists from different acquisition methods or processing software
- Each CSV must include columns for feature identifiers, m/z values, retention time, and other metadata fields

## Outputs

- Harmonized feature table (CSV or tabular format) with unified feature IDs and normalized metadata
- Comparison report documenting feature overlap statistics, source method tags, and presence/absence across input lists

## How to apply

Load all CSV feature lists using pandas, then apply harmonization by matching on common fields (m/z, retention time, feature identifiers) and resolving naming conflicts through field mapping and tolerance-based matching (e.g., m/z and RT windows). Cross-list comparison logic identifies overlapping features across all input lists by finding matches within specified tolerances, while tagging each feature with its source method. Generate a structured comparison report documenting feature overlap statistics, harmonized feature IDs, source method annotations, and presence/absence patterns. The rationale is that direct string matching or ID comparison fails when software or instruments use different scales or nomenclature; tolerancing on continuous fields (m/z, RT) enables robust reconciliation even when nomenclature differs.

## Related tools

- **Comparador** (Primary tool for harmonizing and comparing feature lists from different acquisition methods or processing software; ingests CSV files and applies reconciliation logic) — https://github.com/pnnl/IonToolPack
- **IonToolPack** (Software suite housing Comparador; provides GUI interface for feature list comparison and supports multiple MS data formats and omics domains) — https://github.com/pnnl/IonToolPack

## Evaluation signals

- Harmonized feature IDs are consistent and resolvable across all input lists; no duplicate or conflicting IDs in the output
- Feature overlap statistics are symmetric or near-symmetric (if list A matches feature X from list B, then list B's feature X is tagged as matching list A)
- All input features are accounted for in the output; presence/absence patterns match the input data
- Retention time and m/z values in the harmonized output fall within the specified tolerance windows of their matched counterparts
- Source method tags correctly identify the origin of each feature and reflect input list boundaries

## Limitations

- Harmonization accuracy depends on the choice of tolerance windows for m/z and retention time matching; overly tight tolerances may miss true matches across methods; overly loose tolerances may create false matches
- Naming conflicts in feature identifiers may not be fully resolved if software systems use orthogonal naming schemes with no common measurable anchor
- The workflow assumes CSV format; other feature list formats (e.g., mzTab, proprietary binary formats) must be converted to CSV first
- Cross-method comparison is limited to common fields present in all input lists; method-specific metadata may be lost during harmonization

## Evidence

- [other] Comparador ingests feature lists in CSV format from different acquisition methods or processing software, applies harmonization procedures, and performs comparative analysis on the results.: "Comparador ingests feature lists in CSV format from different acquisition methods or processing software, applies harmonization procedures, and performs comparative analysis on the results."
- [other] Harmonize feature identifiers, retention time, m/z, and other metadata across lists by matching on common fields and resolving naming conflicts.: "Harmonize feature identifiers, retention time, m/z, and other metadata across lists by matching on common fields and resolving naming conflicts."
- [other] Identify overlapping and unique features across all input lists using cross-list comparison logic.: "Identify overlapping and unique features across all input lists using cross-list comparison logic."
- [other] Generate a structured comparison report (CSV or tabular format) documenting feature overlap statistics, harmonized feature IDs, source method tags, and analysis results.: "Generate a structured comparison report (CSV or tabular format) documenting feature overlap statistics, harmonized feature IDs, source method tags, and analysis results."
- [readme] Tool to compare lists of features (CSV files) from different acquisition methods or processing software, by harmonizing and analyzing results.: "Tool to compare lists of features (CSV files) from different acquisition methods or processing software, by harmonizing and analyzing results."