False Discovery Rate Interpretation

---
name: false-discovery-rate-interpretation
description: Use when when you have executed database search pipelines (Dereplicator,
  VarQuest, or Dereplicator+) on centroided LC-MS/MS spectra in MGF format and obtained
  match results with associated p-values and false discovery rates.
license: CC-BY-4.0
metadata:
  edam_operation: http://edamontology.org/operation_3927
  edam_topics:
  - http://edamontology.org/topic_3375
  - http://edamontology.org/topic_0121
  - http://edamontology.org/topic_3520
  tools:
  - NPDtools 2.5.0
  - MetaMiner
  - Dereplicator
  - ProteoWizard
  - Python
  - VarQuest
  - Dereplicator+
  techniques:
  - LC-MS
  license_tier: open
  provenance_tier: literature
derived_from:
- doi: 10.1038/s41467-018-06082-8
  title: dereplicator
evidence_spans:
- The latest version is available in the Natural Product Discovery toolkit (NPDtools)
  at https://github.com/ablab/npdtools
- MetaMiner is a metabologenomic pipeline which integrates metabolomic (tandem mass
  spectra) and genomic data to identify novel RiPPs
- MetaMiner is a metabologenomic pipeline which integrates metabolomic (tandem mass
  spectra) and genomic data to identify novel Ribosmally synthesized and Post-translationally
  modified Peptides (RiPPs)
- matches tandem mass spectra against the constructed post-translationally modified
  RiPPs structure database using Dereplicator
- MetaMiner natively supports MGF, mzXML, mzData and uses msconvert utility from the
  ProteoWizard package to convert spectra in other formats to MGF
- uses msconvert utility from the ProteoWizard package to convert spectra in other
  formats to MGF
claims: []
provenance:
  collection: https://w3id.org/holobiomicslab/asb-skill/collection/metabolomics/v2
  assembled_by: scripts/collect_metabolomics_collection.py
  sources:
  - build: coll_dereplicator
    doi: 10.1038/s41467-018-06082-8
    title: dereplicator
  dedup_kept_from: coll_dereplicator
schema_version: 0.2.0
attribution:
  generator: AgenticScienceBuilder
  original_doi: 10.1038/s41467-018-06082-8
  all_source_dois:
  - 10.1038/s41467-018-06082-8
  zenodo_doi: 10.5281/zenodo.20794027
  curators: []
  promoter: Louis-Félix Nothias
  sponsor: CNRS & Université Côte d'Azur
---

# false-discovery-rate-interpretation

## Summary

Interpretation of false discovery rates (FDR) from database search pipelines to distinguish genuine natural product identifications from spurious matches in mass spectrometry data. This skill involves extracting FDR values from pipeline outputs and using them as a quality filter to rank and validate spectral matches.

## When to use

When you have executed database search pipelines (Dereplicator, VarQuest, or Dereplicator+) on centroided LC-MS/MS spectra in MGF format and obtained match results with associated p-values and false discovery rates. Use this skill when you need to discriminate between high-confidence and low-confidence identifications across multiple pipelines or search modes (e.g., standard vs. blind/PTM-tolerant searches).

## When NOT to use

- Input spectra are not centroided or are in unsupported formats (non-MGF/mzML/mzXML/mzData).
- Database search has not yet been executed; raw spectra alone without match results cannot be analyzed for FDR.
- FDR values are not calculated by the pipeline or are missing from output files.

## Inputs

- significant_matches.tsv files from MetaMiner, Dereplicator, VarQuest, or Dereplicator+
- Scan identifiers from LC-MS/MS spectra (MGF or mzML format)
- Match scores from database search results
- P-values from spectral matching
- False discovery rate values per match

## Outputs

- Tabulated comparison of FDR values across pipelines
- Filtered match sets ranked by FDR threshold
- Cross-pipeline hit pattern comparison matrix
- Tool-specific and common identification sets
- FDR-stratified sensitivity analysis by search mode

## How to apply

Parse the significant_matches.tsv output files generated by each database search pipeline, extracting scan identifiers, match scores, p-values, and false discovery rates. Sort and filter matches by FDR threshold (typically stricter FDR values indicate higher-confidence identifications). Cross-compare FDR distributions and hit patterns across pipelines (e.g., Dereplicator vs. VarQuest vs. Dereplicator+) to identify tool-specific detections and common identifications. Record FDR alongside match scores to quantify sensitivity differences between search modes (standard parameters vs. --blind flag for arbitrary PTM detection). Use FDR as the primary ranking criterion when reporting identifications and when evaluating which pipeline or search mode is most effective for your spectral dataset.

## Related tools

- **Dereplicator** (Database search pipeline that matches tandem mass spectra against post-translationally modified RiPP structure databases and outputs FDR-qualified match results) — https://github.com/ablab/npdtools
- **VarQuest** (Modification-tolerant variant database search pipeline that identifies novel PNP variants with p-values and FDR scores) — https://github.com/ablab/npdtools
- **Dereplicator+** (Extended database search pipeline for both peptidic and non-peptidic metabolite identification with FDR-qualified results) — https://github.com/ablab/npdtools
- **MetaMiner** (Metabologenomic pipeline that integrates genomic and metabolomic data; uses Dereplicator for spectral matching and generates significant_matches.tsv with FDR values) — https://github.com/ablab/npdtools
- **NPDtools 2.5.0** (Toolkit containing Dereplicator, VarQuest, Dereplicator+, and MetaMiner; provides unified interface and output format for FDR interpretation across pipelines) — https://github.com/ablab/npdtools

## Examples

```
python parse_matches.py --dereplicator-tsv significant_matches_derep.tsv --varquest-tsv significant_matches_vq.tsv --dereplicator-plus-tsv significant_matches_derep_plus.tsv --fdr-threshold 0.05 --output comparison_table.csv
```

## Evaluation signals

- FDR values are present for all reported matches and are numeric (typically 0–1 or 0–100 scale depending on pipeline output format).
- FDR distributions differ meaningfully across search modes (standard vs. --blind) or across pipelines (Dereplicator vs. VarQuest vs. Dereplicator+), indicating mode-dependent sensitivity.
- Matches filtered by FDR threshold show expected trade-offs: stricter thresholds reduce false positives but may miss true identifications; permissive thresholds increase hit counts but include lower-confidence matches.
- Cross-pipeline comparison reveals both tool-specific detections (high FDR in one pipeline, absent in others) and common identifications (low FDR across multiple pipelines), validating consistency.
- Scan identifiers are correctly preserved and traceable from raw spectra through match results to final FDR-ranked tables.

## Limitations

- FDR computation assumes that the reference chemical structure database is comprehensive and representative; incomplete databases may systematically inflate FDRs for true identifications not represented in the database.
- Different pipelines (Dereplicator, VarQuest, Dereplicator+) may use different statistical models or thresholds to compute FDR, making cross-pipeline FDR comparison non-quantitative without normalization.
- The article does not provide explicit FDR cutoff thresholds (e.g., FDR < 0.05) for accepting identifications; interpretation depends on prior knowledge and downstream validation.
- FDR does not account for sample-specific factors (e.g., contamination, instrument drift) that may inflate or deflate confidence in a match within a particular run.

## Evidence

- [methods] Parse and tabulate significant_matches.tsv outputs from all three tools, recording scan identifiers, match scores, p-values, and false discovery rates.: "Parse and tabulate significant_matches.tsv outputs from all three tools, recording scan identifiers, match scores, p-values, and false discovery rates."
- [methods] Cross-compare hit sets to identify tool-specific detections, common identifications, and mode-dependent sensitivity differences.: "Cross-compare hit sets to identify tool-specific detections, common identifications, and mode-dependent sensitivity differences."
- [readme] matches tandem mass spectra against the constructed post-translationally modified RiPPs structure database using Dereplicator: "matches tandem mass spectra against the constructed post-translationally modified RiPPs structure database using Dereplicator"
- [methods] Execute MetaMiner again with --blind flag enabled to search for arbitrary post-translational modifications.: "Execute MetaMiner again with --blind flag enabled to search for arbitrary post-translational modifications."