Self Contained Pipeline Construction

---
name: self-contained-pipeline-construction
description: Use when when you have metabolomics datasets (loaded as SummarizedExperiment
  objects) and need to apply a sequence of data-processing and analytical steps (e.
license: CC-BY-4.0
metadata:
  edam_operation: http://edamontology.org/operation_3937
  edam_topics:
  - http://edamontology.org/topic_0121
  - http://edamontology.org/topic_3172
  tools:
  - R
  - SummarizedExperiment
  - magrittr
  - maplet
  license_tier: open
  provenance_tier: literature
derived_from:
- doi: 10.1093/bioinformatics/btab741
  title: maplet
evidence_spans:
- maplet is an R package
- The toolbox builds upon the bioconductor package SummarizedExperiment (SE)
- maplet is designed to work with a pipe operator - either the popular %>% operator
  from the magrittr package
claims: []
provenance:
  collection: https://w3id.org/holobiomicslab/asb-skill/collection/metabolomics/v2
  assembled_by: scripts/collect_metabolomics_collection.py
  sources:
  - build: coll_maplet_cq
    doi: 10.1093/bioinformatics/btab741
    title: maplet
  dedup_kept_from: coll_maplet_cq
schema_version: 0.2.0
attribution:
  generator: AgenticScienceBuilder
  original_doi: 10.1093/bioinformatics/btab741
  all_source_dois:
  - 10.1093/bioinformatics/btab741
  zenodo_doi: 10.5281/zenodo.20794027
  curators: []
  promoter: Louis-Félix Nothias
  sponsor: CNRS & Université Côte d'Azur
---

# self-contained-pipeline-construction

## Summary

Construct modular, reproducible metabolomics analysis pipelines by chaining maplet functions through pipe operators (magrittr's %>% or base R's |>) that operate on a shared SummarizedExperiment container. This approach eliminates temporary variables and improves readability while maintaining full traceability of data, analysis steps, and results.

## When to use

When you have metabolomics datasets (loaded as SummarizedExperiment objects) and need to apply a sequence of data-processing and analytical steps (e.g., annotation, statistical analysis, visualization, reporting) in a linear, reproducible workflow without intermediate object assignments or manual state management.

## When NOT to use

- Input data is already a feature table with no need for annotation or statistical analysis.
- Analysis requires non-linear or conditional branching logic (e.g., different downstream steps based on intermediate QC results); maplet pipelines are designed for linear chaining.
- Intermediate steps require manual inspection or interactive decision-making between pipeline stages; self-contained pipelines are best suited for fully scripted, deterministic workflows.

## Inputs

- Raw metabolomics data (e.g., CSV, mzML, or other format supported by maplet loaders)
- SummarizedExperiment container (initialized or from prior pipeline step)
- Annotation reference data (optional, for annotation step)
- Statistical parameters or configuration (e.g., for analysis functions)

## Outputs

- SummarizedExperiment object containing input data, analysis results, and pipeline metadata
- RDS file (serialized SummarizedExperiment)
- Visualizations (plots, reports) generated by maplet visualization and reporting functions
- Statistical test results and summaries stored in assays or colData/rowData of SummarizedExperiment

## How to apply

Initialize a SummarizedExperiment container using maplet's data-loading functions, then chain one or more maplet functions (for annotation, statistical analysis, visualization, or reporting) using the %>% pipe operator from magrittr or the |> base R operator. Each pipe passes the entire SummarizedExperiment object forward, accumulating analysis steps and results within the same container. Execute the complete chained pipeline as a single script block, which preserves modularity and readability. Save the final SummarizedExperiment object using saveRDS() to archive the complete analysis history and outputs.

## Related tools

- **maplet** (Provides data loading, annotation, statistical analysis, visualization, and reporting functions that operate on SummarizedExperiment containers and are designed to be chained via pipe operators.) — https://github.com/krumsieklab/maplet
- **SummarizedExperiment** (Bioconductor package that serves as the central container for storing metabolomics data, analysis steps, and results; all maplet functions read from and write to this object.) — https://bioconductor.org/packages/release/bioc/vignettes/SummarizedExperiment/inst/doc/SummarizedExperiment.html
- **magrittr** (Provides the %>% pipe operator used to chain maplet functions without temporary variable assignments; enables smooth connections between pipeline steps.) — https://magrittr.tidyverse.org/

## Examples

```
library(maplet)
se <- mt_load_metabolomics_data('data.csv') %>%
  mt_anno_lipidmaps() %>%
  mt_statistics_univariate() %>%
  mt_reporting_html()
saveRDS(se, 'pipeline_output.rds')
```

## Evaluation signals

- The final RDS file contains a valid SummarizedExperiment object with populated assays, colData, and rowData reflecting all pipeline steps.
- All intermediate analysis results (e.g., statistical p-values, annotations, feature metadata) are present and consistent within the SummarizedExperiment container.
- The pipeline script runs without errors from start to finish, demonstrating that all chained functions are compatible and properly format their inputs/outputs.
- Re-running the same pipeline script on the same input data produces identical outputs (deterministic reproducibility).
- The maplet pipeline is shorter and more readable than an equivalent script using temporary variable assignments, confirming the modularity benefit.

## Limitations

- maplet is in active development; commits without release tags are not guaranteed to be stable.
- Pipelines are inherently linear and do not support conditional branching or dynamic routing based on intermediate results.
- The skill requires all analysis steps to be compatible with the SummarizedExperiment structure; external analysis functions must be wrapped or adapted to this container model.
- No built-in changelog or version tracking is available to document changes between pipeline runs.

## Evidence

- [full_text] maplet pipelines operate by chaining functions via pipe operators: "maplet pipelines operate by chaining functions via pipe operators (either %>% from magrittr or |> from base R) that connect pipeline steps without requiring temporary variables or multiple"
- [readme] SummarizedExperiment as central repository: "SummarizedExperiment (SE), which serves as a central repository for each pipeline's data, analysis steps, and results."
- [readme] maplet function categories: "maplet provides a suite of functions for interacting with this container including but not limited to data loading, annotation, statistical analysis, visualization, and reporting."
- [readme] Pipe operator benefit: "This operator allows for smooth connections between pipeline steps, without the need for temporary variables or multiple assignments."
- [readme] Pipeline properties: "The combination of these elements allows for the creation of pipelines which are simple to follow, highly modular, and easily reproducible."
- [readme] Active development caveat: "maplet is in active development. Any commit without a release tag is not guaranteed to be stable."