Cosmoflow Hpc Scaling

---
name: cosmoflow-hpc-scaling
description: Evaluates the compute efficiency and horizontal scalability of a 3D convolutional neural network framework on supercomputers, measuring sustained floating-point throughput and parallel scaling efficiency across thousands of nodes. Use when the user has predictions and gold and needs to compute Pflop/s.
metadata:
  skill_kind: metric
  source_arxiv: 1808.04728
  bibtex_key: [REDACTED]
  confidence: high
---

# cosmoflow-hpc-scaling

> CosmoFlow: Using Deep Learning to Learn the Universe at Scale — Mathuriya et al. (2018) (arXiv:1808.04728, 2018)

## What this evaluates

Evaluates the compute efficiency and horizontal scalability of a 3D convolutional neural network framework on supercomputers, measuring sustained floating-point throughput and parallel scaling efficiency across thousands of nodes.

## Datasets

- **Cosmological dark matter simulations** — total ?; splits: train (-1), validation (-1)

## Metrics

- `Pflop/s` **(primary)** — range: other
  - Total floating-point operations per epoch divided by the wall-clock time per epoch. The paper reports single-precision performance.
- `parallel efficiency` — range: percent
  - Speedup on N nodes divided by N, expressed as a percentage relative to single-node performance. Captures end-to-end system capability including I/O and communication.
- `throughput` — range: other
  - Wall-clock time required to complete one full training epoch, including training loop and loss averaging across MPI ranks.

## Input / output format

**Input**: 3D cosmological simulation data samples (dark matter distributions) processed in mini-batches of size one per node.

**Output**: Cosmological parameter predictions (Ωₘ, σ₈, nₛ) and scalar loss values for convergence tracking.

## Scoring recipe

```python
def compute_sustained_flops(total_flops_per_epoch, walltime_per_epoch):
    return total_flops_per_epoch / walltime_per_epoch

def compute_parallel_efficiency(single_node_time, n_node_time, n_nodes):
    speedup = single_node_time / n_node_time
    return (speedup / n_nodes) * 100

# Note: walltime includes validation and loop overheads.
# Mini-batch size is fixed at 1 per node; global batch scales with nodes.
```

## Common pitfalls

- I/O bottlenecks from shared filesystems (e.g., Lustre) can artificially deflate scaling efficiency if burst buffers are not utilized.
- Walltime measurements include validation and loop overheads, which may differ from pure compute step time and affect reported speedups.
- Fixed mini-batch size of 1 per node means global batch size scales linearly with node count, which can alter convergence dynamics compared to fixed global batch setups.

## Evidence (verbatim from paper)

> We achieve an average sustained performance of slightly over 3.5 Pflop/s single precision for 8192 nodes with a parallel efficiency of 77% relative to a single node (6324X speedup).

## Citation

```bibtex
@misc{mathuriya2018cosmoflow,
  title={CosmoFlow: Using Deep Learning to Learn the Universe at Scale},
  author={Mathuriya et al. (2018)},
  year={2018},
  note={arXiv:1808.04728}
}
```

- arXiv: 1808.04728