Fanstore Io Eval

---
name: fanstore-io-eval
description: Evaluates the I/O throughput and bandwidth of a distributed runtime file system (FanStore) across varying node counts and file sizes, comparing it against local SSDs, FUSE, and shared file systems like Lustre. Use when the user wants to benchmark on ImageNet-1k, SRGAN, FRNN, Custom Synthetic Benchmark, or asks about evaluating this task. Reports bandwidth (MB/s), throughput (files/s).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1809.10799
  bibtex_key: zhang2018fanstore
  confidence: high
---

# fanstore-io-eval

> FanStore: Enabling Efficient and Scalable I/O for Distributed Deep Learning — Zhang et al. (2018) (arXiv:1809.10799, 2018)

## What this evaluates

Evaluates the I/O throughput and bandwidth of a distributed runtime file system (FanStore) across varying node counts and file sizes, comparing it against local SSDs, FUSE, and shared file systems like Lustre.

## Datasets

- **ImageNet-1k** — total ?; splits: full (-1)
- **SRGAN** — total 455000000000; splits: full (-1)
- **FRNN** — total 54000000000; splits: full (-1)
- **Custom Synthetic Benchmark** — total ?; splits: full (-1)

## Metrics

- `bandwidth (MB/s)` **(primary)** — range: other
  - Total bytes read divided by the total time taken to read all files across all nodes.
- `throughput (files/s)` **(primary)** — range: other
  - Total number of files read divided by the total time taken to read all files across all nodes.
- `scaling efficiency (%)` — range: percent
  - Ratio of throughput at N nodes to baseline throughput, normalized by the node count increase.

## Input / output format

**Input**: Distributed directory of files accessed concurrently by multiple compute nodes via POSIX system calls intercepted by FanStore.

**Output**: Aggregated bandwidth (MB/s) and throughput (files/s) per node scale and file size configuration.

## Scoring recipe

```python
def compute_metrics(total_files, total_bytes, elapsed_seconds):
    throughput = total_files / elapsed_seconds
    bandwidth = total_bytes / elapsed_seconds
    return throughput, bandwidth

def compute_scaling_efficiency(throughput_n, throughput_baseline, nodes_n, nodes_baseline):
    return (throughput_n / throughput_baseline) / (nodes_n / nodes_baseline) * 100
```

## Common pitfalls

- Assuming shared file systems (e.g., Lustre) scale linearly; performance actually fluctuates and degrades under high concurrency.
- Ignoring that compression benefits are highly dependent on file size and whether the bottleneck is CPU, network latency, or network bandwidth.
- Treating local SSD performance as the achievable baseline for networked storage, rather than an upper bound.

## Evidence (verbatim from paper)

> This benchmark has four file sizes: 128 KB, 512 KB, 2 MB, and 8 MB. Each file size has {128K, 32K, 8K, 2K} file count, respectively. At each scale, each node reads all files in the directory, and reports time-to-solution and bandwidth. The benchmark reports the aggregated bandwidth and throughput as results.

## Citation

```bibtex
@misc{zhang2018fanstore,
  title={FanStore: Enabling Efficient and Scalable I/O for Distributed Deep Learning},
  author={Zhang et al. (2018)},
  year={2018},
  note={arXiv:1809.10799}
}
```

- arXiv: 1809.10799