Evaluates the throughput, tail latency, and power efficiency of a dynamic scheduling system for at-scale neural recommendation inference across various industry models, hardware platforms, and tail-latency constraints. Use when the user wants to benchmark on Industry-representative recommendation models (DLRM-RMC1/2/3, WND, MT-WND, NCF, DIN, DIEN), or asks about evaluating this task. Reports QPS.
Installs into .claude/skills of the current project.
Are you the author of Deeprecsys Eval?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/qhjqhj00-deeprecsys-eval)
---
name: deeprecsys-eval
description: Evaluates the throughput, tail latency, and power efficiency of a dynamic scheduling system for at-scale neural recommendation inference across various industry models, hardware platforms, and tail-latency constraints. Use when the user wants to benchmark on Industry-representative recommendation models (DLRM-RMC1/2/3, WND, MT-WND, NCF, DIN, DIEN), or asks about evaluating this task. Reports QPS.
metadata:
skill_kind: dataset_eval
source_arxiv: 2001.02772
bibtex_key: [REDACTED]
confidence: high
---
# deeprecsys-eval
> DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference — Gupta et al. (2020) (arXiv:2001.02772, 2020)
## What this evaluates
Evaluates the throughput, tail latency, and power efficiency of a dynamic scheduling system for at-scale neural recommendation inference across various industry models, hardware platforms, and tail-latency constraints.
## Datasets
- **Industry-representative recommendation models (DLRM-RMC1/2/3, WND, MT-WND, NCF, DIN, DIEN)** — total ?; splits: test (-1)
## Metrics
- `QPS` **(primary)** — range: other
- Queries Per Second, normalized to the baseline static scheduler's QPS at the low tail latency case.
- `Tail Latency (p95, p99)` — range: other
- 95th and 99th percentile of inference latency, measured in milliseconds.
- `QPS/Watt` — range: other
- Throughput normalized by power consumption, measuring queries processed per watt of energy.
## Input / output format
**Input**: Recommendation inference queries with varying sizes and arrival rates, processed by neural recommendation models.
**Output**: System throughput (QPS), tail latency percentiles (p95, p99), and power efficiency (QPS/Watt) under different tail latency targets and hardware configurations.
## Scoring recipe
```python
def compute_metrics(predictions, gold):
# predictions: list of per-query inference latencies
# gold: dict with 'time_seconds' and 'power_watts'
qps = len(predictions) / gold['time_seconds']
p95 = np.percentile(predictions, 95)
p99 = np.percentile(predictions, 99)
qps_per_watt = qps / gold['power_watts']
return {'QPS': qps, 'p95': p95, 'p99': p99, 'QPS/Watt': qps_per_watt}
```
## Common pitfalls
- Tail latency targets (low, medium, high) significantly impact optimal batch size and hardware offloading strategy.
- Power efficiency (QPS/Watt) does not always improve with GPU offloading; memory-intensive models may see degradation due to GPU power overhead.
- Production traffic distributions (heavier tail) differ from synthetic lognormal distributions, requiring careful batch-size tuning.
## Evidence (verbatim from paper)
> Overall, DeepRecSched-CPU achieves 1.7×, 2.1×, and 2.7× higher QPS across all models for the low, medium, and high tail latency targets, respectively. DeepRecSched-CPU is able to increase the overall system throughput by operating at the optimal batch size configuration. Furthermore, DeepRecSched-GPU increases performance improvement to 4.0×, 5.1×, and 5.8× at the low, medium, and high tail latency targets, respectively.
## Citation
```bibtex
@misc{gupta2020deeprecsys,
title={DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference},
author={Gupta et al. (2020)},
year={2020},
note={arXiv:2001.02772}
}
```
- arXiv: 2001.02772
