P50 Latency

---
name: p50-latency
description: Evaluates the end-to-end latency and energy efficiency of CPU/GPU scheduling strategies for agentic AI workloads under batched request arrivals. Use when the user has predictions and gold and needs to compute P50 latency.
metadata:
  skill_kind: metric
  source_arxiv: 2511.00739
  bibtex_key: raj2025cpucentric
  confidence: high
---

# p50-latency

> A CPU-Centric Perspective on Agentic AI — Ritik Raj et al. (2025) (arXiv:2511.00739, 2025)

## What this evaluates

Evaluates the end-to-end latency and energy efficiency of CPU/GPU scheduling strategies for agentic AI workloads under batched request arrivals.

## Datasets

- **LangChain** — total ?; splits: test (-1)
- **Haystack RAG** — total ?; splits: test (-1)
- **SWE-Agent** — total ?; splits: test (-1)

## Metrics

- `P50 latency` **(primary)** — range: seconds
  - 50th percentile of request completion times across a batch.
- `P90 latency` — range: seconds
  - 90th percentile of request completion times.
- `P99 latency` — range: seconds
  - 99th percentile of request completion times.
- `CPU dynamic energy` — range: joules
  - Total energy consumed by CPU during execution, calculated as power multiplied by execution time.

## Input / output format

**Input**: Agentic AI workloads (RAG queries, code execution, web search) submitted as closed-loop batches (B=128 or B=256) arriving simultaneously at t=0.

**Output**: Per-request completion timestamps aggregated into P50/P90/P99 latency values and total CPU dynamic energy.

## Scoring recipe

```python
def compute_metrics(completion_times, cpu_power_profile):
    sorted_times = sorted(completion_times)
    p50 = sorted_times[len(sorted_times) // 2]
    p90 = sorted_times[int(len(sorted_times) * 0.9)]
    p99 = sorted_times[int(len(sorted_times) * 0.99)]
    energy = sum(cpu_power_profile) * max(completion_times)
    return {'p50': p50, 'p90': p90, 'p99': p99, 'energy': energy}
```

## Common pitfalls

- Evaluates only single-run times despite noting ~5% statistical variance across multiple runs.
- Assumes a closed-loop arrival model where all B requests arrive simultaneously at t=0, unlike real-world streaming traffic.
- Results are hardware-specific to Intel Emerald Rapids CPU and B200 GPU; scaling to other architectures requires re-evaluation.

## Evidence (verbatim from paper)

> We assume a closed-loop arrival system (all $B$ requests arrive simultaneously at t\=0) to evaluate CGAM ([Section 6.2]) and MAWS ([Section 6.3]) for $B\=128$ and combined MAWS+CGAM ([Section 6.4]) for $B\=256. ... CGAM results in 2.11$\times$ (11.21 s to 5.32 s), 1.94$\times$ (42.87 s to 22.12 s) and 1.72$\times$ (65.08 s to 37.82 s) respective reduction in P50 latency ... MAWS performs 1.17$\times$ better in terms of P99 latency while maintaining similar P50 latency.

## Citation

```bibtex
@misc{raj2025cpucentric,
  title={A CPU-Centric Perspective on Agentic AI},
  author={Ritik Raj et al. (2025)},
  year={2025},
  note={arXiv:2511.00739}
}
```

- arXiv: 2511.00739