Latency Budget Architect

---
name: latency-budget-architect
description: 'Decompose an end-to-end latency target into per-hop engineering budgets — map the path (edge → gateway → services → stores → third parties), allocate milliseconds per hop including overhead (serialization, queue wait, connection setup, retries), and do tail math with explicit dependence assumptions. Under independent branch tails, fan-out exposure is 1-(0.99)^N; correlated tails require measured or bounded joint behavior. Derive timeouts and retries from budgets, keep explicit headroom, and define violation attribution plus the rule that a new dependency must claim budget. CONSUMES the journey target from slo-reliability-architect (or flags its assumption): SLOs say what users must experience; budgets say which component may spend which milliseconds. Use when allocating a latency target across services or deriving timeouts/retries. Do NOT use to set SLOs/error budgets or alerting (slo-reliability-architect), or to measure (performance-test-harness).'
---

# Latency Budget Architect

## Purpose

Every service on the path believes it is fast enough, and the user
experiences the sum: 40ms of gateway, 60ms of auth, 120ms of service
time, 80ms of database, 90ms of serialization nobody counted, and a
retry — 700ms against a 300ms promise, with no component at fault
because no component ever owned a number. This skill turns an
end-to-end latency target into per-hop budgets: an explicit allocation
of milliseconds per component, tail-composition math done honestly,
timeouts and retry policies derived from the budgets they must respect,
headroom held in reserve, and an attribution method so a violated
budget names its hop. The target itself comes from the user-journey
service level objective (SLO)
(`slo-reliability-architect`) — this skill engineers the spend plan
that makes the target achievable by design rather than by hope.

## Use When

- Use when: an end-to-end latency target (from an SLO, a contract, or
  a product requirement) needs decomposing into component-level
  budgets that teams can own and be reviewed against.
- Use when: timeouts and retry policies are folklore (30-second
  defaults, retry-everything) and need deriving from an actual budget.
- Use when: a new dependency is entering a latency-sensitive path and
  the question is what it may cost — the budget-claim review.
- Use when: fan-out amplification (parallel calls to N shards or
  services) needs its tail math checked against the end-to-end
  promise.
- Use when: end-to-end latency degraded and no hop admits ownership —
  the attribution design (which hop blew its budget) is missing.
- Do NOT use when: the question is what the TARGET should be, which
  journeys matter, what pages an operator, or how error budgets govern
  release velocity — that is `slo-reliability-architect`; this skill
  starts from its output (or states an assumed target loudly and
  flags it for ratification).
- Do NOT use when: building the measurement instrument — pass/fail
  latency measurement, baselines, and regression gates are
  `performance-test-harness`; budgets become its thresholds.
- Do NOT use when: the work is implementing trace instrumentation or
  dashboards — `observability-operator` *(manual-only)* wires what this design
  requires.
- Do NOT use when: a specific hop needs its time reduced — that is
  the owning skill's job (`query-plan-reader` *(manual-only)* for a query,
  `caching-strategy-designer` for a cache, `frontend-perf-engineer`
  for the browser share); this skill decides how much time the hop
  MAY spend, not how it gets there.

## Inputs to Inspect

1. The end-to-end target and its provenance: the SLO catalog entry
   (`slo-reliability-architect` output), contract clause, or product
   requirement — with its percentile (pXX is the latency that XX% of
   requests stay within, so p99 covers all but the slowest 1%; a p50 budget and a p99 budget
   are different documents; know which is being allocated).
2. The request path, as it actually is: every hop a request traverses
   — edge/content delivery network (CDN), gateway, auth, services, databases/caches, queues,
   third parties — from trace data where it exists, architecture docs
   where it doesn't (and mark the doc-only hops as unverified).
3. Current per-hop reality: measured per-hop latency distributions
   (p50/p95/p99) from traces or logs — the budget negotiates with
   reality, not with wishes.
4. Fan-out and serialization structure: which calls are parallel,
   which serial, where responses assemble — the composition math
   depends on the shape.
5. Existing timeout/retry configuration per hop: what is set today,
   where the defaults came from, and where retries can multiply
   (retry storms at two layers compound).
6. The frontend share, if the target is user-perceived: how much of
   the budget the browser consumes (render, hydration) — allocated
   with `frontend-perf-engineer`'s metrics vocabulary.

## Workflow

For any user-facing build choice, define terms and each viable path's purpose;
compare case-specific pros/cons and money, setup, and maintenance costs (mark
unknowns). Recommend one and why, then ask one atomic decision question. If a
deciding fact is missing, ask only for that fact this turn. A choice does not
authorize execution.

1. **Fix the target and its percentile.** One budget document per
   percentile that matters (typically p95 or p99 user-perceived).
   If no ratified target exists, state the assumption in the header
   ("ASSUMED 300ms p95 — requires slo-reliability-architect
   ratification") — an assumed target produces a real budget with a
   flagged foundation, never a silent one.
2. **Map the path and its composition shape.** Serial chains ADD.
   Parallel fan-out composes at the SLOWEST branch — and the tail
   math is where designs lie to themselves: hitting N parallel
   dependencies, the request experiences a dependency's p99 with
   probability 1-(0.99)^N only when branch tail events are independent —
   at N=10, roughly one request in ten rides
   a tail. Fan-out on the path means per-branch budgets must be set
   at a stricter percentile than the end-to-end promise.
3. **Allocate the budget per hop.** Start from measured reality
   (input 3), assign each hop a number, and include the overhead rows
   folklore omits: serialization/deserialization, connection
   establishment (cold pools, Transport Layer Security (TLS) handshakes), queue wait under load, and ONE
   retry of the most-likely-to-retry hop. The sum of allocations plus
   headroom equals the target — arithmetic that must visibly close.
4. **Hold headroom explicitly.** 15–25% of the target stays
   unallocated — for growth, incident conditions, and the dependency
   nobody predicted. Headroom is a line item with an owner (the path
   owner), not slack silently absorbed by whoever asks first.
5. **Derive timeouts from budgets.** A hop's timeout ≈ its budget
   (plus a stated tolerance), because a timeout longer than the
   budget means the caller has already failed the end-to-end promise
   while still waiting. Cascade correctly: an upstream timeout must
   exceed the longest critical path through downstream calls, including
   serial segments, parallel branch maximums, retries and backoff, or the
   upstream gives up while the downstream still works. The 30-second
   default is a bug in a 300ms path.
6. **Derive retry policy from what the budget affords.** Retries
   spend budget: one retry of a 50ms hop costs 50ms+backoff from
   SOMEONE's allocation. State which hops may retry (idempotent
   ones), how many times (usually one, at the innermost layer only —
   two layers retrying compounds into storms), and hedging only where
   the budget's headroom explicitly funds it.
7. **Design violation attribution.** Per-hop spans with the budget
   annotated (trace attribute or naming convention), so "the p95
   regressed" decomposes into "hop X exceeded its 40ms by 60ms" —
   wiring via `observability-operator` *(manual-only)*; pre-release verification of
   budgets via `performance-test-harness` gates where they exist.
8. **Install the budget-claim review rule.** A new dependency, new
   hop, or new work on a budgeted path claims budget in the design
   review: from headroom (path owner signs), from another hop
   (that owner signs), or the end-to-end target is renegotiated
   (`slo-reliability-architect` loop). Unclaimed spend is how 300ms
   paths become 700ms paths one pull request (PR) at a time. If the owner must choose a
   claim path, define headroom as reserved time, reallocation as moving
   a named hop's allowance with its owner's agreement, and renegotiation
   as changing the user-facing target through the SLO owner. Compare
   only viable paths against measured latency and the 15% to 25% reserve:
   why each fits, user and hop effects, delivery time, money and upkeep
   costs or unknowns, and benefits and drawbacks. Recommend the path that
   keeps the target and reserve credible, explain why and what evidence
   could change it, then ask one decision question. Do not silently
   consume headroom, rewrite another hop, or change the SLO here.
9. **Deliver** in the Output Format: the budget table whose
   arithmetic closes, timeout/retry derivations, attribution design,
   and the review rule — with per-hop owners named.

Tail-composition math, the overhead-rows checklist, timeout-cascade
worksheet, and the budget-claim review template:
[references/tail-math-worksheet.md](references/tail-math-worksheet.md).

## Output Format

When asking for a build choice, show the explained options and justified
recommendation before the one question.

```
LATENCY BUDGET — <path/journey> @ <percentile>
Target: <N ms p<XX>> — provenance: <SLO ref | contract | ASSUMED (flagged for ratification)>
Path & shape: <hop chain; parallel sections marked; fan-out N noted>
Budget table:
  <hop>: <ms> — owner=<team/role> — current=<measured pXX> — notes=<incl. overhead rows>
  serialization/connection/queue-wait rows: <ms each>
  retry allowance: <hop, count, backoff — funded from whose budget>
  HEADROOM: <ms (15–25%)> — owner=<path owner>
  SUM CHECK: allocations + headroom = target ✓
Tail math: <branch dependence assumption; under independence 1-(0.99)^N = <exposure>; measured or bounded joint tail otherwise>
Timeouts: <per hop: timeout ≈ budget + tolerance; cascade check upstream ≥ downstream×retries+ε>
Retries:  <which hops, innermost-layer-only rule, hedging iff headroom-funded>
Attribution: <span/budget annotation convention → observability-operator;
              pre-release check → performance-test-harness>
Review rule: new dependency on this path claims budget (headroom | reallocation | renegotiate SLO)
Owner claim choice (only if needed): <terms; viable paths and reasons;
  user/hop effects, pros/cons, time/money/upkeep or unknowns;
  recommendation and why; one decision question; owners still sign>
Reduction work routed: <hop over budget today → owning skill (query-plan-reader | caching-strategy-designer | frontend-perf-engineer | ...)>
```

## Validation Checklist

- [ ] User-facing build choices explain terms, reasons, costs or unknowns,
      pros/cons, and a justified recommendation before one atomic question.
- [ ] The target's percentile is stated and singular per document; the
      provenance is an SLO ref or a LOUD assumption flag.
- [ ] The budget table's arithmetic closes: allocations + headroom =
      target, visibly.
- [ ] Overhead rows exist (serialization, connections, queue wait) —
      the budget that omits them is fiction by construction.
- [ ] Fan-out sections carry the tail-exposure calculation and
      stricter per-branch percentiles.
- [ ] Every timeout derives from a budget and the cascade check passes
      (upstream ≥ downstream × retries + ε).
- [ ] Retries are budgeted spend at one layer only; hedging is
      headroom-funded or absent.
- [ ] Headroom is a named line item with an owner, 15–25%.
- [ ] Every hop has an owner; hops over budget today have their
      reduction work ROUTED, not absorbed into this design.
- [ ] The budget-claim review rule is stated.
- [ ] Any owner-facing claim choice explains terms, effects, costs or
      unknowns, pros/cons, and a reasoned recommendation before one
      question; no budget or SLO changes silently.

## Gotchas

- Percentiles don't add: p99(A)+p99(B) is not p99(A+B). The sum can
  overestimate or underestimate the path's p99 depending on dependence and
  distribution. Budget per-hop using measured or bounded joint behavior,
  and state the assumptions.
- The fan-out tail trap: with ten independent branches, each having a 1%
  chance of exceeding its threshold, 1-(0.99)^10 ≈ 9.6% of requests have
  at least one exceedance. Correlated branches can change that figure.
  Choose branch budgets from measured or bounded joint tails, or fund a
  hedging strategy from headroom.
- Serialization is a hop: JSON encode/decode of a fat payload can
  cost more than the service call it wraps — and it appears in no
  service's dashboard because it happens in the caller. The overhead
  rows exist because the unmeasured milliseconds are still spent.
- Timeout inversion: an upstream 2s timeout over a downstream
  3s-timeout×2-retries path means the upstream abandons work the
  downstream completes — wasted load AND a failed request. The
  cascade check is mechanical; run it.
- Two-layer retries compound: the gateway retries twice, the client
  library retries twice — one flaky hop becomes 4× load and a storm.
  Retries live at ONE stated layer (usually innermost idempotent).
- Queue wait is invisible until it isn't: async hops budgeted at
  processing time only will blow their budget under load exactly when
  it matters — queue wait at target load is a budget row.
- The budget negotiated against wishes: allocating 20ms to a hop
  currently measuring p95=80ms is not a budget, it's a hope — the
  gap becomes ROUTED reduction work (the owning skill) or the
  allocation is honest about today.
- Cold connections on the tail: pool exhaustion and TLS setup land on
  exactly the requests already unlucky — tail budgets include
  connection establishment or the p99 promise excludes cold starts
  explicitly.

## Stop Conditions

- No end-to-end target exists and none can be assumed responsibly
  (no SLO, no contract, no product number) → stop; route target-
  setting to `slo-reliability-architect`. A budget without a target
  is allocation theater.
- Asked to SET the SLO, define error budgets, or design paging as
  part of the budget work → decline that slice; that is
  `slo-reliability-architect`'s scope, and doing it here forks the
  reliability source of truth.
- The measured reality exceeds the target before allocation begins
  (the path is already slower than the promise) → present the honest
  gap and stop for a decision: renegotiate the target (SLO loop) or
  commission the reduction work (routed to owning skills) — a budget
  cannot be arithmetic'd into existence.
- Per-hop measurements don't exist and the path map is doc-only →
  deliver only a provisional budget marked as such, with the tracing
  gap named as the first work item (via `observability-operator` *(manual-only)*);
  refuse to present a doc-derived budget as engineering truth.
- A third-party dependency's latency distribution is outside the
  budget and cannot be bounded (no service level agreement (SLA), observed tail beyond the
  whole target) → surface the structural conflict for a human
  decision (remove from path, make async, accept the miss rate) —
  do not bury an unbudgetable hop inside a closing table.

## Supporting Files

- [references/tail-math-worksheet.md](references/tail-math-worksheet.md)
  — tail-composition math with worked fan-out examples, the
  overhead-rows checklist, timeout-cascade worksheet, retry-spend
  accounting, and the budget-claim review template.
- `evals/evals.json` — behavior cases including the fan-out tail edge
  and the refuse-to-invent-the-SLO case.
- `evals/trigger-evals.json` — discrimination against
  `slo-reliability-architect` (THE seam), `performance-test-harness`,
  `observability-operator`, and the per-hop reduction skills.