Nvidia Triton Inference Serving Review

---
name: nvidia-triton-inference-serving-review
description: Use this skill when reviewing Triton Inference Server deployments statically — `model_repository/` layout and `config.pbtxt` files, dynamic batching configuration, ensemble and BLS pipelines, custom backend (Python, C++, ONNX, OpenVINO, vLLM) trust posture, gRPC and HTTP endpoint authentication, response cache configuration, rate-limit and metrics exposure. Trigger when the user asks whether a Triton model repository or `tritonserver` invocation follows NVIDIA's published guidance and security expectations.
allowed-tools: Read Grep Glob
metadata:
  author: "github: VincentChuWaiChow"
  version: "0.1.0"
  updated: "2026-05-10"
  category: platform
---

# NVIDIA Triton Inference Server Review

## Purpose

Static review of Triton Inference Server deployments against NVIDIA's Triton documentation — model repository layout, dynamic batching, ensemble pipelines, custom backend trust, gRPC/HTTP authentication, model encryption at rest, response cache poisoning surface. This skill is doc-anchored: it grounds review findings in NVIDIA's published documentation rather than in a certification blueprint, because no NVIDIA certification currently covers this developer-facing surface as a standalone exam objective.

## Lean operating rules

- Prefer the user's actual `model_repository/` tree and `config.pbtxt` files as evidence; otherwise fall back to documentation-based inference.
- Treat custom Python or C++ backends loaded from non-pinned sources or without code review as a critical finding — in-process RCE.
- Treat gRPC or HTTP endpoints exposed without authentication, mTLS, or a restricted-protocol gateway as a critical finding for multi-tenant deployments.
- Treat model repository directories with world-writable permissions or a writable `--model-repository` mount as a high finding — silent model substitution.
- Treat response caching enabled across tenants without per-request cache-key partitioning as a high finding — cross-tenant cache poisoning.
- Treat ensemble or BLS pipelines that pass user-supplied tensors directly to a Python backend without input validation as a medium finding — deserialization surface.
- Treat metrics endpoints (`:8002`) exposed to the public network without scraping ACLs as a medium finding — model name and shape leakage.
- Treat dynamic batching `max_queue_delay_microseconds` left at default with latency SLOs in the millisecond range as a low finding — throughput-vs-latency tuning is wrong by default.
- Always emit the exact `tritonserver` and `perf_analyzer` commands the user should run — do not execute them.

## Response minimum

Return, at minimum:
- the scoped target (model repository layout and provenance, backend trust posture, endpoint and auth posture, batching and ensemble posture, response cache and metrics posture, recommended tritonserver/perf_analyzer invocations) and evidence level,
- findings labelled critical / high / medium / low,
- recommended NVIDIA-tooling invocations the user should run themselves,
- safe next actions and assumptions or blockers.