Observability And Operations

---
name: observability-and-operations
description: "Configure BentoML logging, metrics, monitoring/data collection,
  tracing, resource settings, scaling/autoscaling, gateways, testing, and
  production operational troubleshooting. Use when work involves operational
  knobs rather than defining APIs, packaging Bentos, basic serving, or cloud
  deployment CRUD."
disable-model-invocation: true
metadata:
  disco-role: operating
license: Apache 2.0
---

# BentoML Observability and Operations

Use this sub-skill when a BentoML Service needs production diagnostics, telemetry, resource/concurrency tuning, or operational validation.

## Route Here For

- Access logging controls, `bentoml` library logger setup, trace/log correlation fields, and log-level confusion.
- Prometheus `/metrics`, default/custom metric definitions, histogram bucket tuning, and missing metric debugging.
- `bentoml.monitor(...)` data collection, local monitor log files, OTLP monitoring targets, and plugin collector caveats.
- OpenTelemetry tracing with `zipkin`, `jaeger`, or `otlp`, including missing exporter dependencies and environment-variable precedence.
- `resources`, `workers`, `traffic`, `runner_probe`, health endpoints, GPU allocation, CUDA visibility, batching/concurrency symptoms, and BentoCloud autoscaling/gateway planning.
- Operational testing patterns for health, metrics, ASGI/HTTP behavior, and safe BentoCloud E2E test design.

## Boundaries

- For service class and API decorator shape, route to `../service-authoring/SKILL.md`.
- For `bentoml serve`, HTTP/gRPC clients, and server startup basics, route to `../serving-and-clients/SKILL.md`.
- For `bentoml deploy`, deployment CRUD, secrets, and cloud CLI management, route to `../cli-and-cloud/SKILL.md`.
- For Bento build/runtime image and package configuration, route to `../packaging-and-containerization/SKILL.md`.

## Core Workflow

1. Identify the operating target: local serve, container runtime, BentoCloud Deployment, Gateway, or test-only environment.
2. Place runtime config on the `@bentoml.service(...)` decorator unless the project intentionally uses a deployment config file for BentoCloud-only scaling policies.
3. Keep local controls separate from BentoCloud controls: `max_concurrency`, `workers`, metrics, logs, tracing, and monitoring affect local/runtime behavior; `resources`, `traffic.concurrency`, `traffic.external_queue`, scaling min/max, policies, and gateways are BentoCloud scheduling/routing concerns.
4. Verify safe endpoints first: `/livez`, `/readyz`, `/healthz`, then `/metrics`; only then debug external collectors or BentoCloud routing.
5. Use `references/troubleshooting.md` for symptom-to-cause triage before changing unrelated service code.

## References

- `references/observability.md`: logging, metrics, monitoring, tracing, and endpoint checks.
- `references/configuration.md`: runtime config fields, resources/workers/GPU, traffic, autoscaling, gateways, and testing plans.
- `references/troubleshooting.md`: operational symptoms and fixes.

## Bundled Scripts

- `scripts/check_observability_config.py`: validates a local JSON/YAML service or deployment-like config file for common observability/operations mistakes.
- `scripts/resource_plan_template.py`: emits a self-contained local/BentoCloud resource, GPU, autoscaling, gateway, and verification plan template.

Both scripts are local-only helpers. They do not contact BentoCloud, Prometheus, tracing backends, gateways, or external collectors.