Observability Stack

Name: Observability Stack
Author: anubhavg-icpl

ASecurity

Pick an LLM observability stack (development platform + gateway + optional scale layer) given stack, scale, budget, and license posture, and define the OpenTelemetry GenAI attribute set. Use when you need help with observability stack.

9 stars
0 votes
0 copies
4 views
Added September 8, 2026

development

Security analysis

A100/100

Scanned September 8, 2026

npx -y skills add anubhavg-icpl/vibe --skill observability-stack --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Observability Stack?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Observability Stack

[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/anubhavg-icpl-observability-stack/badge)](https://www.skillsdirectory.com/skills/anubhavg-icpl-observability-stack)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro

SKILL.md

---
name: observability-stack
description: Pick an LLM observability stack (development platform + gateway + optional scale layer) given stack, scale, budget, and license posture, and define the OpenTelemetry GenAI attribute set. Use when you need help with observability stack.
license: CC-BY-NC-SA-4.0
phase: 17
lesson: 13
metadata:
  version: 1.0.0
  tags: [observability, langfuse, langsmith, phoenix, arize, helicone, opik, opentelemetry, genai-conventions]
---

Given stack (LangChain / DSPy / raw SDK), scale (traces/day), budget, license posture (MIT-only vs commercial OK), and self-host requirement, produce an observability plan.

Produce:

1. Development platform choice. Langfuse (OSS), LangSmith (LangChain-first commercial), Opik (Comet OSS), or none. Justify with stack and license.
2. Gateway/telemetry choice. Helicone (proxy + gateway), SigNoz (full APM), OpenLLMetry (pure OTel). If already using an AI gateway (Phase 17 · 19), name the integration.
3. Scale/lake layer. Optional; Arize AX or raw Iceberg for long-term analytics, Phoenix for RAG drift.
4. OTel GenAI conventions. Specify the minimum attribute set: `gen_ai.system`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.request.temperature`, `gen_ai.response.finish_reasons`, plus org-specific (tenant_id, user_id, task).
5. Sampling policy. 100% errors, 100% high-cost (>$0.10/call), N% success sampling rate. Raw-retention window (14d / 30d / 90d). Aggregates retained longer.
6. Alerting. Five metrics that must have alerts: error rate, P99 TTFT, cost/request, prompt-cache hit rate, refusal rate.

Hard rejects:
- Instrumenting inside framework-specific SDK without an OTel fallback. Refuse — framework lock-in.
- Keeping 100% of traces at Datadog-class pricing >$500/mo for a non-regulated workload. Refuse — recommend sampling.
- Ignoring OpenTelemetry GenAI conventions. Refuse — 2026 interop requires them.

Refusal rules:
- If traces/day > 5M and the team insists on full Datadog retention, refuse without a cost forecast.
- If the team is MIT-only and picks LangSmith, refuse — Langfuse is the MIT equivalent.
- If the team has no AI gateway and picks Helicone as gateway AND observability, accept — the proxy doubles as gateway up to ~500 RPS (Phase 17 · 19 covers gateway scale).

Output: a one-page plan naming dev platform, gateway, scale layer (if any), OTel attribute set, sampling rule, five alerts. End with the single metric that signals stack drift: percentage of LLM calls with complete OTel GenAI attributes over last 7 days.