Otel Profiles

---
name: otel-profiles
description: OpenTelemetry profiles signal and the eBPF profiler (otelcol-ebpf-profiler, `profiling` receiver). Use when setting up continuous profiling with OpenTelemetry, deploying the eBPF profiler on Linux hosts, Docker, or Kubernetes (DaemonSet, kind, k3d), building a profiles pipeline, sending OTLP profiles to Pyroscope, Grafana Cloud Profiles, or another backend, getting flame graphs from OTel, enabling off-CPU profiling, or debugging profiler errors about tracefs, PID namespaces, missing pods, or the service.profilesSupport feature gate. Not for SDK tracing or metrics setup.
---

# OpenTelemetry Profiles

The main OpenTelemetry producer is the [eBPF profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler), shipped as the `profiling` receiver in the [`otelcol-ebpf-profiler`](https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-ebpf-profiler) Collector distribution. It profiles every process on a Linux host with no application changes.

Maturity: the signal entered [public Alpha](https://opentelemetry.io/blog/2026/profiles-alpha/) in March 2026, the OTLP profiles proto (`v1development`) is Development, the `profiling` receiver is Development, and the Collector gate `service.profilesSupport` is alpha. Expect breaking changes; pin the profiler, Collector, and backend versions together.

Collector-side component facts (OTLP receiver/exporter profile support, the fixed `/v1development/profiles` HTTP path, `k8s_attributes`, `memory_limiter`, `filter` `profile_conditions`) live in the `otel-collector` skill's component pages; OTTL `profile`/`profilesample` contexts live in `otel-ottl`.

`otelcol-ebpf-profiler` `0.162.0` bundles only: receivers `profiling`, `nop`; processors `memory_limiter`, `k8s_attributes`, `resource_detection`, `resource`, `transform` (plus `batch`, which does not support profiles); exporters `otlp_grpc`, `otlp_http`, `debug`, `file`, `nop`; extensions `health_check`, `pprof`, `opamp`. For anything else (`filter`, the `basicauth` extension, vendor exporters), relay to a regular Collector or build a distribution (`otel-collector-builder` skill).

## Version matrix (Verified 2026-10-15)

| Artifact | Version | What it means |
|---|---|---|
| `otel/opentelemetry-collector-ebpf-profiler` | `0.162.0` (newest release at verification, 2026-09-29) | Pins ebpf-profiler `v0.0.202636`: boolean `pid_namespace_translation`; does not bundle the `offcpu` extension. |
| Same image, `0.163.0-nightly.*` | prerelease | Bundles `offcpu` from `0.163.0-nightly.3d037de` (2026-10-05); pins ebpf-profiler `v0.0.202640`, with `pid_namespace_translation_mode` ([#1801](https://github.com/open-telemetry/opentelemetry-ebpf-profiler/pull/1801)), from `0.163.0-nightly.2e6f5c3` (2026-10-07). Release `0.163.0` is unpublished; until it is, these need a nightly or an OCB build. |

Evidence: [Docker Hub tags](https://hub.docker.com/r/otel/opentelemetry-collector-ebpf-profiler/tags) (no `0.163.0`), [collector-releases](https://github.com/open-telemetry/opentelemetry-collector-releases/releases) (latest `v0.162.0`), the [distribution manifest](https://github.com/open-telemetry/opentelemetry-collector-releases/blob/main/distributions/otelcol-ebpf-profiler/manifest.yaml) history.

Read [references/receiver.md](references/receiver.md) when setting keys beyond `samples_per_second`, enabling off-CPU or OBI correlation, or pinning a release not in this table (keys follow the pinned ebpf-profiler, not the Collector version).

## Workflow

1. **Pick the image and pin.** Use the latest release unless a needed key or component is missing from it (see the matrix). Alternatives: a pinned nightly, labeled as a prerelease with the release that replaces it, or an OCB build (`otel-collector-builder` skill) based on the [distribution manifest](https://github.com/open-telemetry/opentelemetry-collector-releases/blob/main/distributions/otelcol-ebpf-profiler/manifest.yaml).
2. **Pick the environment.**
   - Linux host or plain Docker: [Run on a host](#run-on-a-host) below.
   - Kubernetes: [references/kubernetes.md](references/kubernetes.md) (DaemonSet, RBAC, Kubernetes metadata, memory sizing).
   - Kubernetes nodes that are themselves containers (kind, k3d, Docker-in-Docker, minikube with the docker driver): also [references/nested-containers.md](references/nested-containers.md).
3. **Wire the pipeline.** `profiles` pipeline: `profiling` receiver, `memory_limiter` first, enrichment processors, then `otlp_grpc` or `otlp_http`. Start the Collector with `--feature-gates=+service.profilesSupport`. Run `validate --config=<file> --feature-gates=+service.profilesSupport` with the target image; it catches wrong keys for the pinned version.
4. **Export and verify.** Add `debug` (`verbosity: basic`) first: each `reporter_interval` it logs `"resource profiles": N, "sample records": M`; no lines or zero counts mean the receiver is not producing (see troubleshooting). `verbosity: detailed` shows resource attributes. Then add the backend: [references/backends.md](references/backends.md) for endpoints, auth, and per-backend label behavior.
5. **Troubleshoot** by exact error string: [references/troubleshooting.md](references/troubleshooting.md).

## Run on a host

The profiler needs root or `CAP_SYS_ADMIN`/`CAP_BPF`/`CAP_PERFMON`, the host PID namespace, and a mounted tracefs. A container gets a fresh `/sys` without tracefs even when privileged, so bind-mount it:

```bash
docker run -d --name otel-ebpf-profiler --privileged --pid=host \
  -v /sys/kernel/tracing:/sys/kernel/tracing \
  -v "$PWD/config.yaml:/etc/otelcol/config.yaml:ro" \
  otel/opentelemetry-collector-ebpf-profiler:0.162.0 \
  --config=/etc/otelcol/config.yaml --feature-gates=+service.profilesSupport
```

```yaml
receivers:
  profiling:
    samples_per_second: 20      # default
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 800
    spike_limit_mib: 200
exporters:
  debug:
    verbosity: basic
  otlp_grpc:
    endpoint: profiles-backend:4317   # backend-specific, see references/backends.md
    tls:
      insecure: true
service:
  pipelines:
    profiles:
      receivers: [profiling]
      processors: [memory_limiter]
      exporters: [otlp_grpc, debug]
```

The image is built `FROM scratch`: no shell, so debug from logs, not `exec`. Expected startup lines: `eBPF tracer loaded`, `Attached sched monitor`, `Everything is ready`.

## Known limitations

- The receiver profiles every process; it has no process, container, or namespace selection, and upstream declined to add one ([#358](https://github.com/open-telemetry/opentelemetry-ebpf-profiler/issues/358)). To keep a subset, filter on enrichment attributes with `filter` `profile_conditions`; see "Keep only some workloads" in [references/kubernetes.md](references/kubernetes.md). Sampling cost on the node stays the same; only exported volume drops.
- Linux `amd64`/`arm64`, kernel 5.10+ (checked at config validation; `no_kernel_version_check` bypasses it). Backends may lag the proto; check which OTLP profiles version they accept.
- Trace-to-profile correlation is partial: OTLP profiles can link samples to `trace_id`/`span_id`, but the profiler fills them only from a context producer (OBI via `obi_process_ctx`; the [thread-context OTEP](https://github.com/open-telemetry/opentelemetry-specification/pull/4947) implementation is in progress). Go pprof labels are read as custom labels.