Buildbuddy Api

---
name: buildbuddy_api
description: >-
  Reference for querying the BuildBuddy API: build/test logs, invocations, RBE
  execution details, cache stats, undeclared test outputs, target pass/fail
  history for bisecting, workflow re-runs. Use for any why-did-CI-fail,
  show-me-the-log, or when-did-this-test-break task.
---

# BuildBuddy API

## Prerequisites

All commands require `BUILDBUDDY_API_KEY` to be set (session hook exports it automatically).

## Official `bb` CLI

The official BuildBuddy CLI (`bb`) handles build logs and execution details:

```bash
# View build log for an invocation (replaces bbapi invocation log)
bb view <invocation-id-or-url> [--lines=100000]

# Fetch cached execution response for a single execution ID
bb execution get <execution-id> [--output=json]

# Download raw blob from CAS by digest
bb download <digest>/<size> [--type=Action|Command]
```

## `bbapi` CLI

`bbapi` provides API query features not available in `bb`:

```bash
# Show invocation details (shows child invocation IDs for workflows)
bbapi invocation <invocation-id>

# List recent invocations (auto-detects repo from git remote)
bbapi invocation list [--repo URL] [--count N]

# Download test.log for a specific target (most common for debugging failures).
# A sharded target has one log per shard: the failing ones are printed, each under a header
bbapi target log <invocation-id> <target-label-or-substring>
bbapi target log <invocation-id> <target> --shard 4    # one shard (numbered from 1)
bbapi target log <invocation-id> <target> --all        # every shard, each under a header

# List targets in an invocation (auto-resolves workflow/runner IDs to child)
bbapi target <invocation-id> [--filter SUBSTR] [--label LABEL]

# Show pass/fail/flake history for targets (auto-detects group_id)
bbapi target history [--repo URL] [--label LABEL] [--failures-only]

# Show flake statistics for targets (last 7 days)
bbapi target stats [--repo URL]

# Show flake samples for a specific target
bbapi target flakes <target-label> [--repo URL]

# List artifacts for an invocation (test outputs and build outputs). RESULT and STATUS
# tell the shards, runs and attempts of one test apart
bbapi artifact list <invocation-id>

# Narrow to one kind: "build" is a completed target's output files, "test" is
# test.log / test.xml / undeclared outputs
bbapi artifact list <invocation-id> --kind build

# Stream a matching artifact to stdout
bbapi artifact cat <invocation-id> <name-substring>

# Download a matching artifact to a file (defaults to artifact basename;
# override with -o/--output)
bbapi artifact download <invocation-id> <name-substring> [-o PATH]

# List invocation-level build tool logs such as Bazel command profiles
bbapi tool-log list <invocation-id>

# Stream an inline or bytestream-backed build tool log to stdout
bbapi tool-log cat <invocation-id> "critical path"

# Download a Bazel JSON profile emitted as a build tool log
bbapi tool-log download <invocation-id> command.profile.gz [-o PATH]

# Download logs from all child invocations of a workflow/runner invocation
bbapi tool-log download <runner-invocation-id> command.profile.gz --all -o profiles/

# List remote executions for an invocation
bbapi execution <invocation-id>

# Search remote executions across invocations
bbapi execution search <query>

# List the output files of a remote execution (path, size, digest, exec bit)
# The execution-id is the EXECUTION column from `bbapi execution <invocation-id>`
bbapi execution files <invocation-id> <execution-id>

# Aggregated invocation statistics (by branch, user, commit, etc.)
bbapi invocation stat [--agg-type branch|user|host|repo|commit|pattern] [--repo URL] [--limit N]

# AI analysis of a build/test failure (works on any invocation, unlike `bb ask`)
# Sends the last ~1000 lines of the build log (from first "ERROR:" onward, max 8KB)
# to OpenAI. Best for build errors with clear error messages in the log tail.
# Less useful for test timeouts (log tail may show unrelated warnings instead).
bbapi ask <invocation-id> [--prompt TEXT]

# List configured BuildBuddy workflows
bbapi workflow

# Trigger a workflow run (auto-detects workflow ID, branch, commit from git)
bbapi workflow run [--workflow-id ID] [--branch BRANCH] [--commit SHA] [--action NAME] [--async]

# Show cache scorecard (per-action hit/miss)
bbapi cache <invocation-id>

# Get metadata for a cached artifact by digest
bbapi cache metadata <digest> <size-bytes>

# Show build performance trends
bbapi trend [--days N] [--repo URL]
```

All `bbapi` commands support `--json` for raw JSON output.

Responses are parsed against pinned BuildBuddy protos but unknown fields are
discarded, so `bbapi` tolerates API/proto drift between repins (newly added
response fields) instead of hard-failing on the parse.

## Investigating Failed CI Builds

Typical workflow for debugging a failed CI build:

```bash
# 1. Get invocation details — note the Child: line for workflow invocations
bbapi invocation <invocation-id>

# 2. Get the build log to see which tests failed
bb view <invocation-id>

# 3. Download the test.log for a specific failed target
#    Works with both workflow and child invocation IDs (auto-resolves)
bbapi target log <invocation-id> <target-substring>
#    Example: bbapi target log 870a5be1-c296-4792-8c8a-77def20b2dcc test_handlers
```

### Workflow vs Child Invocations

BuildBuddy CI runs use **workflow invocations** that spawn **child invocations**.
The workflow invocation (command: `workflow run`) is a wrapper; the child
invocation contains the actual `bazel test` results, targets, and artifacts.

- `bbapi invocation` shows `Child: <child-id>` for workflow invocations
- `bbapi artifact {list,cat,download}` and `bbapi target log` auto-resolve
  workflow invocations to their children — you can pass either the workflow
  or child ID
- `bbapi target` also auto-resolves workflow invocations to their children

### Artifact Name Matching

`bbapi artifact {cat,download}` and `bbapi target log` match against `"label/name"`:

- `"test_lifecycle"` matches `//mcp_infra/compositor:test_lifecycle/test.log`
- `"test_lifecycle/test.xml"` matches the XML output specifically
- `"compositor/test_lifecycle"` matches `//mcp_infra/compositor:test_lifecycle/test.log`

When no match is found, the CLI prints available labels as hints, and the names that would
match if `-` and `_` were alike. Names are what the test wrote, not what you would derive from
a target or a scenario (a visual scenario's PNG is `test.outputs/<outputName>-actual.png`, where
the outputName is hyphenated for some scenarios and not for others), so `bbapi artifact list`
first beats guessing.

### Sharded, repeated and retried tests

A test with `shard_count`, `--runs_per_test` or `--flaky_test_attempts` reports one result per
shard, run and attempt, and each has its own `test.log`, `test.xml` and outputs under the same
names. `bbapi artifact list` gives each its `RESULT` (`shard 4/6`, `run 2/3`, `attempt 2`) and
its own `STATUS` (`PASSED`, `FAILED`, `TIMEOUT`, ...); `--json` adds `shard`, `shardCount`,
`run`, `runCount`, `attempt` and `status`. Numbers count from 1, as in
`bazel-testlogs/.../shard_4_of_6/`.

Narrow with flags on `artifact {list,cat,download}` and `target log`:

```bash
bbapi artifact list <id> --failed                    # only results that did not pass
bbapi artifact download <id> test.log --shard 4      # one shard's log
bbapi artifact download <id> test.log --all -o logs/ # every shard's log, none overwriting another
bbapi target log <id> visual --failed                # the failing shards' logs, each with a header
bbapi target log <id> visual --all                   # every shard's log
```

Several matches never mean "pick one silently": `target log` lists every match with its shard and
status on stderr, then prints the results that failed, or the first if none did. `artifact cat`
and `download` without `--all` still take the first match, and say which. With `--all`, files of
one name are saved as `<label>__shard_4_of_6__test.log`; a name nothing else shares keeps its own.

### Three kinds of file, three commands

Bazel files attached to the invocation itself, such as `command.profile.gz`, are BES
`BuildToolLogs` — use `bbapi tool-log {list,cat,download}`. They can be inline
(`elapsed time`, `critical path`, `process stats`) or bytestream-backed
(`command.profile.gz`); `bbapi tool-log cat` handles both.

Everything a target produced is `bbapi artifact`, in two kinds:

- `--kind test` — outputs of a test action: `test.log`, `test.xml`, files under
  `test.outputs/`.
- `--kind build` — files in a completed target's output groups: a wheel, a
  `.skill`, an `oci_image`'s `.json.sha256`. Lint aspects contribute their own
  groups, so filter on the `GROUP` column (`default` for a target's real
  outputs, `rules_lint_report` / `mypy` / `clippy_checks` / `rustfmt_checks` for
  the aspects).

**Deviation from what BuildBuddy's web UI shows:** the UI surfaces test artifacts
only. Build outputs reach the BES stream as `TargetComplete` → output group →
`NamedSetOfFiles`, and `bbapi` walks that graph.

`--json` carries each file's `digest` and `size` as BES reports them. For a
single-file release the digest is the content identity, so a caller can compare
against a published tag without fetching the bytes.

**Gotcha:** identify a file by `pathPrefix` + `name`, never `name` alone. A
source file has an empty prefix while the generated file of the same name lives
under `bazel-out/k8-fastbuild/bin`, and a configuration transition writes to
`bazel-out/k8-fastbuild-ST-<hash>/bin`. On devel's `//...` sweep four names
collide across prefixes.

For CI phase profiling, start with:

```bash
bb view <runner-or-child-invocation> --lines=200000
bbapi invocation <runner-invocation>   # shows child invocations
bbapi tool-log list <runner-invocation>
bbapi tool-log cat <child-invocation> "process stats"
bbapi tool-log download <runner-invocation> command.profile.gz --all -o profiles/
```

The runner log line `Syncing existing repo...` identifies a warm outer `bb remote` runner
workspace, but it does not by itself prove the inner Bazel analysis cache survived. Check
Bazel's package/configuration counts, elapsed time, `targetConfiguredCount`, and
`command.profile.gz`. Treat `targetConfiguredCount` as the configured graph size/result
count, not by itself proof that those configured targets were recomputed; profile markers,
`discarding analysis cache` warnings, and time-to-first-action are better recomputation
signals.

## Bisecting Test Failures with Target History

When a test is failing and you need to find the commit that broke it, use target
history instead of `git bisect` — BuildBuddy already has all the results:

```bash
# 1. Check recent pass/fail history for the target (failures only)
bbapi target history --failures-only --label //path/to:test_target

# 2. Identify the transition point (last pass → first fail)
#    The output shows invocation IDs and commit SHAs for each run

# 3. Narrow to the commit range
git log --oneline <last-pass-commit>..<first-fail-commit>

# 4. Read the test log from the first failing invocation
bbapi target log <first-failing-invocation-id> test_target
```

This is much faster than `git bisect` because it doesn't require re-running the
test — the results are already in BuildBuddy's database.

## Diagnosing Executor Environments

Use `bb execute` to run one-off commands directly on a BuildBuddy executor to probe the container image, check installed tools, verify library paths, etc. Useful when builds fail due to missing dependencies or environment issues.

```bash
# Probe what's in a container image
bb execute \
  -exec_properties=container-image=docker://ghcr.io/agentydragon/rbe-worker:nix-devtools \
  -- bash -c 'gcc --version; python3 --version; ldd --version | head -1'

# Check if a specific library exists
bb execute \
  -exec_properties=container-image=docker://ghcr.io/agentydragon/rbe-worker:latest \
  -- bash -c 'find / -name "libstdc++.so*" 2>/dev/null'

# Test pip wheel compatibility (manylinux tags)
bb execute \
  -exec_properties=container-image=docker://ghcr.io/agentydragon/rbe-worker:latest \
  -- bash -c 'python3 -m pip install --dry-run some-package==1.0'
```

`bb execute` runs on the default executor image (Ubuntu 16.04, glibc 2.23) unless you specify `-exec_properties=container-image=...`. Use `-exec_properties=workload-isolation-type=firecracker` to test with Firecracker isolation.

### Interactive debugging with `bb ssh`

For interactive debugging, start an SSH server on a BuildBuddy executor:

```bash
# On the executor (in one terminal)
bb ssh-server my-debug-session

# Connect from your machine (in another terminal)
bb ssh my-debug-session
```

This gives a full shell on the executor for investigating build failures, inspecting the filesystem, testing commands interactively, etc.

## BuildBuddy Concepts

**Group ID**: BuildBuddy's organization identifier (e.g., `GR7963402054611859571`).
Scopes API queries to the org's data. Auto-detected by `bbapi` from a recent invocation's
ACL — no manual configuration needed.

**Workflow ID**: For repos using `buildbuddy.yaml` + GitHub app, workflow IDs are synthetic:
`WF#GitRepository:{group_id}:{repo_url}`. The `bbapi workflow run` command auto-constructs
this from the detected group_id and repo URL. `GetWorkflows` returns empty for these repos
(it only lists explicitly created workflows).

## Raw API Fallback

If `bbapi` is not available, use the Twirp JSON API at `app.buildbuddy.io` directly
with curl. Read `devinfra/buildbuddy_cli/client.go` for how the CLI talks to the API
(Twirp JSON over HTTP). The API key comes from `BUILDBUDDY_API_KEY` env var, or
parse it from `~/.config/bazel/buildbuddy.bazelrc` (`x-buildbuddy-api-key=...`).

Proto definitions for request/response schemas:

- <https://github.com/buildbuddy-io/buildbuddy/blob/master/proto/buildbuddy_service.proto> (internal, ~70 RPCs)
- <https://github.com/buildbuddy-io/buildbuddy/blob/master/proto/api/v1/service.proto> (public, 9 endpoints)

## Known Limitations

**Fork PRs have BuildBuddy invocations only through `trusted-fork-pr-ci.yml`.**
GitHub Actions does not pass `BUILDBUDDY_API_KEY` to `pull_request` workflows
triggered by forks (head repo != base repo), so `ci.yml` skips `bazel-ci` for them.
Revisions authored by the agent identity or the repository owner run it
automatically through `trusted-fork-pr-ci.yml`; every other author waits for
review of the `fork-ci-review` environment. When investigating a fork PR that
has not run there, BuildBuddy has no record of the run — check GitHub Actions
logs directly instead.