Docs

# ffmpeg-skill relationship

color-grading-skill is a client of ffmpeg-skill's **public contract** (`ffmpeg-skill contract --json`,
`contract_version 1.0`). It works with whichever ffmpeg-skill version is installed, as long as that version has the
flags below: compatibility is detected from the checkout itself, never decided by a version window (docs/decisions.md
ADR-19). CI runs against ffmpeg-skill main and against the oldest verified commit (0.12.3). It never calls `ffmpeg` or
`ffprobe` itself.

## Tools and flags used

| ffmpeg-skill tool | used for | flags emitted |
|---|---|---|
| `ffmpeg-skill/probe` | source facts, every artifact's validation | positional `inputs` (one path) |
| `ffmpeg-skill/color` | `HDR_TO_SDR` (`--to-sdr`), `LUT_APPLY` (`--lut`), `RETAG` (`--retag`), `STRIP_DOVI` (`--strip-dovi`), `PRIMARY_CORRECTION` (`--correct`) | `--to-sdr`, `--lut`, `--retag`, `--strip-dovi`, `--correct` (mutually exclusive, exactly one required — matches this skill's one-operation-per-node model exactly), `--tonemap`, `--peak`, `--desat`, `--force`, `--lut-strength`, `--exposure`, `--contrast`, `--saturation`, `--temperature`, `--tint`, `--gamma`, `--lift`, `--gain`, `--levels-in-black`, `--levels-in-white`, `--levels-out-black`, `--levels-out-white`, `--curves`, `--audio-stream`, `--quality` or `--crf` (whichever the checkout has), `--preset`, `-o`, `--json` |

`doctor` checks that every flag listed above exists in the tool's generated `input_schema` (and, when the contract
declares it, that `color` has `video_required: true`); a missing flag is a `fail` naming the flag, and `run` refuses
with `TOOL_ERROR` (`ffmpeg_skill_incompatible`). The version number is reported, never compared.

## Capability detection (no version window)

`adapter.FfmpegSkill.info()` reads the flags from the first source that answers: `scripts/_contract.py --json
--static`, `scripts/_contract.py --json`, `scripts/contract.py --json`, and finally each tool's own `--help`
(`adapter.flags_from_help`). What it found is in `doctor --json`'s `checks.ffmpeg_skill.detected`:

| difference between releases | detected by | what this skill does |
|---|---|---|
| `--crf` became `--quality` (added 1.10, `--crf` removed from `color.py` in 2.0; same CRF scale and default) | which of `quality` / `crf` the `color` input_schema has | emits `--quality` when present, else `--crf`, with the request's `crf` value (`encoder_quality_flag`) |
| `scripts/_common.py` became a `scripts/_common/` package (2.0) | the files on disk | nothing: this skill only runs `probe.py`/`color.py`; reported as `common_layout` |
| an existing output is refused unless `--overwrite` (1.10 opt-in, 2.0 default) | `overwrite` in the `color` input_schema | nothing: every intermediate path is deleted before its `color.py` call, so the tool never sees an existing output and no overwrite consent is given; reported as `overwrite_flag` |
| probe's `hdr` narrowed to PQ/HLG/Dolby Vision; `bt2020_or_hdr` carries the 1.x meaning (2.0) | which key probe's `video` object has | `adapter.probe_hdr` reads `bt2020_or_hdr` when present, else `hdr`: BT.2020 primaries or a PQ/HLG transfer in every version |
| no `_contract.py doctor` (a checkout without it) | the file on disk | ffmpeg/ffprobe/filter capabilities are `unknown`, not `unsupported`; the run and output validation decide |

The guarantees the former window encoded are implied by the flags: `--gamma`/`--lift`/`--gain`/`--levels-*`/
`--curves` first exist in 0.12.3, which is after both `--audio-stream` (0.12.0) and honest `dropped_non_av_streams`
reporting from `--to-sdr`/`--lut`/`--correct` (0.12.1). So a checkout that passes the flag check also has those, and
no version number needs comparing. The result JSON top-level keys this skill reads (`status`, `error`,
`measurements`, `reencoded`, `dropped_non_av_streams`, `commands`) are unchanged across 0.12.3 – 2.x.

## Why 0.12.3 is the oldest release that works

Every flag this skill emits was read directly from `scripts/color.py`'s argparse definition and confirmed present in
`scripts/_contract.py --json --static`'s generated `input_schema` at ffmpeg-skill 0.12.3 (0.9.1 lacked `--correct`
and its five original parameter flags, so PRIMARY_CORRECTION raised the window to 0.9.2 first, docs/decisions.md
ADR-15; `--audio-stream` does not exist in `color.py` before 0.12.0, and `--to-sdr`/`--lut`/`--correct` did not
honestly report `dropped_non_av_streams` before 0.12.1 — RETAG's own `reencoded`/`dropped_non_av_streams` pair is
0.12.0, docs/decisions.md ADR-17; `--gamma`/`--lift`/`--gain`/`--levels-*`/`--curves` do not exist in `color.py`
before 0.12.3, docs/decisions.md ADR-18). An earlier checkout lacks those flags, so the flag check refuses it and
names them; no version comparison is needed for that. There is no upper bound: ffmpeg-skill's 2.x compatibility
promise covers argument names and JSON keys, and the one rename it made (`--crf` → `--quality`) is detected.

## Observed behaviour this skill relies on (measured by reading ffmpeg-skill 0.12.3's `scripts/color.py`)

- `color.py` requires a video stream (`die("input has no video stream")` when `probe` reports none) and refuses to
  run without exactly one of `--to-sdr` / `--lut` / `--retag` / `--strip-dovi` / `--correct` (an argparse
  mutually-exclusive, required group) — this is exactly this skill's one-operation-per-node model; there is no
  request shape in which color-grading-skill could ask ffmpeg-skill/color to do more than one of these in a single
  call.
- `--to-sdr` refuses a source that is not tagged HDR (`color_transfer` / `color_primaries`) unless `--force` is
  given; it builds a `zscale → format=gbrpf32le → zscale → tonemap → zscale → format=yuv420p` chain from the
  *measured* input tags (falls back to `smpte2084`/`bt2020`/`bt2020nc`/`tv` when a tag is missing) and always
  re-encodes with libx264 (`x264_args`), carrying audio through AAC when present. **Measured**: `--force` on a
  source that genuinely never was PQ/HLG-encoded (no HDR tags, ordinary SDR content) can make the zscale/tonemap
  filter chain itself fail ("Nothing was written into output file..."), since the chain assumes a PQ-range input.
  This is a real ffmpeg failure surfaced as `TOOL_ERROR`, not a defect in this skill's validation; `force: true` is
  the caller's explicit request to treat unlabelled content as HDR anyway, and that can legitimately fail.
- `--retag`'s stream-copy path (`-c copy -colorspace/-color_primaries/-color_trc`) only reliably changes what
  `ffprobe` reports when the source's own bitstream (the H.264/HEVC SPS VUI) does not already carry an explicit,
  conflicting colour description. **Measured** with ffmpeg 6.1.1: retagging a file whose H.264 stream was encoded
  with explicit `-colorspace/-color_primaries/-color_trc` (baked into the SPS) leaves `ffprobe`'s reported tags
  unchanged after `--retag`, even though `color.py` reports `wrote ... (tags -> ...)` and exits 0 — the container-
  level override does not rewrite already-encoded VUI bits, and `color.py`'s fallback to a re-encode only triggers
  on a non-zero ffmpeg exit code, which this case does not produce. Retagging a source with *no* explicit tags (the
  common case `--retag` is documented for: "the colours are tagged wrong", i.e. missing or incorrect metadata on
  otherwise untagged footage) works as documented. This skill's own output validation (`executor._validate_artifact`,
  the `RETAG` branch) re-probes the output and compares the exact tag triple against `model.RETAG_TAGS[target]`, so
  a retag that silently did not take effect is reported as `VALIDATION_ERROR` (`reason: retag_mismatch`), never as a
  false success — the gap is ffmpeg-skill's, the honesty about it is this skill's.
- `--lut` applies `lut3d=file=<path>:interp=tetrahedral`; **`--lut-strength` blends the graded and original streams
  only for a value strictly between 0 and 1** (`0 < args.lut_strength < 1` in the source) — a strength of exactly
  `0.0` is *not* "no LUT": ffmpeg-skill applies the LUT at full strength for both `0.0` and `1.0`. This skill
  accepts the documented `[0.0, 1.0]` range unchanged (it is ffmpeg-skill's contract, not a bug this skill works
  around) and states the caveat in the parameter's own description (`model.OPERATION_TYPES["LUT_APPLY"]`).
  Always re-encodes with libx264; an HDR source keeps whatever tags the input had (the LUT path does not retag).
- `--retag` attempts a stream copy first (`-c copy -colorspace … -color_primaries … -color_trc …`); ffmpeg-skill
  itself falls back to a re-encode (`x264_args(crf, preset, keep_bt709=False)`) only when the stream copy fails for
  a given codec/container combination. This skill's `RETAG` parameters carry no `crf`/`preset` (unlike
  `HDR_TO_SDR`/`LUT_APPLY`): the common case is a stream copy where they have no effect, and ffmpeg-skill's own
  defaults (18 / medium) apply on the rare fallback. `audio_stream` is different: `color.py` forwards it
  (`-map 0:a:{audio_stream}?`) into that same re-encode fallback, so `RETAG` does carry `audio_stream` even though
  it carries no `crf`/`preset` — on the common stream-copy path it has no effect (every audio track is kept, `-map
  0`), same as `crf`/`preset`, but it is not simply irrelevant to `RETAG` the way they are. The four targets and
  their exact tag triples (`color_space, color_primaries, color_transfer`) are ffmpeg-skill's own mapping, copied
  verbatim into `model.RETAG_TAGS` and used again to verify the output.
- `--strip-dovi` only applies to HEVC video (`die` otherwise) and is always a stream copy
  (`-bsf:v filter_units=remove_types=62 -tag:v hvc1`), removing the Dolby Vision RPU NAL unit; it does not require
  the input to already carry detected Dolby Vision side data (it warns and proceeds), but this skill still verifies
  the *output* no longer reports `dolby_vision` after the call.
- `--correct` builds a fixed filter chain from four always-present stages plus two stages appended only when asked
  for: `exposure=exposure=<value>` (the dedicated `exposure` filter, distinct from `eq`'s brightness — confirmed
  via `ffmpeg -h filter=exposure` vs. `-h filter=eq`), `colortemperature=temperature=<value>` (6500 = neutral),
  `colorbalance=rs=<lift>:gs=<lift>:bs=<lift>:rm=<tint/2>:gm=<-tint>:bm=<tint/2>:rh=<gain>:gh=<gain>:bh=<gain>`
  (a single `colorbalance` call covering all three tonal ranges: `lift`/`gain` set the shadow/highlight channels
  directly, `tint` is expressed as a green(-1)/magenta(+1) midtone balance shift — magenta raises red and blue, cuts
  green), and `eq=contrast=<value>:saturation=<value>:gamma=<value>` (gamma folded into the same `eq` term
  contrast/saturation already use, as `eq`'s own gamma option) — always in that order, and always present because
  every one of those eight flags (`exposure`/`contrast`/`saturation`/`temperature`/`tint`/`gamma`/`lift`/`gain`) has
  an identity default value, so emitting the flag is harmless even unchanged. `colorlevels=<r/g/b in/out min/max>`
  (the four `--levels-*` 8-bit values divided by 255.0 into the filter's native 0..1 range) is appended only when
  they differ from `(0, 255, 0, 255)`, and `curves=preset=<name>` only when `--curves` was given — omitting either
  already gets the identity result without an extra filter term. The whole chain re-encodes with libx264
  (`--crf`/`--preset`). Every one of the thirteen typed flags is range-checked by `color.py` itself before ffmpeg
  runs (`die()` on an out-of-range value, or on `levels_in_black >= levels_in_white` / `levels_out_black >=
  levels_out_white`, matching the same ranges and ordering this skill's `PRIMARY_CORRECTION` parameters declare and
  validate) — this skill's own range and ordering checks in `model.py` duplicate ffmpeg-skill's guarantee at the
  request boundary (fail fast, before a subprocess starts) rather than relying on it exclusively.
  `--correct` also reports a `measurements` field: `{"input": analyze_levels(source), "output":
  analyze_levels(output)}`, reusing ffmpeg-skill's existing signalstats-based `analyze_levels` (the same primitive
  `probe.py --analyze` uses for Log-footage detection) — real, technical numbers (`y_min`/`y_max`/`y_avg`/
  `saturation_avg`), never a subjective "looks better" judgement. This skill carries that field through unchanged
  into `NodeState.measurements`, the manifest and provenance.
- Failure document: `{"status": "failed", "error": {"kind": "input|ffmpeg|missing_tool", "message"}}` with a
  non-zero exit; parsed into `TOOL_ERROR` with `details.error_kind`.
- `ffmpeg-skill/probe`'s `video.hdr` (before 2.0; `video.bt2020_or_hdr` from 2.0, where `hdr` means the transfer
  only) is `true` when `color_transfer` is `smpte2084` or `arib-std-b67`, or `color_primaries` is `bt2020`, or
  Dolby Vision side data is present; `video.dolby_vision` is `null` or a dict with
  `profile`/`level`/`bl_compatibility_id`. This skill's `HDR_TO_SDR` / `STRIP_DOVI` validation reads exactly these
  fields (`adapter.probe_hdr`), never re-derives HDR-ness from raw tags itself.
- **`--audio-stream N` (default 0, ffmpeg-skill >= 0.12.0)** selects which audio stream of a multi-track input to
  keep, 0-based in file order; `color.py` validates it against `probe()`'s own `audio_streams` count and `die()`s on
  an out-of-range value. It only affects paths that re-encode audio: `--to-sdr`/`--lut`/`--correct` always, and
  `--retag`'s automatic re-encode fallback (`-map 0:a:{audio_stream}?`) — a successful `--retag` stream copy and
  `--strip-dovi` both keep every audio track untouched (`-map 0`), so the flag has nothing to select for either,
  and this skill does not accept an `audio_stream` parameter on `STRIP_DOVI`. This skill's `HDR_TO_SDR`/
  `LUT_APPLY`/`RETAG`/`PRIMARY_CORRECTION` parameters all carry `audio_stream`, forwarded unchanged to
  `--audio-stream`; ffmpeg-skill's own range check against the real stream count is the authority (this skill only
  requires it to be a non-negative integer at the request boundary, since the real bound depends on the source).
- **`reencoded` / `dropped_non_av_streams` (ffmpeg-skill >= 0.12.0/0.12.1).** `--retag`'s JSON output reports
  `reencoded` (`true` when the stream-copy path failed and a re-encode fallback ran) and `dropped_non_av_streams`
  (`true` only when even the subtitle/data-preserving re-encode failed and streams beyond video+audio had to be
  dropped) since 0.12.0; `--to-sdr`/`--lut`/`--correct` gained `dropped_non_av_streams` the same way in 0.12.1 (they
  always re-encode, so `reencoded` would be redundant there and is not reported). This skill carries both fields
  (when present) onto `NodeState`, into every intermediate's manifest, and into the output provenance chain, the
  same way `measurements` already is (`executor._execute_node`). `executor._validate_artifact` also independently
  re-probes the output's `subtitle_streams`/`data_streams` counts against the source's whenever the source has any,
  and raises `VALIDATION_ERROR` (`reason: stream_loss_unreported`) if a count fell without ffmpeg-skill having
  reported `dropped_non_av_streams: true` for that call — the honest-drop case is surfaced, never hidden, but is not
  itself treated as this skill's own failure; only an *unreported* loss is.
- **`--lut` and a Windows drive-letter path.** `color.py` builds `lut3d=file=<escape_filter_path(args.lut)>:interp=
  tetrahedral` for the `-vf` filter graph. `escape_filter_path` backslash-escapes a colon (`C:\...` → `C\:/...`) so
  the graph-level parser does not treat it as a filter separator. **Measured** on a Windows CI runner (gyan.dev
  ffmpeg 9.0.1 essentials, Windows Server 2025): that escaping is rejected by this build's filter-*option* value
  parser (`No option name near '...invert.cube:interp=tetrahedral'`, `Error parsing filterchain ...`), so an
  absolute Windows LUT path made `--lut` fail outright — not a corrupted result, a hard failure ffmpeg-skill itself
  reports (`kind: ffmpeg`), which this skill's adapter turns into `TOOL_ERROR`. Since only the drive letter's colon
  triggers the bug, this skill sidesteps it entirely from its own side, without touching ffmpeg-skill:
  `executor._execute_node` runs the `ffmpeg-skill/color` subprocess for `LUT_APPLY` with `cwd` set to the LUT
  file's own directory (`adapter.run_tool`/`_popen` take an optional `cwd` override for this, defaulting to the
  ffmpeg-skill checkout as before for every other call) and `executor._lut_arg` passes just the LUT's bare file
  name as `--lut` — no drive letter, no colon, nothing for `escape_filter_path` to (mis)escape, regardless of what
  drive the LUT, the caller's workspace and the ffmpeg-skill checkout each happen to be on. (A relative-path
  attempt was tried first and was not enough: GitHub Actions Windows runners put the repository checkout and the
  OS temp directory — where a caller's own workspace typically lives — on different drives, and no relative path
  can cross a Windows drive at all.)

## Compatibility gaps (required capability → not implemented here)

| wanted operation | missing in ffmpeg-skill's public contract | consequence |
|---|---|---|
| Exposure / contrast / saturation / white balance (temperature + tint) | — implemented since ffmpeg-skill 0.9.2's `--correct` | `PRIMARY_CORRECTION` (this doc, above); no longer in `UNSUPPORTED_OPERATIONS` |
| Gamma / three-way lift / gain / levels / curves | — implemented since ffmpeg-skill 0.12.3's `--correct` flags | `PRIMARY_CORRECTION`'s `gamma`/`lift`/`gain`/`levels_*`/`curves` parameters (this doc, above; docs/decisions.md ADR-18); no longer in `UNSUPPORTED_OPERATIONS` |
| White balance as its own single operation type | no single "white balance" flag; only the two separate `temperature`/`tint` parameters that together achieve it | declared (`WHITE_BALANCE`); use `PRIMARY_CORRECTION`'s `temperature`/`tint` instead |
| Container / format conversion alongside grading | `color.py` writes to whatever extension `-o` names, no explicit "convert to X"; that is `export.py`'s job | outputs must keep the source's container (ADR-8); use ffmpeg-skill/export separately |
| Multiple LUTs / multiple colour ops in one ffmpeg process | `color.py`'s mode flags are mutually exclusive | each operation is its own `ffmpeg-skill/color` call and its own intermediate; a chain of N operations costs N re-encodes (documented in README's Current limitations) |
| Filter/encoder capability detection on FFmpeg ≥ 8.0 | ffmpeg-skill 0.9's `_ff_list` matches the pre-8.0 three-flag format; FFmpeg 8 prints two flags, so every filter is reported "missing" | when the doctor reports ffmpeg but zero filters, `filter:*` capabilities are `unknown`, not `unsupported`; execution proceeds and output validation decides (docs/decisions.md ADR-10) |

Each gap is a request to ffmpeg-skill (or a future second backend), not something this skill works around with its
own ffmpeg invocation or a raw filter string.