Lipsync

---
name: lipsync
description: |
  Synchronize mouths in an existing video to supplied final speech audio, only
  when the user explicitly wants mouth movements changed to match that audio.
  Replacing dialogue alone does not imply lip sync; use voice for new dialogue
  with unchanged picture. Does not translate, clone voices, generate speech, or
  create a new avatar performance. Integrated lip-synced translation uses
  video-translation; independently generated audio uses this skill only when
  mouth synchronization is requested.
---

# Lipsync

## Scope and input roles

`submit_lipsync` takes an existing project video and the final speech audio whose
words should be heard in the output. It changes mouth motion to match that audio.
It does not translate words, create a voice or derive a new performance from a photo.
For a new presenter performance from a likeness, use `digital-human`.

For any translation request, first load `video-translation`. It owns the route
choice and may call this skill with independently prepared final audio. Do not
restart translation or voice selection here.

If the user supplies final audio, reuse it directly. For a wholly new script,
use `voice` to generate final speech, then obtain approval of that actual audio and
its measured duration before paid rendering, as described below. A cloning sample
is not final dialogue. If no mouth synchronization is requested, use dialogue
replacement instead of this tool. Integrated HeyGen translation includes its own mouth
synchronization; do not lip-sync its output a second time by default.

When a generation tool returns `FEATURE_NOT_INCLUDED` for a `generation.*` feature, use its payer and access details. If the project owner needs Pro, explain that the owner pays and must upgrade; the member's own Pro does not unlock the project. Do not give the member a personal upgrade link or send them to their own billing menu. They can find the owner in the project's existing members view when available. Otherwise share the returned upgrade link, or ChatCut’s avatar menu → Upgrade. Existing credits are preserved and generation still consumes credits after upgrading. This rejection does not charge for generation; existing agent conversation fees are separate, so do not claim the whole conversation was free. Preserve the exact original tool input and project, then stop. Do not automatically search for or substitute stock media; offer alternatives only if the user explicitly asks.

When the user returns with “continue”, “继续”, “jixu”, or says they upgraded, recheck current payer eligibility and credits for the original input with `estimate_generation` when supported, otherwise the original tool’s precheck. If there is no separate precheck, one retry of the original tool lets its server check current access. If eligible, submit the original input once; if still blocked, explain the current blocker and stop. Do not ask the user to choose between the original generation and stock media again or state the old rejection as current status without checking. If a job was already accepted, track it instead of submitting again. Never switch generation tools, providers or services to bypass a gate.

## Prepare, submit and inspect

1. Resolve the exact ready source-video and speech-audio assets. Prepare the requested
   section as its own asset when needed, using
   [media preparation](../video-translation/SKILL.md#prepare-the-processing-source). This tool processes that entire asset,
   without a speaker or spatial selector; cutting a time range does not isolate
   one visible person. Resolve any non-target preservation requirement against
   that actual scope. Inspect the actual video and final audio using available
   perception to check the approved words and desired voice.
2. Check source/audio durations, visible speaker, face obstruction and scene cuts.
   Do not promise arbitrary multi-person targeting or that a long new speech can fit
   a short original performance. Resolve material duration differences before paying
   for rendering. Keeping duration and keeping natural speech pace can conflict.
   A missing cloud copy is not a missing local source. Do not stop because
   `sourceAccess.cloud` is `not-available` or the original upload is `deferred`.
   Submit the resolved assets when ready; the tool requests any required upload.
   If it reports an upload in progress, follow its wait/retry instructions with
   the same video, audio and request ID. Ask for relinking only when the tool
   reports that the source file cannot be read.
3. In Grok Bot, first present the actual audio through the host's supported playback,
   measured durations, proposed treatment and known cost, and obtain approval
   before calling `submit_lipsync`. Do not assume a ChatCut Desktop confirmation
   dialog will protect this hosted call. If playback is unavailable, ask the user
   to listen in the ChatCut editor and approve before submission. Reuse
   approval for unchanged audio without asking again about identity or script.
   Supply `sourceVideoAssetId`, `speechAudioAssetId` and required `durationPolicy`.
   `preserve-video` requires audio/video durations within 250 ms; `allow-adjustment`
   currently admits a longer/shorter duration ratio up to 1.15 as ChatCut's
   conservative input limit, not a verified supplier formula. Adapt media outside
   that range before submission. It does not guarantee exact audio-length matching,
   unlimited stretching or a natural extended performance.
   Never silently truncate approved speech. Do not supply transcript,
   voice-clone samples, avatar IDs or invented speaker-selection parameters.
4. Follow the returned job via `track_progress`, retaining job/asset IDs to resume
   existing work. Continue requested timeline assembly and export when the asset is
   ready. Report generation/placement from their actual receipts and duration from
   metadata. Check wording from actual audio or its transcript, voice similarity
   from listening, and mouth alignment from continuous audiovisual inspection.
   Asset metadata and static frames cannot establish natural lip sync or voice similarity. If that perception was not
   available or performed, state that quality remains unverified rather than treating
   successful generation or timeline placement as a quality check.

Preserving the original scene is the goal, not a promise of identical pixels,
container, alpha or frame rate. Follow actual returned limitations. Do not promise
background music preservation or use a voice-isolation tool to extract a background bed.

## Recover the affected stage

- Wrong words, pronunciation or voice: fix the final audio first and only then redo
  mouth rendering. Lipsync cannot repair the audio.
- Audio correct but lips poor: inspect face visibility, occlusion, multiple speakers,
  cuts and duration. Adjust the affected input within available controls.
- Unknown submit outcome: inspect the existing job; retry with identical inputs and
  the same optional `requestId`. By default identical inputs reuse the existing job,
  including a completed output. Only when the user explicitly wants a fresh render,
  create one new UUID as `requestId`, retain it, and reuse it across all retries of
  that attempt. Never create new IDs simply to bypass failures or uncertain outcomes.
  A changed final audio/video asset or duration policy is a different input. A
  request-conflict error requires restoring the original inputs, not blind retries.
- Source not ready or invalid: wait for processing or prepare valid media; do not
  substitute a photo-avatar workflow that discards the user's footage.
- Render succeeded but import failed: recover the existing output when possible;
  do not regenerate or loop forever.
- Missing Lipsync pricing configuration: report unavailable pricing and stop; do not
  invent a cost or repeatedly submit.
- Capability/permission unavailable: report the observed blocker and preserve valid
  input audio. Offer alternatives only with their changed picture/cost implications.