Generation

---
name: generation
description: "Routes Stable Diffusion walk, music-video, still-image,
  audio-timing, and upsampling workflows for stable_diffusion_videos."
disable-model-invocation: true
metadata:
  disco-role: operating
license: Apache 2.0
---

# generation

Use this sub-skill when the task is to create images or videos with the
Diffusers pipelines exposed by `stable_diffusion_videos`.

## What this route owns

- `StableDiffusionWalkPipeline.walk(...)`
- `StableDiffusionWalkPipeline.make_clip_frames(...)`
- `generate_images(...)`
- `generate_images_flax(...)` as an optional backend reference
- `get_timesteps_arr(...)`
- `make_video_pyav(...)`
- `RealESRGANModel` loading and folder upsampling
- prompt / seed validation, frame layouts, and audio-driven interpolation

## What this route excludes

- Gradio UI launchers and tab wiring, which belong to `ui`
- repository maintenance and packaging tasks
- heavyweight training or checkpoint creation

## Read these bundled files

- `references/workflows.md` for end-to-end command patterns and output layout.
- `references/troubleshooting.md` for generation-specific failures and recovery.
- `references/flax-and-tpu.md` if you need the optional JAX/Flax/TPU path.
- `../../references/api-reference.md` for exact public signatures.
- `../../references/repo-provenance.md` for source snapshot and evidence paths.

## Common tasks

### Make a prompt walk

Start with `references/workflows.md` or run the template script with `--dry-run`
first. Then switch to `--run` when you are ready to download the model and use
CUDA.

### Make a music video

Use the audio offsets recipe, compute interpolation steps from the offsets and
FPS, and confirm the frame count before generating.

### Generate still images

Use `generate_images(...)` when you want prompt-conditioned image batches
instead of a walk.

### Preview audio timing

Use `scripts/preview_audio_timesteps.py` to inspect the interpolation weights for
a local audio file before running a long generation.

### Upsample frames or folders

Use `RealESRGANModel` only when you really need the 4x upsampling path. It is
optional, heavier, and can be slow on CPU.

## Backends and prerequisites

- The main walk workflow is GPU-oriented.
- `height` and `width` must be multiples of 8.
- `ffmpeg` with `libx264` must be available for MP4 encoding.
- `librosa` is needed for audio-timed interpolation.
- `realesrgan` is only needed for the upsampling path.

## Workflow boundaries

If you are unsure whether a request is a generation or UI task, choose this
sub-skill when the user names `walk`, `generate_images`, `audio_filepath`,
`get_timesteps_arr`, `make_video_pyav`, `RealESRGANModel`, or `upsample`.
Choose `ui` when the request is about the Gradio demo or launch parameters.