Video Upscaling

---
name: video-upscaling
description: Upscaling craft: Real-ESRGAN model selection, when to upscale, denoise settings, face enhancement.
---

# Upscaling Usage for OpenMontage

> Sources: Real-ESRGAN documentation, GFPGAN face enhancement docs, Real-ESRGAN paper
> (Wang et al., 2021), practical upscaling benchmarks

## Quick Reference Card

```
DEFAULT MODEL:    RealESRGAN_x4plus — real-world photos and video frames
DEFAULT SCALE:    4x (480p→1080p, 720p→4K)
ANIME MODEL:     RealESRGAN_x4plus_anime_6B — flat color areas, illustrations
FACE ENHANCE:    Enable face_enhance for footage with people (uses GFPGAN)
DENOISE:         0.5 default, raise to 0.8 for very noisy inputs
```

## When to Upscale

| Situation | Upscale? | Notes |
|-----------|----------|-------|
| User-provided footage is 480p or 720p, target is 1080p/4K | Yes | Most common use case |
| Generated images need higher resolution for video frames | Yes | AI image output is often 512-1024px |
| Thumbnail or still frames need crisp detail | Yes | Single-frame upscale is fast |
| Old/archival footage restoration | Yes | Combine with higher denoise_strength |
| Source is already 1080p+ and target is 1080p | **No** | Wastes compute, can introduce artifacts |
| Source is already 4K | **No** | Over-sharpening degrades quality |

## Model Selection

| Model | Best For | Notes |
|-------|----------|-------|
| `RealESRGAN_x4plus` | Real-world photos, video frames | Default choice |
| `RealESRGAN_x4plus_anime_6B` | Anime, illustrations, motion graphics | Preserves flat color areas |
| `RealESRNet_x4plus` | Fastest option, slightly lower quality | When speed matters |

## Scale Factor Guidance

| Scale | Use Case | Example |
|-------|----------|---------|
| 4x | Standard upscale for low-res sources | 480p→1080p, 720p→4K |
| 2x | Moderate upscale when 4x is overkill | 720p→1080p |

- **4x** is the most common choice. Use it for 480p sources targeting 1080p, or 720p targeting 4K.
- **2x** is appropriate when the source is already 720p and the target is 1080p — avoids unnecessary processing and potential artifacts.
- **Never upscale beyond 4x in a single pass.** Quality degrades sharply, and hallucinated details become obvious.

## Face Enhancement

- Enable `face_enhance` when the video contains human faces
- Uses GFPGAN internally to enhance face regions while Real-ESRGAN handles the rest
- Particularly valuable for webcam footage and old video
- Do NOT enable for content without faces — adds processing time with no benefit

## Denoising Strength

| Source Quality | denoise_strength | Rationale |
|---------------|-----------------|-----------|
| Clean digital source | 0.5 (default) | Minimal denoising needed |
| Slight compression artifacts | 0.6 | Light cleanup without over-smoothing |
| Old/noisy footage | 0.7-0.8 | Aggressive denoising for archival content |
| Very noisy / low-light footage | 0.8 | Maximum practical denoising |

Do not exceed 0.8 — higher values destroy legitimate detail.

## Video Upscaling Notes

- Video upscaling extracts frames, upscales each, reassembles
- This is **SLOW** — budget 5-10x real-time on GPU
- For long videos, consider upscaling only key scenes/clips rather than the full video
- Audio is preserved from the original
- Output file size will be significantly larger (~16x for 4x upscale)

## Common Workflows

### Workflow 1 — User-Provided Low-Res Footage

```
1. Assess source resolution (e.g., 480p webcam recording)
2. Choose scale factor: 4x for 480p→1080p, 2x for 720p→1080p
3. Enable face_enhance if footage contains people
4. Set denoise_strength based on source quality
5. Upscale → inspect output → proceed to compose stage
```

### Workflow 2 — AI-Generated Image Frames

```
1. Generate images at native model resolution (512-1024px)
2. Upscale with RealESRGAN_x4plus to target video resolution
3. Keep denoise_strength at 0.5 — AI output is clean
4. Do NOT enable face_enhance unless faces are prominent
```

### Workflow 3 — Manim / Motion Graphics Frames

```
1. Render Manim at default resolution
2. Upscale with RealESRGAN_x4plus_anime_6B (preserves flat colors)
3. Keep denoise_strength at 0.5
4. Verify text and line art remain sharp
```

### Workflow 4 — Archival Footage Restoration

```
1. Assess noise level and resolution
2. Set denoise_strength to 0.7-0.8
3. Enable face_enhance for footage with people
4. Use RealESRGAN_x4plus at 4x
5. Carefully inspect output for hallucinated details
```

## Quality Checklist

- [ ] Upscaled output is sharp without visible artifacts
- [ ] Faces look natural (no over-smoothing or distortion)
- [ ] Text/UI elements in screen recordings remain readable
- [ ] No hallucinated details in flat color areas
- [ ] File size is reasonable (4x upscale = ~16x file size)

## Applying to OpenMontage

When using the `upscale` tool in the asset stage:

1. **Upscale BEFORE the compose stage** — it is an asset-prep step, not a post-processing step
2. **Use `face_enhance=true` for any talking-head footage** — GFPGAN dramatically improves face quality
3. **Use `RealESRGAN_x4plus_anime_6B` model for Manim outputs** or flat illustration frames — preserves clean edges and flat color areas
4. **For budget-conscious pipelines**, upscale only hero shots and thumbnails rather than every frame
5. **Set `denoise_strength` to 0.7-0.8 for old/noisy footage**, keep at 0.5 for clean digital sources
6. **Check upscaled output for artifacts** — over-sharpening, hallucinated texture, face distortion
7. **Prefer 2x over 4x when the source is already 720p and target is 1080p** — less compute, fewer artifacts