Video Prompt Engineering

---
name: video-prompt-engineering
id: SK-FTV-006
version: 1.0.0
description: Optimize prompts for AI video generation platforms including Sora, Runway, Pika, and Kling
specialization: film-tv-production
graph:
  domains: [domain:arts-culture]
  skillAreas: [skill-area:prompt-engineering, skill-area:video-processing, skill-area:media-encoding]
  roles: [role:creative-director, role:media-engineer]
---

# Video Prompt Engineering Skill

## Purpose

Create optimized prompts for AI video generation platforms that produce cinematic, production-quality footage. These prompts must communicate action, camera movement, timing, and style in a format that translates across different AI platforms.

## Universal Prompt Structure

```
[SCENE SETUP] + [CHARACTER/SUBJECT] + [ACTION SEQUENCE] +
[CAMERA MOVEMENT] + [LIGHTING/ATMOSPHERE] + [STYLE/AESTHETIC]
```

### Component Details

| Component | Content | Example |
|-----------|---------|---------|
| Scene Setup | Location, time, environment | "A rain-soaked Tokyo street at night" |
| Subject | Who/what appears | "A woman in a red coat" |
| Action | What happens (sequential) | "walks forward, stops, turns to look back" |
| Camera | Movement and framing | "slow tracking shot, eye level" |
| Lighting | Light sources, mood | "neon signs reflecting on wet pavement" |
| Style | Visual aesthetic | "cinematic, blade runner aesthetic" |

## Platform-Specific Optimization

### Sora (OpenAI)

**Strengths:**
- Complex scenes
- Multiple subjects
- Consistent physics
- Long duration

**Prompt Style:**
```
Natural language, paragraph format.
Describe the scene as if telling a story.
Include subtle details about atmosphere.
Mention specific camera movements by name.

Example:
"A close-up tracking shot follows a single snowflake as it falls
through the air, passing snow-covered pine branches and eventually
landing on a red mitten. The camera holds on the crystalline
structure of the snowflake for a beat before it begins to melt.
Soft, diffused winter light. Shallow depth of field with gentle
bokeh in the background."
```

### Runway Gen-3

**Strengths:**
- Motion control
- Style transfer
- Consistent aesthetics
- Subject coherence

**Prompt Style:**
```
Structured, detailed prompts.
Specify motion types explicitly.
Reference camera movements precisely.
Include duration indicators.

Example:
"Cinematic shot, slow motion. A detective in a trench coat walks
through a crowded train station. Camera dollies backward maintaining
medium shot. People blur past in the foreground. Harsh overhead
lighting creates deep shadows. 1940s noir aesthetic. 4 seconds."
```

### Pika

**Strengths:**
- Quick generation
- Image-to-video
- Style consistency
- Character animation

**Prompt Style:**
```
Concise, focused prompts.
One primary action per prompt.
Strong style keywords.
Clear motion direction.

Example:
"Close-up of woman's face, wind blowing through hair,
looking off camera left, soft golden hour lighting,
cinematic film grain, subtle movement"
```

### Kling

**Strengths:**
- Longer duration
- Complex action
- Multiple subjects
- Realistic motion

**Prompt Style:**
```
Detailed action sequences.
Step-by-step motion description.
Clear spatial relationships.
Timing indications.

Example:
"A chef in a professional kitchen. Wide shot. He tosses
vegetables in a wok, flames rise dramatically (2 sec),
plates the dish with precise movements (3 sec),
wipes his brow and smiles at camera (2 sec).
Warm kitchen lighting, steam rising, professional quality."
```

## Camera Movement Vocabulary

### Static Shots
```
locked off, tripod, stable, still camera
```

### Movement Types
```
- PAN: horizontal pivot (pan left, pan right)
- TILT: vertical pivot (tilt up, tilt down)
- DOLLY: camera moves (dolly in, dolly out, dolly alongside)
- TRACKING: follows subject (tracking shot, follow shot)
- CRANE: vertical lift (crane up, crane down)
- STEADICAM: smooth handheld (steadicam walk, floating camera)
- HANDHELD: naturalistic shake
- ZOOM: lens change (slow zoom, crash zoom)
- ORBIT: circles subject (360 orbit, arc shot)
```

### Speed Modifiers
```
slow, gentle, smooth, quick, whip, crash, gradual
```

## Action Description

### Effective Action Verbs
```
walks → strides, shuffles, marches, stumbles
runs → sprints, jogs, dashes, bolts
looks → glances, stares, gazes, peers
turns → spins, pivots, rotates, wheels around
picks up → grabs, snatches, lifts, retrieves
```

### Timing Language
```
slowly, gradually, suddenly, immediately,
after a beat, in one motion, over X seconds
```

## Scene Prompt Template

```markdown
## Scene [Number]: [Title]

### Setup
- **Location:** [Specific environment description]
- **Time:** [Time of day, lighting conditions]
- **Atmosphere:** [Weather, mood, ambience]

### Subject
- **Character(s):** [Who appears, wardrobe, positioning]
- **Key Props:** [Important objects in scene]

### Action Sequence
1. [First action with timing]
2. [Second action with timing]
3. [Third action with timing]

### Camera
- **Shot Type:** [Size and angle]
- **Movement:** [Specific movement description]
- **Speed:** [Movement speed]

### Technical
- **Duration:** [Total seconds]
- **Aspect Ratio:** [16:9, 2.39:1, etc.]
- **Style:** [Visual aesthetic reference]

### Platform Prompts

**Universal/Sora:**
[Paragraph-form natural language prompt]

**Runway:**
[Structured prompt with style keywords]

**Pika:**
[Concise action-focused prompt]

### Negative Prompt
[What to avoid: jittery motion, morphing, artifacts, etc.]
```

## Style Keywords

### Cinematic Quality
```
cinematic, film grain, anamorphic, 35mm film,
professional quality, movie scene, theatrical
```

### Lighting
```
golden hour, blue hour, harsh shadows, soft light,
rim lighting, volumetric, neon glow, practical lighting
```

### Motion
```
smooth motion, fluid movement, realistic physics,
natural motion, consistent speed, seamless
```

### Atmosphere
```
atmospheric, moody, dramatic, serene, tense,
energetic, contemplative, mysterious
```

## Common Issues & Solutions

| Issue | Solution |
|-------|----------|
| Subject morphing | Describe subject consistently throughout |
| Jittery motion | Add "smooth" and "fluid" keywords |
| Wrong timing | Specify durations explicitly |
| Inconsistent style | Use strong style anchors |
| Background issues | Describe environment in detail |
| Physics problems | Describe motion realistically |

## Quality Checklist

- [ ] Scene environment clearly described
- [ ] Subject/character specified in detail
- [ ] Action sequence is logical and timed
- [ ] Camera movement explicitly stated
- [ ] Duration specified
- [ ] Style/aesthetic defined
- [ ] Platform-optimized version created
- [ ] Negative prompts included where needed