Incorporate the structured taxonomy from Lin et al. "Building a Precise Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2). The paper proves prompts structured around five aspects (Subject / Subject Motion / Scene / Spatial Framing / Camera) unlock controllable cinematography in fine-tuned video generation models. Off-the-shelf VLMs already nail subject and scene; the gains live in motion, spatial, and camera, which prompts routinely omit. Universal layer (skills/creative/video-gen-prompting.md, +125 lines): - 5-aspect prompt skeleton replaces flat formula - Camera movements regrouped (translation / rotation / lens-only) with dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations - New primitive tables: camera height, camera angle, POV, lens distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking), playback speed (6 modes), subject transitions - Order-matters and self-contained-prompt rules - Identity anchoring rule for multi-shot - Strict static-shot rule, anti-subjective callout, overlays-not-depth - Per-model word-count guidance Per-model guides (sora, veo, hunyuan, ltx, seedance): - Add the primitives each model honors literally - Word-count sweet spots per model - Strengthen seedance verbatim-identity and subject-transition guidance Pipeline directors (cinematic / explainer / animation scene-director, cinematic / explainer asset-director): - 5-aspect scene-plan checklist (per-pipeline adapted) - Overlays-not-depth callout - Pre / critique / post self-review loop for generation prompts Reviewer (skills/meta/reviewer.md): - CHAI critique-quality rules: accurate / complete / constructive - Critical findings now require a proposed_fix Storytelling, cinematic, broll, video-reference-analyst: - Anti-subjective rule (replace mood adjectives with visual causes) - Camera-intent-per-beat for script writers - POV column in stock-footage query templates - 5-aspect structured output mandatory for reference-video analysis skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
6.0 KiB
Scene Director - Animation Pipeline
When To Use
You are converting the script into a feasible animation plan. This is the stage that decides whether the project feels designed or chaotic.
Prerequisites
| Layer | Resource | Purpose |
|---|---|---|
| Schema | schemas/artifacts/scene_plan.schema.json |
Artifact validation |
| Prior artifacts | state.artifacts["script"]["script"], state.artifacts["proposal"]["proposal_packet"] |
Beat map and tool path |
| Playbook | Active style playbook | Palette, typography, motion consistency |
Process
1. Make An Animatic-Minded Plan
For each scene, define:
- what appears first,
- what changes,
- what is held,
- how the scene exits.
2. Limit Transition Families
Choose a small set of transition meanings:
- cut,
- fade,
- slide,
- transform.
3. Match Scene Type To Tool Path
Use:
diagramscenes for structured explanation,animationscenes for motion-first sequences,text_cardfor clean high-impact copy moments,generatedonly where needed.
For image_animation approach (anime/illustration style):
Use anime_scene type for each scene. Plan:
- Images per scene: 2-3 images built from the same visual system and nearby seeds for crossfade effect
- Camera motion: choose from
zoom-in,zoom-out,pan-left,pan-right,ken-burns,drift-up,drift-down,parallax,static— vary per scene to prevent monotony - Particle type: choose from
fireflies,petals,sparkles,mist,light-rays— match to scene mood - Lighting: optional
lightingFrom/lightingTogradient for atmospheric shifts within the scene - Vignette:
truefor cinematic framing (default),falsefor bright/open scenes - Scene duration: 4-7 seconds per scene. Longer scenes need more images for crossfade variety.
Scene variety rules for image_animation:
- Don't use the same camera motion for consecutive scenes
- Alternate between warm and cool particle types
- Mix close-up and wide establishing shots
- Use overlays (
hero_title,section_title) to add narrative structure
JSON prop name mapping (use these exact field names in the composition JSON):
| Concept | JSON Field | Example Values |
|---|---|---|
| Camera motion | animation |
"zoom-in", "pan-right", "ken-burns" |
| Particle effect | particles |
"fireflies", "sparkles", "mist" |
| Particle color | particleColor |
"#FFE082" |
| Particle density | particleCount |
20 (range: 1-50) |
| Particle brightness | particleIntensity |
0.5 (range: 0-1) |
| Lighting start | lightingFrom |
"rgba(255,200,100,0.15)" or "transparent" |
| Lighting end | lightingTo |
"rgba(255,107,157,0.08)" or "transparent" |
| Cinematic edge darken | vignette |
true / false |
| Scene background | backgroundColor |
theme-derived value such as "#0A0A1A" or "#F6F1E8" |
Reference: remotion-composer/public/demo-props/mori-no-seishin.json — 6 scenes using this pattern.
Reference: remotion-composer/public/demo-props/deep-ocean.json — 6 underwater scenes with different palette.
4. Use Metadata For Timing Rules
Recommended metadata keys:
animatic_rulestransition_ruleshold_rulestool_path_mapreusable_motifs
5. 5-Aspect Scene-Plan Checklist
Every scene must specify all five aspects, BUT the load shifts with the scene's
animation_mode. Manim and other diagrammatic/programmatic scenes care most about Subject and Spatial Framing — Camera and Subject Motion in the cinematographic sense often map to N/A or to abstract equivalents. AI-video /image_animation/anime_scenescenes care about all five and behave like cinematic shots. Marking an aspect as N/A is allowed but must be explicit per scene; silent omission is forbidden.
- Subject — type + key visual attributes; for Manim, the equation/object/graph being foregrounded; for
anime_scene, the character or environment in focus.- Subject Motion — for Manim, the order of
Create/Transform/FadeInand what each animation conveys; for AI-video, the actions and interactions in temporal order.- Scene — overlays (separately!) + POV + setting + time of day + scene dynamics. For Manim, "setting" is the canvas background + axis style; for
anime_scene, the environment + lighting gradient.- Spatial Framing — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE. Manim cares about layout grid + element positions; AI-video cares about full cinematographic framing.
- Camera — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement. For Manim and pure motion-graphics, default to N/A unless using a virtual camera move (
MoveCamera,self.frame). Foranime_sceneand AI-video, specify fully.Tie this back to
animation_modein scene metadata: a Manim scene that lists Camera fully is over-specified; an AI-video scene that omits Camera is under-specified. Seeskills/creative/video-gen-prompting.mdfor the primitive vocabulary.
Overlays callout. Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds,
hero_title,section_title,provider_chip) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (overlays: [...]) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output.
6. Quality Gate
- every scene has a clear timing intent,
- the 5-aspect checklist is satisfied for the scene's
animation_mode(with explicit N/A where appropriate), - overlays live under
overlays:, never inside the framing description, - the transition system is limited and meaningful,
- the tool path is explicit,
- the sequence feels like one designed system.
Common Pitfalls
- Adding a new transition idea in every scene.
- Planning scenes that have no realistic production path.
- Overanimating text-heavy scenes.