mirror of https://github.com/hanzoai/video.git synced 2026-11-02 12:13:09 +00:00

Files

T

calesthio f45ae2f933 docs(prompting): adopt 5-aspect video specification across skills

Incorporate the structured taxonomy from Lin et al. "Building a Precise
Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2).
The paper proves prompts structured around five aspects (Subject /
Subject Motion / Scene / Spatial Framing / Camera) unlock controllable
cinematography in fine-tuned video generation models. Off-the-shelf VLMs
already nail subject and scene; the gains live in motion, spatial, and
camera, which prompts routinely omit.

Universal layer (skills/creative/video-gen-prompting.md, +125 lines):
- 5-aspect prompt skeleton replaces flat formula
- Camera movements regrouped (translation / rotation / lens-only) with
  dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations
- New primitive tables: camera height, camera angle, POV, lens
  distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking),
  playback speed (6 modes), subject transitions
- Order-matters and self-contained-prompt rules
- Identity anchoring rule for multi-shot
- Strict static-shot rule, anti-subjective callout, overlays-not-depth
- Per-model word-count guidance

Per-model guides (sora, veo, hunyuan, ltx, seedance):
- Add the primitives each model honors literally
- Word-count sweet spots per model
- Strengthen seedance verbatim-identity and subject-transition guidance

Pipeline directors (cinematic / explainer / animation scene-director,
cinematic / explainer asset-director):
- 5-aspect scene-plan checklist (per-pipeline adapted)
- Overlays-not-depth callout
- Pre / critique / post self-review loop for generation prompts

Reviewer (skills/meta/reviewer.md):
- CHAI critique-quality rules: accurate / complete / constructive
- Critical findings now require a proposed_fix

Storytelling, cinematic, broll, video-reference-analyst:
- Anti-subjective rule (replace mood adjectives with visual causes)
- Camera-intent-per-beat for script writers
- POV column in stock-footage query templates
- 5-aspect structured output mandatory for reference-video analysis

skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.

2026-04-28 08:11:31 -07:00

4.3 KiB

Raw Permalink Blame History

VEO 3.1 / VEO 3 — Prompting Guide

Source: Vertex AI Video Gen Prompt Guide For universal vocabulary, see: skills/creative/video-gen-prompting.md

Word count: VEO 3.1 sweet spot is 100–250 words; longer prompts stop helping.

VEO-Specific 14-Component Structure

VEO responds to the most comprehensive prompt structure of any model:

Subject — who/what the action revolves around
Action — movements, interactions, expressions
Scene / Context — location, time, weather, period
Camera Angles — shot type and perspective
Camera Movements — dynamic motion
Lens / Optical Effects — how the camera "sees"
Lighting — source, direction, quality
Tone / Mood — emotional register
Artistic Style — photorealistic, cinematic, animation, art movement
Ambiance — color palettes, atmospheric effects, textures
Temporal Elements — pacing, time flow, rhythm
Audio — sound effects, ambient, dialogue (VEO 3 generates dialogue)
Cinematic Terms — editing techniques (match cut, montage, split diopter)
Negative Prompt — what to exclude

VEO-Specific Strengths

Dialogue generation: VEO 3 natively generates character speech. Write dialogue naturally.
Audio integration: Ambient sound, music, and voice are generated together with video.
Negative prompts: Explicitly supported — "no text overlays, no watermarks, no lens flare"
Editing vocabulary: Understands "match cut", "jump cut", "montage", "split diopter" as prompt terms.

Camera vocabulary VEO honors literally

VEO 3.1 distinguishes the three camera-motion families and treats their tokens as separate primitives. Mixing them up (e.g. asking for a "zoom" when you mean a "dolly") will produce the wrong move.

Translation (rig physically moves): dolly (in/out along the lens axis), truck (left/right laterally), pedestal (up/down vertically)
Rotation (rig stays put, camera rotates): pan (yaw, left/right), tilt (pitch, up/down), roll (Dutch / Z-axis)
Lens-only (rig and body don't move): zoom (focal length change), rack focus / pull focus / focus tracking (focal-plane change)

dolly ≠ zoom; pan ≠ truck. VEO follows whichever token leads.

VEO Lens Effects (Unique)

VEO specifically responds to optical effects most models ignore:

Effect	Prompt Language
Rack focus	"rack focus from foreground flower to background figure" (snap shift)
Pull focus	"slow pull focus from the candle in the foreground to the doorway behind" (gradual, slower than rack)
Focus tracking	"focus tracks the runner as she crosses frame; background stays soft" (focus follows a moving subject)
Dolly zoom (vertigo)	"vertigo effect as character realizes the truth"
Fisheye	"fisheye lens distortion, skatepark POV"
Anamorphic lens flare	"anamorphic lens flare streaking horizontally from setting sun"

These three focus modes (rack, pull, tracking) are different — VEO 3.1 honors the distinction per the paper.

VEO Art Movement References

VEO responds well to specific art movements as style anchors:

"Van Gogh-inspired swirling sky"
"Surrealist Dalí-esque melting landscape"
"Art Deco geometric patterns in the architecture"
"Bauhaus clean lines and primary colors"
"Gritty graphic novel illustration style"
"Chinese ink wash painting animation"

Subtitle Prevention

VEO may add subtitles by default for dialogue. To prevent:

Add to negative prompt: "no subtitles, no captions, no text overlays"

Example

Subject: A lone astronaut in a weathered white spacesuit
Action: Slowly turns to face the camera, visor reflecting a dying star
Scene: Surface of a barren moon, cracked grey terrain, massive ringed
       planet filling the horizon
Camera: Low-angle medium shot, slow arc around subject
Lens: Wide-angle, deep focus keeping both astronaut and planet sharp
Lighting: Harsh rim light from the star behind, cool blue fill from
          planet reflection, no atmosphere diffusion
Mood: Awe, isolation, quiet grandeur
Style: Photorealistic sci-fi cinematography, IMAX-scale
Audio: Breathing inside helmet, faint radio static, low rumble
Negative: No text, no HUD overlay, no lens flare

4.3 KiB Raw Permalink Blame History Unescape Escape