mirror of https://github.com/hanzoai/video.git synced 2026-11-03 04:21:39 +00:00

Files

T

calesthio f45ae2f933 docs(prompting): adopt 5-aspect video specification across skills

Incorporate the structured taxonomy from Lin et al. "Building a Precise
Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2).
The paper proves prompts structured around five aspects (Subject /
Subject Motion / Scene / Spatial Framing / Camera) unlock controllable
cinematography in fine-tuned video generation models. Off-the-shelf VLMs
already nail subject and scene; the gains live in motion, spatial, and
camera, which prompts routinely omit.

Universal layer (skills/creative/video-gen-prompting.md, +125 lines):
- 5-aspect prompt skeleton replaces flat formula
- Camera movements regrouped (translation / rotation / lens-only) with
  dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations
- New primitive tables: camera height, camera angle, POV, lens
  distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking),
  playback speed (6 modes), subject transitions
- Order-matters and self-contained-prompt rules
- Identity anchoring rule for multi-shot
- Strict static-shot rule, anti-subjective callout, overlays-not-depth
- Per-model word-count guidance

Per-model guides (sora, veo, hunyuan, ltx, seedance):
- Add the primitives each model honors literally
- Word-count sweet spots per model
- Strengthen seedance verbatim-identity and subject-transition guidance

Pipeline directors (cinematic / explainer / animation scene-director,
cinematic / explainer asset-director):
- 5-aspect scene-plan checklist (per-pipeline adapted)
- Overlays-not-depth callout
- Pre / critique / post self-review loop for generation prompts

Reviewer (skills/meta/reviewer.md):
- CHAI critique-quality rules: accurate / complete / constructive
- Critical findings now require a proposed_fix

Storytelling, cinematic, broll, video-reference-analyst:
- Anti-subjective rule (replace mood adjectives with visual causes)
- Camera-intent-per-beat for script writers
- POV column in stock-footage query templates
- 5-aspect structured output mandatory for reference-video analysis

skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.

2026-04-28 08:11:31 -07:00

3.3 KiB

Raw Permalink Blame History

Scene Director - Cinematic Pipeline

When To Use

You are deciding how each cinematic beat will look and transition. This is where mood becomes a visual plan.

Prerequisites

Layer	Resource	Purpose
Schema	`schemas/artifacts/scene_plan.schema.json`	Artifact validation
Prior artifacts	`state.artifacts["script"]["script"]`, `state.artifacts["proposal"]["proposal_packet"]`	Beat map and source truth
Tools	`frame_sampler`, `scene_detect`	Source inspection and reframing checks
Playbook	Active style playbook	Color and typography consistency

Process

1. Make Hero Frames Explicit

Every cinematic piece needs a few memorable frames. Define them directly:

opening image,
reveal image,
final image,
any title-card hero moments.

2. Keep Source-Led Scenes Primary

If source footage exists, let it carry the piece. Generated inserts or text cards should support transitions, emphasis, or missing coverage, not dominate the timeline.

3. Limit Transition Vocabulary

Choose a small set:

hard cut,
fade to black,
slow dissolve,
restrained push or punch-in.

Too many transition types kill the mood.

4. Use Metadata For Visual Rules

Recommended metadata keys:

hero_frames
transition_rules
aspect_ratio_rules
title_card_rules
support_insert_rules

5. 5-Aspect Scene-Plan Checklist

Every scene beat — and especially every hero frame — must specify all five aspects. Cinematic relies on a small number of memorable frames; vague hero-frame specs are the single most common failure mode and produce unpredictable model output. Marking an aspect as N/A is allowed but must be explicit (e.g., "no subject — establishing scenery shot"). Silent omission is forbidden.

Subject — type + key visual attributes; if multiple, how to disambiguate. For hero frames, identity must be anchored verbatim across shots.

Subject Motion — actions in temporal order; subject↔object / subject↔subject interactions.

Scene — overlays (separately!) + POV + setting + time of day + scene dynamics.

Spatial Framing — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE across the beat.

Camera — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement.

See skills/creative/video-gen-prompting.md for the primitive vocabulary.

Overlays callout. Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, name plates, end-tag cards) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (overlays: [...]) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output.

6. Quality Gate

every beat has a scene treatment,
hero frames are identifiable AND fully specified across all 5 aspects,
support inserts are justified,
overlays are recorded under overlays:, never inside the depth/framing description,
the visual language stays consistent across the piece.

Common Pitfalls

Using title cards as filler.
Treating generated inserts like the primary story without saying so.
Planning flashy transitions for every beat.

3.3 KiB Raw Permalink Blame History