calesthio
50891a75b7
Separates creative grammar (renderer_family) from technical engine (render_runtime) so HyperFrames can stand alongside Remotion as a first-class runtime instead of masquerading as a Remotion sub-case. Locks runtime choice at proposal stage and enforces it end-to-end: the schemas require it, video_compose routes by it, the reviewer fails closed on silent swaps, and a parametrized contract test walks every pipeline manifest to ensure each planning-stage skill explains the conversation to the user. Adds hyperframes_compose (scaffold/lint/ validate/render/doctor/add_block), a playbook -> CSS style bridge, and vendored HyperFrames Layer 3 skills from commit d291358, pinned via PROVENANCE.md for future re-sync. Final_review now records render_runtime_used and runtime_swap_detected so compose lies are catchable after the fact.
2.0 KiB
Step 5: Generate VO + Map Timing
Audition voices
Never use the first voice you find. Audition 2-3 voices with the first sentence of SCRIPT.md:
- Kokoro (try first — free, no API key) —
npx hyperframes tts SCRIPT.md --voice af_nova --output narration.wav. Runs locally on CPU. Requires Python 3.10+ (macOS system Python 3.9 won't work — if it fails with an onnxruntime error, move to the next option). - ElevenLabs (best voice quality, widest selection) —
mcp__elevenlabs__search_voicesto browse,mcp__elevenlabs__text_to_speechto generate. Does not return timestamps — transcribe separately after. - HeyGen TTS (returns word timestamps automatically — saves a transcribe step) —
mcp__claude_ai_HeyGen__text_to_speech. Use when you want timestamps without a separate transcription pass.
Pick the voice that sounds most natural and conversational. Listen for pacing — does it breathe between sentences? Does it sound like a person or a robot?
Generate full narration
Generate the full script as narration.wav (or .mp3) in the project directory.
Transcribe for word-level timestamps
npx hyperframes transcribe narration.wav
Produces transcript.json with [{ text, start, end }] for every word. These timestamps are the source of truth for all beat durations.
Map timestamps to beats
Go through STORYBOARD.md beat by beat. For each beat:
- Find the first word of that beat's VO cue in
transcript.json - Find the last word of that beat's VO cue
- Set
beat.start = firstWord.start,beat.end = lastWord.end - Add 0.3-0.5s padding at the end for visual breathing room
Update STORYBOARD.md with real durations. Replace estimated times (e.g., "0:00-0:05") with actual timestamps (e.g., "0.00-3.21s").
Beat boundaries land on word onsets — hard cuts to the VO.
Update index.html
Update each scene slot's data-start and data-duration to match the real beat timings from the transcript. Also update the total composition duration and audio element duration.