zooqueen
ae80da5998
Voice is POST /v1/audio/speech and word timings come from POST /v1/audio/transcriptions over the rendered audio. Music, sound effects, images, and video generate through /v1/audio/music, /v1/audio/foley, /v1/images/generations, and /v1/videos/generations. The shared catalog is a listing under media/<type>/ in our own object store, browsed through /v1/s3 — an empty prefix reports itself as empty rather than reading as a bad query. One client (scripts/lib/api.mjs) holds every route and the one credential: a Hanzo IAM token in $HANZO_API_KEY, read from Hanzo KMS. No credential file, no per-repo .env, no per-provider CLI to install or keep current. $HANZO_BASE_URL points the same code at a local Hanzo Engine. Word timings are passed through, never interpolated: a caption cut on invented boundaries reads as correct and drifts against the voice. When the service returns text without them, the word array is empty and the caller is told. Deletes the provider cascade this replaces, its model ladders, and the account linking in telemetry — the install id stands alone.
1.3 KiB
Voice to captions
When no recorded voiceover exists, render one and take its word timing from the same audio.
node skills/media-use/audio/scripts/speak.mjs \
script.txt --output narration.wav --words narration.words.json
speak.mjs renders the line through POST /v1/audio/speech, then reads word
timings back through POST /v1/audio/transcriptions over the audio it just
wrote. narration.words.json lands in the [{ id, text, start, end }] shape
the captions pipeline consumes.
Two calls, not one, because they answer two questions — a piece with no captions never pays for the alignment pass. Aligning against the rendered audio (rather than trusting a synthesis-time estimate) is also what keeps caption timing matched to delivery.
When there are no timings
If the transcription returns text without per-word timings, --words writes
nothing and says so. Timings are never interpolated from the audio duration: a
caption cut on invented boundaries reads as correct and drifts against the
voice. Captions then fall back to line-level timing, which the audio engine
already records as voices[].duration_s.
For a whole script — every line, plus music and effects, with timings folded
into audio_meta.json — use the audio engine rather than calling speak.mjs
per line. See ../../references/audio.md.