mirror of https://github.com/hanzoai/frames.git synced 2026-09-24 00:08:03 +00:00

Files

T

zooqueen ae80da5998 refactor(media-use): resolve every media type through api.hanzo.ai

Voice is POST /v1/audio/speech and word timings come from
POST /v1/audio/transcriptions over the rendered audio. Music, sound
effects, images, and video generate through /v1/audio/music,
/v1/audio/foley, /v1/images/generations, and /v1/videos/generations. The
shared catalog is a listing under media/<type>/ in our own object store,
browsed through /v1/s3 — an empty prefix reports itself as empty rather
than reading as a bad query.

One client (scripts/lib/api.mjs) holds every route and the one credential:
a Hanzo IAM token in $HANZO_API_KEY, read from Hanzo KMS. No credential
file, no per-repo .env, no per-provider CLI to install or keep current.
$HANZO_BASE_URL points the same code at a local Hanzo Engine.

Word timings are passed through, never interpolated: a caption cut on
invented boundaries reads as correct and drifts against the voice. When
the service returns text without them, the word array is empty and the
caller is told.

Deletes the provider cascade this replaces, its model ladders, and the
account linking in telemetry — the install id stands alone.

2026-08-13 17:18:37 -07:00

1.6 KiB

Raw Permalink Blame History

Audio engine — voiceover, music, SFX, captions, transcription

For a full audio pass (TTS voiceover + background music + sound effects in one shot), use the shared engine at audio/scripts/audio.mjs. It takes a neutral audio_request.json and writes audio_meta.json plus assets under .media/audio/{voice,bgm,sfx}:

node <SKILL_DIR>/audio/scripts/audio.mjs --request ./audio_request.json --out ./audio_meta.json

Request { lang?, speed?, voice?, lines: [{ id, text, sfx?: [names] }], bgm: { mode?, query?, prompt? } }: id joins each line back to your model; bgm.mode = catalog | compose | none (omit for auto). --only tts,bgm,sfx runs a subset and merges into an existing --out.
Output audio_meta.json (id-keyed): voices[].{path,duration_s,words[]} (word timestamps for captions), sfx[], bgm, total_duration_s.
One credential: $HANZO_API_KEY (and $HANZO_ORG for the shared catalog) unlocks voice, music, foley, and catalog reads. Run node <SKILL_DIR>/scripts/resolve.mjs --doctor before assuming any of them will work.
Every route is synchronous: when the engine returns, every file it reports is on disk.

Single-shot helper: audio/scripts/speak.mjs (one line, one file, optional word timings). Per-topic guides live in audio/references/ (tts.md, bgm.md, sfx.md, transcribe.md, remove-background.md, captions/).

Word timings come from scripts/transcribe.mjs -> POST /v1/audio/transcriptions. When the service returns text without per-word timings, the word array is empty and the engine records an anomaly; captions then fall back to line timing. Nothing is interpolated.

1.6 KiB Raw Permalink Blame History

Audio engine — voiceover, music, SFX, captions, transcription

1.6 KiB

Raw Permalink Blame History