mirror of
https://github.com/hanzoai/speech.git
synced 2026-10-04 07:37:19 +00:00
zeekayandHanzo Dev
6388fd9ab0
The live gate caught this on its first run: 40 tests passed and the build was still red, exit 134. A raw daemon thread is not joined at interpreter shutdown, so a decode still inside ctranslate2's C++ when the runtime tore down aborted — `terminate called without an active exception`. The failure is in the exit code, not in the report, which is why the fast suite and the summary line both looked fine. In a pod it is a container that cannot drain on SIGTERM. A pool fixes it because its workers ARE joined, and it is the better shape anyway: decoding is CPU-bound and the pod is capped at 4 cores, so bounding concurrent decodes stops many sessions from thrashing each other. Per-session single flight is unchanged. The test shuts a child down mid-decode and asserts it exits 0 — with the model loaded first and a pause after pushing, so shutdown lands inside the decode by construction rather than by luck. Co-authored-by: Hanzo Dev <[email protected]>
speech
CPU speech for the Hanzo platform: STT (faster-whisper, CTranslate2 int8) and TTS (kokoro-onnx, 82M) behind one OpenAI-shaped surface.
Routes
POST /v1/audio/transcriptions— multipartfile+model(+language,response_format) →{"text": …}POST /v1/audio/speech—{model, input, voice, response_format}→ audio bytes (mp3 default, wav native)GET /v1/models,GET /healthz
Trust model
No auth of its own, in-cluster only. api.hanzo.ai fronts it through the ai
plane, which authenticates and meters every caller — a provider row pointing at
http://speech.hanzo.svc/v1 is the whole integration. Weights bake into the
image at build; the container boots without the network.
Run
uv sync && uv run uvicorn main:app --port 8000
Phase 2 is /v1/realtime: websocket full-duplex (silero-vad segmentation,
partial transcripts, kokoro frames, barge-in) plus camera-frame relay into the
multimodal models.