mirror of
https://github.com/hanzoai/speech.git
synced 2026-10-04 07:37:19 +00:00
chore: symlink AGENTS.md + GEMINI.md + CLAUDE.md -> LLM.md
Co-authored-by: Hanzo Dev <[email protected]>
This commit is contained in:
Hanzo Dev
@@ -1,200 +0,0 @@
|
||||
# speech — CPU STT + TTS, one OpenAI-shaped surface
|
||||
|
||||
Five files are the service: `main.py` (routes), `stt.py` (faster-whisper),
|
||||
`tts.py` (kokoro + ffmpeg for mp3), `transcript.py` (the growing transcript),
|
||||
`vad.py` (the squelch in front of it).
|
||||
Nothing here authenticates: the ai plane (hanzoai/ai, behind api.hanzo.ai)
|
||||
fronts this service and meters callers — including the anon grant tier (~3 uses
|
||||
per anon) which lives THERE, not here.
|
||||
|
||||
## The streaming transcript (`/v1/audio/transcript`)
|
||||
|
||||
Batch stays at `/v1/audio/transcriptions`. The streaming sibling is a resource
|
||||
that grows: `POST` to open, `POST` chunks to it, `DELETE` to close and flush.
|
||||
|
||||
- **Half duplex over plain HTTP, and this is forced, not preferred.** `ai` is a
|
||||
child process of the cloud pod reached over ZAP, and `zap-proto/http` builds a
|
||||
request as ONE frame — there is `FrameResponseHead` for streaming responses but
|
||||
no request-head frame, and `zip` contains no `Hijack`. So a WebSocket and a
|
||||
streamed request body BOTH cannot reach this service, no matter what the edge
|
||||
supports. `hanzoai/ingress` does pass upgrades, so testing `wss://` against
|
||||
ingress alone answers the wrong question — the wall is inside the pod.
|
||||
- Audio is raw little-endian int16, mono, 16 kHz, in the body. No base64 (that
|
||||
is a WebSocket text-frame tax we do not pay), no container to demux.
|
||||
- **Metering: `duration` on each ack is the seconds THAT push put in front of a
|
||||
decoder**, named as the batch route names it, so `ai` reads it into the same
|
||||
`AudioDurationSeconds` it already meters (`stt/openai.go:133`) with no new
|
||||
billing code. It is arithmetic on the bytes that got past the squelch, so it
|
||||
does not depend on the decoder returning anything, and summing the pushes
|
||||
yields the audio decoded — never more than the audio submitted. A push the
|
||||
squelch drops reports 0.0; the push that OPENS it reports more than its own
|
||||
length, because it carries the lead-in that earlier pushes withheld.
|
||||
- **Silence is not decoded and not billed** (`vad.py`). Silero already ships
|
||||
inside faster-whisper and already runs inside every decode, so running it in
|
||||
FRONT of the decoder costs no new weight and no new bake step — 0.67 ms per
|
||||
250 ms push, measured on the pod, against a decode of seconds. It is a squelch,
|
||||
not a per-window classifier: two thresholds to open and close, `HANG` of quiet
|
||||
before it closes, and `LEAD` of audio held back and handed over by the push
|
||||
that opens. Measured on the pod: an idle room bills 0.0 up to about -45 dBFS
|
||||
RMS of room tone (it starts to open on tone above that); a keyboard click and
|
||||
full-scale white noise do not open it; a meeting bills its speech plus about
|
||||
0.9 s per utterance and nothing for the lulls, so the share saved is 0% when
|
||||
someone is talking continuously and 86% at 30 s between turns.
|
||||
**The lead-in is the part that is easy to get wrong and expensive to lose.**
|
||||
Sliding a word across a push shows why: with `LEAD = 0`, 8 of 32 alignments
|
||||
clip up to 71.8 ms off the front of the word, because a push is admitted whole
|
||||
and a word beginning near the end of one has nothing in hand. At 0.5 s, none
|
||||
clip and the worst case still keeps 428 ms of margin. Hold the lead by slicing
|
||||
from the FRONT — `[-int(LEAD * SECOND):]` reads as the last N bytes and is the
|
||||
whole buffer when N is 0, which keeps every second of silence and hands the lot
|
||||
to the decoder on the next word.
|
||||
- Whisper decodes a span, not a sample. The window is re-decoded as it fills;
|
||||
segments ending a guard-length before the audio are committed and their bytes
|
||||
freed, which is what bounds the window and stops committed text being revised.
|
||||
|
||||
**Sessions are pod-local, and `speech.yaml` sets `replicas: 2`.** Pushes reach
|
||||
`speech.hanzo.svc`, which balances, so a session opened on one pod is not found
|
||||
on the other. Whoever routes must pin a session to the pod that owns it; that
|
||||
belongs in `ai` (which already does provider routing), not here — this service
|
||||
decodes, it does not route. Until that exists the endpoint is not usable through
|
||||
the plane, and `ai` has no route for it yet either.
|
||||
|
||||
## What it sustains, and where that number comes from
|
||||
|
||||
`bench.py` is the instrument: `say` once for a corpus of real speech carrying a
|
||||
word-to-time map, then `cost` for what a decode pass costs and `talk` to drive N
|
||||
sessions in real time. Everything below was measured with it **on a four vCPU
|
||||
DigitalOcean worker** — the shape a speech pod runs in. That qualifier is the
|
||||
whole point: the same code and the same run on four threads of a desktop Zen 5
|
||||
retires 13.7 audio-seconds per wall-second where the pod retires 5.7. A capacity
|
||||
number measured on the box is 2.4x the truth.
|
||||
|
||||
**Ack latency cannot see any of this and none of these numbers is one.** A push
|
||||
returns whatever text the session has, so it costs a memcpy whether the decoder
|
||||
is idle or minutes behind — 3.8 ms while the backlog grew fourfold. The metric is
|
||||
backlog, as lag: wall seconds elapsed minus seconds of audio already transcribed.
|
||||
The verdict is its SLOPE over the second half of a run, in seconds of lag added
|
||||
per minute of meeting. Runs must be MINUTES long: a stalling session looks fine
|
||||
for its first four minutes, which is how seconds-long runs came to predict eight
|
||||
concurrent meetings and were wrong by an order of magnitude.
|
||||
|
||||
**A pass costs what it costs, almost regardless of what it covers.** Whisper
|
||||
always encodes a thirty second mel window, so on the pod one pass takes 2.7 s over
|
||||
a one second window and 3.2 s over an eight second one — about 8.4 CPU-seconds
|
||||
either way. Capacity is a budget of PASSES, not of audio, and the way to buy it is
|
||||
to pass less often, never to pass over less.
|
||||
|
||||
**Past the mel window that cost multiplies.** 24 s -> 5.0 s per pass, 30 s -> 7.5,
|
||||
45 s -> 23.5. It is a cliff, not a slope.
|
||||
|
||||
**So the pod has a throughput PEAK, and the peak is the instability.** In steady
|
||||
state a session gets one pass per interval and that pass covers the audio that
|
||||
arrived during it, so audio retired per wall second IS the number of real-time
|
||||
streams served, and lag is roughly the window. Measured on the pod:
|
||||
|
||||
window 8 s 12 s 16 s 24 s 30 s
|
||||
streams served 3.1 4.4 5.0 3.9 2.8
|
||||
|
||||
Below the peak the loop is self-correcting: a longer window is cheaper per second,
|
||||
so lag comes back. Past it, a longer window is DEARER per second, so lag grows,
|
||||
which lengthens the window — runaway. That is why the failure is a cliff and not a
|
||||
gentle degradation, and why the working point has to sit on the rising branch
|
||||
rather than at the top of the curve.
|
||||
|
||||
**Measured, in eight minute runs at 250 ms chunks**, and the two agree to within a
|
||||
few percent — a session settles where its window buys exactly its own concurrency:
|
||||
|
||||
sessions lag slope verdict
|
||||
3 7.7 s -2.36 s/min settles
|
||||
4 12.7 s -0.45 s/min settles
|
||||
|
||||
So: **four concurrent streams of continuous speech per four vCPU pod**, with the
|
||||
transcript about thirteen seconds behind, or three at eight seconds. Two replicas
|
||||
is eight. The curve says five sits ON the peak and six has no fixed point at all.
|
||||
|
||||
The failure past that edge is not a gentle one. The same runs against the earlier
|
||||
two-worker build, which peaked lower, settled at one and two sessions and at FOUR
|
||||
ran away to 326 s of lag at +44 s/min — and closing those sessions then blocked for
|
||||
400 s each, because `DELETE` decodes everything that piled up.
|
||||
|
||||
**A session's ceiling is `LIMIT` = 600 s, so a 45 minute meeting is five sessions,
|
||||
not one.** Closing decodes the whole remaining window and blocks, and the transport
|
||||
admits one outstanding chunk, so the roll costs the microphone that whole pause.
|
||||
At three streams that is a couple of seconds every ten minutes; on a session that
|
||||
has fallen behind it is minutes.
|
||||
|
||||
### What more of it costs
|
||||
|
||||
Per pod, three streams. Per stream of continuous speech that is 1.33 vCPU. So:
|
||||
|
||||
concurrent streams pods (4 vCPU) vCPU
|
||||
10 4 16
|
||||
100 34 136
|
||||
1000 334 1336
|
||||
|
||||
Per-participant streams are nearly free next to that, because the squelch means a
|
||||
quiet participant is never decoded: about 0.27% of a core for the Silero pass on
|
||||
250 ms of quiet. A thousand five-person meetings is ~1000 voiced streams plus
|
||||
~13 cores of squelch, not 5000 streams.
|
||||
|
||||
Buying 334 four-core nodes is the wrong answer, so state the alternatives with the
|
||||
number each has to beat.
|
||||
|
||||
**Batching is not the lever, and that is measured, not assumed.** The fixed thirty
|
||||
second encode looks exactly like the shape that amortizes, and faster-whisper
|
||||
carries `BatchedInferencePipeline`, but on CPU it does not amortize: eight 12 s
|
||||
windows took 11.64 s sequentially and 6.79 s batched — 1.71x the wall clock for
|
||||
**the same CPU** (24.1 vs 23.6 CPU-seconds). Batching overlaps the work, it does
|
||||
not reduce it, so it helps a pod whose cores are idle and does nothing for one
|
||||
that is already saturated. Where it would pay is a GPU, where the fixed cost is
|
||||
underused hardware rather than arithmetic.
|
||||
|
||||
A GPU has to retire 1000 audio-seconds per wall second of distil-small.en over
|
||||
~16 s windows to replace those 334 pods. **No GPU figure here is measured, the
|
||||
cluster has no GPU node, and every large node is tainted for CI**, so that is a
|
||||
procurement decision to be measured on the card and not believed before it is.
|
||||
|
||||
- Weights bake at image BUILD (deterministic image, boots offline). Adding a
|
||||
model = extend `MODELS` in stt.py/tts.py AND the bake step in the Dockerfile.
|
||||
- **There is no fast tier to add: `whisper` already IS the fast one.** Measured
|
||||
in the pod, both models on the same kokoro fixture, three runs each, median,
|
||||
no HTTP in the loop: `whisper` (distil-small.en) runs at 1.35–2.75× realtime
|
||||
and `whisper-small` (small) at 1.03–1.73×, so the default is 1.25–1.64× the
|
||||
faster of the two, and both transcribe the fixture at WER 0.00 — clean, at
|
||||
20 dB SNR and at 10 dB SNR alike. Splitting live captions from a final pass
|
||||
would mean a bigger model for the final one, and `small` at 1.03× is already
|
||||
the largest that keeps up with a meeting on four cores with no GPU in the
|
||||
cluster. `whisper-small` earns its place on LANGUAGE, not on accuracy or
|
||||
speed: distil-small.en is English-only.
|
||||
- `whisper` / `kokoro` are the ROUTED names the ai plane's provider rows send;
|
||||
renaming them is a cross-repo change, not a local one.
|
||||
- CI is `.hanzo/workflows/deploy.yml` (git.hanzo.ai act_runner; GitHub Actions
|
||||
is disabled account-wide). Version = pyproject's; a published tag is never
|
||||
overwritten — bump patch (x.x.x+1) per release. `pytest` gates the build and
|
||||
runs before the registry login, so a red suite cannot publish.
|
||||
- **Two gates, and the second one exists because the first is blind.**
|
||||
`test_speech.py` replaces the MODEL (never `stt.segments` or `Transcript`), so
|
||||
it needs no weights and the window arithmetic is checkable exactly — its
|
||||
stand-in names each segment after the samples inside it, so a wrong trim
|
||||
changes the words, not just a length. But a suite that only ever sees a
|
||||
stand-in cannot tell a working decoder from a deleted one: breaking `stt.load`
|
||||
leaves it fully green. `test_live.py` closes that — kokoro speaks a sentence
|
||||
and whisper has to say it back, against real weights, and it kills that
|
||||
mutation. It runs as the `live` build stage, which sits above `final` so an
|
||||
untargeted `docker build` still produces an image without dev dependencies.
|
||||
Stub the MODEL, not the function: stubbing the function tests the route and
|
||||
leaves the real one free to regress.
|
||||
- Deploy is declarative: universe `charts/app/values/hanzo/speech.yaml` pins
|
||||
the semver; no ingress (in-cluster only).
|
||||
- **Verifying a deploy: the deployment's image is not the served image.** After a
|
||||
pin, `deploy/speech` reports the new tag immediately while the old pods still
|
||||
answer — the image is ~1.5 GB, so the pull alone runs about a minute and the
|
||||
ReplicaSets overlap. `readyReplicas` counts the OLD pods during that window, so
|
||||
it reads 2/2 and means nothing. Check for a READY pod on the new digest, then
|
||||
ask the running app what it serves (`/openapi.json`) or push real audio through
|
||||
it. There is no ingress, so probing is `kubectl port-forward svc/speech` — which
|
||||
attaches to ONE pod, so mid-rollout it may answer from either version.
|
||||
- Pushes to main have twice failed to create an Actions run (the run is simply
|
||||
never created; `workflow_dispatch` on the same ref works, and other repos on
|
||||
the forge still trigger on push). Check for a run after pushing rather than
|
||||
assuming one exists — the previous release was dispatched by hand too.
|
||||
@@ -0,0 +1,200 @@
|
||||
# speech — CPU STT + TTS, one OpenAI-shaped surface
|
||||
|
||||
Five files are the service: `main.py` (routes), `stt.py` (faster-whisper),
|
||||
`tts.py` (kokoro + ffmpeg for mp3), `transcript.py` (the growing transcript),
|
||||
`vad.py` (the squelch in front of it).
|
||||
Nothing here authenticates: the ai plane (hanzoai/ai, behind api.hanzo.ai)
|
||||
fronts this service and meters callers — including the anon grant tier (~3 uses
|
||||
per anon) which lives THERE, not here.
|
||||
|
||||
## The streaming transcript (`/v1/audio/transcript`)
|
||||
|
||||
Batch stays at `/v1/audio/transcriptions`. The streaming sibling is a resource
|
||||
that grows: `POST` to open, `POST` chunks to it, `DELETE` to close and flush.
|
||||
|
||||
- **Half duplex over plain HTTP, and this is forced, not preferred.** `ai` is a
|
||||
child process of the cloud pod reached over ZAP, and `zap-proto/http` builds a
|
||||
request as ONE frame — there is `FrameResponseHead` for streaming responses but
|
||||
no request-head frame, and `zip` contains no `Hijack`. So a WebSocket and a
|
||||
streamed request body BOTH cannot reach this service, no matter what the edge
|
||||
supports. `hanzoai/ingress` does pass upgrades, so testing `wss://` against
|
||||
ingress alone answers the wrong question — the wall is inside the pod.
|
||||
- Audio is raw little-endian int16, mono, 16 kHz, in the body. No base64 (that
|
||||
is a WebSocket text-frame tax we do not pay), no container to demux.
|
||||
- **Metering: `duration` on each ack is the seconds THAT push put in front of a
|
||||
decoder**, named as the batch route names it, so `ai` reads it into the same
|
||||
`AudioDurationSeconds` it already meters (`stt/openai.go:133`) with no new
|
||||
billing code. It is arithmetic on the bytes that got past the squelch, so it
|
||||
does not depend on the decoder returning anything, and summing the pushes
|
||||
yields the audio decoded — never more than the audio submitted. A push the
|
||||
squelch drops reports 0.0; the push that OPENS it reports more than its own
|
||||
length, because it carries the lead-in that earlier pushes withheld.
|
||||
- **Silence is not decoded and not billed** (`vad.py`). Silero already ships
|
||||
inside faster-whisper and already runs inside every decode, so running it in
|
||||
FRONT of the decoder costs no new weight and no new bake step — 0.67 ms per
|
||||
250 ms push, measured on the pod, against a decode of seconds. It is a squelch,
|
||||
not a per-window classifier: two thresholds to open and close, `HANG` of quiet
|
||||
before it closes, and `LEAD` of audio held back and handed over by the push
|
||||
that opens. Measured on the pod: an idle room bills 0.0 up to about -45 dBFS
|
||||
RMS of room tone (it starts to open on tone above that); a keyboard click and
|
||||
full-scale white noise do not open it; a meeting bills its speech plus about
|
||||
0.9 s per utterance and nothing for the lulls, so the share saved is 0% when
|
||||
someone is talking continuously and 86% at 30 s between turns.
|
||||
**The lead-in is the part that is easy to get wrong and expensive to lose.**
|
||||
Sliding a word across a push shows why: with `LEAD = 0`, 8 of 32 alignments
|
||||
clip up to 71.8 ms off the front of the word, because a push is admitted whole
|
||||
and a word beginning near the end of one has nothing in hand. At 0.5 s, none
|
||||
clip and the worst case still keeps 428 ms of margin. Hold the lead by slicing
|
||||
from the FRONT — `[-int(LEAD * SECOND):]` reads as the last N bytes and is the
|
||||
whole buffer when N is 0, which keeps every second of silence and hands the lot
|
||||
to the decoder on the next word.
|
||||
- Whisper decodes a span, not a sample. The window is re-decoded as it fills;
|
||||
segments ending a guard-length before the audio are committed and their bytes
|
||||
freed, which is what bounds the window and stops committed text being revised.
|
||||
|
||||
**Sessions are pod-local, and `speech.yaml` sets `replicas: 2`.** Pushes reach
|
||||
`speech.hanzo.svc`, which balances, so a session opened on one pod is not found
|
||||
on the other. Whoever routes must pin a session to the pod that owns it; that
|
||||
belongs in `ai` (which already does provider routing), not here — this service
|
||||
decodes, it does not route. Until that exists the endpoint is not usable through
|
||||
the plane, and `ai` has no route for it yet either.
|
||||
|
||||
## What it sustains, and where that number comes from
|
||||
|
||||
`bench.py` is the instrument: `say` once for a corpus of real speech carrying a
|
||||
word-to-time map, then `cost` for what a decode pass costs and `talk` to drive N
|
||||
sessions in real time. Everything below was measured with it **on a four vCPU
|
||||
DigitalOcean worker** — the shape a speech pod runs in. That qualifier is the
|
||||
whole point: the same code and the same run on four threads of a desktop Zen 5
|
||||
retires 13.7 audio-seconds per wall-second where the pod retires 5.7. A capacity
|
||||
number measured on the box is 2.4x the truth.
|
||||
|
||||
**Ack latency cannot see any of this and none of these numbers is one.** A push
|
||||
returns whatever text the session has, so it costs a memcpy whether the decoder
|
||||
is idle or minutes behind — 3.8 ms while the backlog grew fourfold. The metric is
|
||||
backlog, as lag: wall seconds elapsed minus seconds of audio already transcribed.
|
||||
The verdict is its SLOPE over the second half of a run, in seconds of lag added
|
||||
per minute of meeting. Runs must be MINUTES long: a stalling session looks fine
|
||||
for its first four minutes, which is how seconds-long runs came to predict eight
|
||||
concurrent meetings and were wrong by an order of magnitude.
|
||||
|
||||
**A pass costs what it costs, almost regardless of what it covers.** Whisper
|
||||
always encodes a thirty second mel window, so on the pod one pass takes 2.7 s over
|
||||
a one second window and 3.2 s over an eight second one — about 8.4 CPU-seconds
|
||||
either way. Capacity is a budget of PASSES, not of audio, and the way to buy it is
|
||||
to pass less often, never to pass over less.
|
||||
|
||||
**Past the mel window that cost multiplies.** 24 s -> 5.0 s per pass, 30 s -> 7.5,
|
||||
45 s -> 23.5. It is a cliff, not a slope.
|
||||
|
||||
**So the pod has a throughput PEAK, and the peak is the instability.** In steady
|
||||
state a session gets one pass per interval and that pass covers the audio that
|
||||
arrived during it, so audio retired per wall second IS the number of real-time
|
||||
streams served, and lag is roughly the window. Measured on the pod:
|
||||
|
||||
window 8 s 12 s 16 s 24 s 30 s
|
||||
streams served 3.1 4.4 5.0 3.9 2.8
|
||||
|
||||
Below the peak the loop is self-correcting: a longer window is cheaper per second,
|
||||
so lag comes back. Past it, a longer window is DEARER per second, so lag grows,
|
||||
which lengthens the window — runaway. That is why the failure is a cliff and not a
|
||||
gentle degradation, and why the working point has to sit on the rising branch
|
||||
rather than at the top of the curve.
|
||||
|
||||
**Measured, in eight minute runs at 250 ms chunks**, and the two agree to within a
|
||||
few percent — a session settles where its window buys exactly its own concurrency:
|
||||
|
||||
sessions lag slope verdict
|
||||
3 7.7 s -2.36 s/min settles
|
||||
4 12.7 s -0.45 s/min settles
|
||||
|
||||
So: **four concurrent streams of continuous speech per four vCPU pod**, with the
|
||||
transcript about thirteen seconds behind, or three at eight seconds. Two replicas
|
||||
is eight. The curve says five sits ON the peak and six has no fixed point at all.
|
||||
|
||||
The failure past that edge is not a gentle one. The same runs against the earlier
|
||||
two-worker build, which peaked lower, settled at one and two sessions and at FOUR
|
||||
ran away to 326 s of lag at +44 s/min — and closing those sessions then blocked for
|
||||
400 s each, because `DELETE` decodes everything that piled up.
|
||||
|
||||
**A session's ceiling is `LIMIT` = 600 s, so a 45 minute meeting is five sessions,
|
||||
not one.** Closing decodes the whole remaining window and blocks, and the transport
|
||||
admits one outstanding chunk, so the roll costs the microphone that whole pause.
|
||||
At three streams that is a couple of seconds every ten minutes; on a session that
|
||||
has fallen behind it is minutes.
|
||||
|
||||
### What more of it costs
|
||||
|
||||
Per pod, three streams. Per stream of continuous speech that is 1.33 vCPU. So:
|
||||
|
||||
concurrent streams pods (4 vCPU) vCPU
|
||||
10 4 16
|
||||
100 34 136
|
||||
1000 334 1336
|
||||
|
||||
Per-participant streams are nearly free next to that, because the squelch means a
|
||||
quiet participant is never decoded: about 0.27% of a core for the Silero pass on
|
||||
250 ms of quiet. A thousand five-person meetings is ~1000 voiced streams plus
|
||||
~13 cores of squelch, not 5000 streams.
|
||||
|
||||
Buying 334 four-core nodes is the wrong answer, so state the alternatives with the
|
||||
number each has to beat.
|
||||
|
||||
**Batching is not the lever, and that is measured, not assumed.** The fixed thirty
|
||||
second encode looks exactly like the shape that amortizes, and faster-whisper
|
||||
carries `BatchedInferencePipeline`, but on CPU it does not amortize: eight 12 s
|
||||
windows took 11.64 s sequentially and 6.79 s batched — 1.71x the wall clock for
|
||||
**the same CPU** (24.1 vs 23.6 CPU-seconds). Batching overlaps the work, it does
|
||||
not reduce it, so it helps a pod whose cores are idle and does nothing for one
|
||||
that is already saturated. Where it would pay is a GPU, where the fixed cost is
|
||||
underused hardware rather than arithmetic.
|
||||
|
||||
A GPU has to retire 1000 audio-seconds per wall second of distil-small.en over
|
||||
~16 s windows to replace those 334 pods. **No GPU figure here is measured, the
|
||||
cluster has no GPU node, and every large node is tainted for CI**, so that is a
|
||||
procurement decision to be measured on the card and not believed before it is.
|
||||
|
||||
- Weights bake at image BUILD (deterministic image, boots offline). Adding a
|
||||
model = extend `MODELS` in stt.py/tts.py AND the bake step in the Dockerfile.
|
||||
- **There is no fast tier to add: `whisper` already IS the fast one.** Measured
|
||||
in the pod, both models on the same kokoro fixture, three runs each, median,
|
||||
no HTTP in the loop: `whisper` (distil-small.en) runs at 1.35–2.75× realtime
|
||||
and `whisper-small` (small) at 1.03–1.73×, so the default is 1.25–1.64× the
|
||||
faster of the two, and both transcribe the fixture at WER 0.00 — clean, at
|
||||
20 dB SNR and at 10 dB SNR alike. Splitting live captions from a final pass
|
||||
would mean a bigger model for the final one, and `small` at 1.03× is already
|
||||
the largest that keeps up with a meeting on four cores with no GPU in the
|
||||
cluster. `whisper-small` earns its place on LANGUAGE, not on accuracy or
|
||||
speed: distil-small.en is English-only.
|
||||
- `whisper` / `kokoro` are the ROUTED names the ai plane's provider rows send;
|
||||
renaming them is a cross-repo change, not a local one.
|
||||
- CI is `.hanzo/workflows/deploy.yml` (git.hanzo.ai act_runner; GitHub Actions
|
||||
is disabled account-wide). Version = pyproject's; a published tag is never
|
||||
overwritten — bump patch (x.x.x+1) per release. `pytest` gates the build and
|
||||
runs before the registry login, so a red suite cannot publish.
|
||||
- **Two gates, and the second one exists because the first is blind.**
|
||||
`test_speech.py` replaces the MODEL (never `stt.segments` or `Transcript`), so
|
||||
it needs no weights and the window arithmetic is checkable exactly — its
|
||||
stand-in names each segment after the samples inside it, so a wrong trim
|
||||
changes the words, not just a length. But a suite that only ever sees a
|
||||
stand-in cannot tell a working decoder from a deleted one: breaking `stt.load`
|
||||
leaves it fully green. `test_live.py` closes that — kokoro speaks a sentence
|
||||
and whisper has to say it back, against real weights, and it kills that
|
||||
mutation. It runs as the `live` build stage, which sits above `final` so an
|
||||
untargeted `docker build` still produces an image without dev dependencies.
|
||||
Stub the MODEL, not the function: stubbing the function tests the route and
|
||||
leaves the real one free to regress.
|
||||
- Deploy is declarative: universe `charts/app/values/hanzo/speech.yaml` pins
|
||||
the semver; no ingress (in-cluster only).
|
||||
- **Verifying a deploy: the deployment's image is not the served image.** After a
|
||||
pin, `deploy/speech` reports the new tag immediately while the old pods still
|
||||
answer — the image is ~1.5 GB, so the pull alone runs about a minute and the
|
||||
ReplicaSets overlap. `readyReplicas` counts the OLD pods during that window, so
|
||||
it reads 2/2 and means nothing. Check for a READY pod on the new digest, then
|
||||
ask the running app what it serves (`/openapi.json`) or push real audio through
|
||||
it. There is no ingress, so probing is `kubectl port-forward svc/speech` — which
|
||||
attaches to ONE pod, so mid-rollout it may answer from either version.
|
||||
- Pushes to main have twice failed to create an Actions run (the run is simply
|
||||
never created; `workflow_dispatch` on the same ref works, and other repos on
|
||||
the forge still trigger on push). Check for a run after pushing rather than
|
||||
assuming one exists — the previous release was dispatched by hand too.
|
||||
Reference in New Issue
Block a user