Files
AIDDSPandGitHub 4283b1c7ad test(qa): audio generation and transcription suite (#149)
Adds qa/audio: TTS container validators for mp3, wav, opus, aac, flac and pcm, a transcript timestamp audit, generated fixtures and live lanes for /v1/audio/speech and /v1/audio/transcriptions. Offline tests run in CI; live lanes skip without an opt-in or key.

Fixes #133

Claude-Session: https://claude.ai/code/session_015XYKLix9PSyn6Jw2HWKprk
2026-11-12 19:19:21 -08:00

87 lines
4.0 KiB
Go

// Copyright 2023-2026 Hanzo AI Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package audio is the QA suite for #133, Audio Generation & Transcription: a
// reproducible test battery for the gateway's OpenAI-compatible speech surface —
// POST /v1/audio/speech (voices, speeds, the six response_format containers, and
// chunked consumption of streamed bytes) and POST /v1/audio/transcriptions
// (multilingual round trips, verbose_json word/segment timestamps, long
// recordings, silence handling, and the key-free /transcriptions/public door).
//
// The offline half is what CI runs: every WAV fixture is generated byte-for-byte
// at run time from fixtures.go (no binary blobs in git), and formats.go proves
// each container validator against hand-built valid AND corrupt containers —
// truncated MP3/Xing, a FLAC stream whose metadata block runs past EOF, an OGG
// page with a broken CRC, a WAVE file whose data chunk lies about its size.
// transcript.go proves the timestamp checker against canned conforming and
// divergent verbose_json bodies: monotonicity breaks, words escaping their
// segment, duration drift, a recording silently cut mid-stream. The loudness
// checks (formats.go Level) pin that a "successful" synthesis actually carries
// signal — the failure mode where a valid container arrives full of silence.
//
// The live half talks to the real gateway and skips without opt-in: the keyed
// lanes (HANZO_QA_LIVE=1 + HANZO_API_KEY) run the format matrix and the TTS→STT
// round trips; one key-free public lane (HANZO_QA_LIVE=1 alone) exercises the
// public transcriber. HANZO_QA_REPORT writes every live verdict to a JSON
// artifact plus a Markdown twin beside it, which is the request/response trace
// the acceptance criteria ask for. See README.md for the lane table.
package audio
import (
"os"
"strings"
"time"
)
// DefaultBase is the production gateway. HANZO_API_URL may name the URL with or
// without the /v1 suffix (the docs print it without). These env names are shared
// with qa/crossmodel (#138) and qa/multimodal (#135) on purpose — one CI secret
// configures all three suites.
const DefaultBase = "https://api.hanzo.ai"
const (
BaseURLEnv = "HANZO_API_URL"
KeyEnv = "HANZO_API_KEY" // keyed speech/STT lanes; absent → those lanes skip
ReportEnv = "HANZO_QA_REPORT" // artifact path for report.json (+ a .md twin beside it)
LivePublicEnv = "HANZO_QA_LIVE" // opts the key-free public checks into real network calls
// RequestTimeout bounds one audio round trip. Synthesis of the long-recording
// fixture speaks ~80 s of audio, so this is generous; every cell still fails
// fast on its own context.
RequestTimeout = 180 * time.Second
)
// The two SKUs under test, pinned from conf/models.yaml so catalog drift fails
// loudly instead of silently testing a different engine. `provider: speech` in
// the yaml is the in-cluster service; these Zen names are what callers send.
const (
SpeechModel = "zen-voice-mini" // Kokoro TTS, served at /v1/audio/speech
ScribeModel = "zen-scribe" // Parakeet/Whisper STT, served at /v1/audio/transcriptions
PublicTranscriptionsPath = "/audio/transcriptions/public" // no credential, ≤1 minute of audio
)
// BaseURL resolves the gateway root, ending in /v1 (the same shape qa/crossmodel
// and qa/multimodal print).
func BaseURL() string {
base := strings.TrimRight(os.Getenv(BaseURLEnv), "/")
if base == "" {
base = DefaultBase
}
if !strings.HasSuffix(base, "/v1") {
base += "/v1"
}
return base
}