mirror of
https://github.com/hanzoai/ml.git
synced 2026-11-20 23:08:29 +00:00
Three-way per file through the rename: base 39355c6c91, theirs the upstream
merge of main with 35d7ae7c (via 7ef6a694d), ours origin/main.
Lands the barrier thread pool, runtime CPU feature dispatch, aarch64 and x86
repacked quantized kernels (indexed_gemv, gemv_fused_shared_lhs), quantized
embedding on CPU/Metal/CUDA, paged flash-attn kernels, cutile fused MoE, the
in-place op aliasing API, parking_lot storage locks, Metal buffer builders and
debug labels, safetensors 0.8, hf-hub 1.0, tokenizers 0.23.
Ours keeps ROCm, Vulkan, wgpu, the IQ/MXFP4/NVFP4/TQ dtypes, and the i-quant
weight/activation block split in matmul. Our backends ride InplaceOpN and the
quantized embedding; Metal get_rows gains the IQ kernels. ug leaves, as upstream.
Co-authored-by: Hanzo Dev <[email protected]>
hanzo-ml-whisper: speech recognition
An implementation of OpenAI Whisper using
hanzo. Whisper is a general purpose speech recognition model, it can be used to
convert audio files (in the .wav format) to text. Supported features include
language detection as well as multilingual speech recognition.
Running some example
If no audio file is passed as input, a sample file is automatically downloaded from the hub.
cargo run --example whisper --release --features="symphonia"
> No audio file submitted: Downloading https://huggingface.co/datasets/Narsil/candle_demo/blob/main/samples_jfk.wav
> loaded wav data: Header { audio_format: 1, channel_count: 1, sampling_rate: 16000, bytes_per_second: 32000, bytes_per_sample: 2, bits_per_sample: 16 }
> pcm data loaded 176000
> loaded mel: [1, 80, 3000]
> 0.0s -- 30.0s: And so my fellow Americans ask not what your country can do for you ask what you can do for your country
In order to use the multilingual mode, specify a multilingual model via the
--model flag, see the details below.
Command line flags
--input: the audio file to be converted to text, in wav format.--language: force the language to some specific value rather than being detected, e.g.en.--task: the task to be performed, can betranscribe(return the text data in the original language) ortranslate(translate the text to English).--timestamps: enable the timestamp mode where some timestamps are reported for each recognized audio extracts.--model: the model to be used. Models that do not end with-enare multilingual models, other ones are English only models. The supported OpenAI Whisper models aretiny,tiny.en,base,base.en,small,small.en,medium,medium.en,large,large-v2andlarge-v3. The supported Distil-Whisper models aredistil-medium.en,distil-large-v2anddistil-large-v3.