Files
zeekayandHanzo Dev 6a363f7ee3 merge: upstream main ddf1b879 and the aarch64 repack branch 35d7ae7c
Three-way per file through the rename: base 39355c6c91, theirs the upstream
merge of main with 35d7ae7c (via 7ef6a694d), ours origin/main.

Lands the barrier thread pool, runtime CPU feature dispatch, aarch64 and x86
repacked quantized kernels (indexed_gemv, gemv_fused_shared_lhs), quantized
embedding on CPU/Metal/CUDA, paged flash-attn kernels, cutile fused MoE, the
in-place op aliasing API, parking_lot storage locks, Metal buffer builders and
debug labels, safetensors 0.8, hf-hub 1.0, tokenizers 0.23.

Ours keeps ROCm, Vulkan, wgpu, the IQ/MXFP4/NVFP4/TQ dtypes, and the i-quant
weight/activation block split in matmul. Our backends ride InplaceOpN and the
quantized embedding; Metal get_rows gains the IQ kernels. ug leaves, as upstream.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-15 15:52:28 -07:00
..
2026-01-16 12:30:55 -08:00

Using ONNX models in ML

This example demonstrates how to run ONNX based models in ML.

It contains small variants of two models, SqueezeNet (default) and EfficientNet.

You can run the examples with following commands:

cargo run --example onnx --features=onnx --release -- --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg

Use the --which flag to specify explicitly which network to use, i.e.

$ cargo run --example onnx --features=onnx --release -- --which squeeze-net --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg

    Finished release [optimized] target(s) in 0.21s
     Running `target/release/examples/onnx --which squeeze-net --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg`
loaded image Tensor[dims 3, 224, 224; f32]
unicycle, monocycle                               : 83.23%
ballplayer, baseball player                       : 3.68%
bearskin, busby, shako                            : 1.54%
military uniform                                  : 0.78%
cowboy hat, ten-gallon hat                        : 0.76%
$ cargo run --example onnx --features=onnx --release -- --which efficient-net --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg

    Finished release [optimized] target(s) in 0.20s
     Running `target/release/examples/onnx --which efficient-net --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg`
loaded image Tensor[dims 224, 224, 3; f32]
bicycle-built-for-two, tandem bicycle, tandem     : 99.16%
mountain bike, all-terrain bike, off-roader       : 0.60%
unicycle, monocycle                               : 0.17%
crash helmet                                      : 0.02%
alp                                               : 0.02%