Files
zeekayandHanzo Dev 6a363f7ee3 merge: upstream main ddf1b879 and the aarch64 repack branch 35d7ae7c
Three-way per file through the rename: base 39355c6c91, theirs the upstream
merge of main with 35d7ae7c (via 7ef6a694d), ours origin/main.

Lands the barrier thread pool, runtime CPU feature dispatch, aarch64 and x86
repacked quantized kernels (indexed_gemv, gemv_fused_shared_lhs), quantized
embedding on CPU/Metal/CUDA, paged flash-attn kernels, cutile fused MoE, the
in-place op aliasing API, parking_lot storage locks, Metal buffer builders and
debug labels, safetensors 0.8, hf-hub 1.0, tokenizers 0.23.

Ours keeps ROCm, Vulkan, wgpu, the IQ/MXFP4/NVFP4/TQ dtypes, and the i-quant
weight/activation block split in matmul. Our backends ride InplaceOpN and the
quantized embedding; Metal get_rows gains the IQ kernels. ug leaves, as upstream.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-15 15:52:28 -07:00
..

hanzo-ml-mamba2: Mamba2 implementation

Hanzo implementation of Mamba2 [1] inference. Mamba2 introduces the State Space Duality (SSD) framework which unifies structured SSMs and attention variants.

Running the example

cargo run --example mamba2 --release -- --prompt "Mamba is the"

Supported models

Model HuggingFace ID
Mamba2-130m AntonV/mamba2-130m-hf
Mamba2-370m AntonV/mamba2-370m-hf
Mamba2-780m AntonV/mamba2-780m-hf
Mamba2-1.3b AntonV/mamba2-1.3b-hf
Mamba2-2.7b AntonV/mamba2-2.7b-hf

Verification

Outputs match the PyTorch transformers Mamba2ForCausalLM reference implementation.

mamba2-130m

cargo run --example mamba2 --release -- \
  --prompt "Mamba is the" \
  --which mamba2-130m \
  --sample-len 20 \
  --repeat-penalty 1.0

Expected output:

Mamba is the most popular and popular game in the world. It is a game where you can play with your friends

mamba2-370m

cargo run --example mamba2 --release -- \
  --prompt "Mamba is the" \
  --which mamba2-370m \
  --sample-len 20 \
  --repeat-penalty 1.0

Expected output:

Mamba is the first game in the series to feature a new character, the Mamba, who is a female version