Files
zeekayandHanzo Dev d0c9e6841e style: rustfmt the workspace
`cargo fmt --all -- --check` is the forge's CI gate (hanzo.yml) and failed
on main. This is rustfmt 1.9.0 (stable 1.98) over the workspace, nothing
else: 30 files, no code change.

Co-authored-by: Hanzo Dev <[email protected]>
2026-10-16 21:01:50 -07:00
..
2026-10-16 21:01:50 -07:00
2026-10-16 21:01:50 -07:00

hanzo-quant

An advanced and highly diverse set of quantization techniques. This crate supports both quantization and optimized inference.

It has grown beyon simply quantization and is used by hanzo to power:

  • ISQ
  • Imatrix collection
  • General quantization features
  • Specific CUDA and Metal features
  • cuBLASlt integration

Currently supported:

  • AFQ: GgufMatMul (2-8 bit quantization optimized for Metal and compatible with MLX)
  • GGUF: GgufMatMul (2-8 bit quantization, with imatrix)
  • Gptq/Awq: GptqAwqLayer (with CUDA marlin kernel)
  • Hqq: HqqLayer (4, 8 bit quantization)
  • FP8: FP8Linear
  • F8Q8: F8Q8Linear
  • Unquantized (used for ISQ): UnquantLinear
  • Bnb: BnbLinear (int8, fp4, nf4)

Some kernels are copied from or based on third-party implementations, among them:

Each such file keeps its license header, and the workspace NOTICE lists every copyright holder.