Files
zeekayandHanzo Dev b2832f1c55 cli: hanzo-engine bench is the one instrument, and board scores it
hanzo-bench is the library behind `hanzo-engine bench` (its `measure` loop) and
`hanzo-engine board score|publish|manifest`; its binary is gone, as is the CLI's
earlier bench that read the response's self-reported timing. The dossier calls
the CLI. Documented in docs/CLI.md and the CLI reference.

Co-authored-by: Hanzo Dev <[email protected]>
2026-09-17 19:15:05 -07:00
..

hanzo-bench

Deprecated: The standalone hanzo-bench package is deprecated. Use hanzo bench instead for the same functionality.

Migration:

# Old
cargo run --release --features cuda --package hanzo-bench -- plain -m model-id

# New
hanzo bench -m model-id

This is our official benchmarking application, which allows you to collect structured information about the speed of hanzo.

Note

You should replace --features ... with one of the features specified here, or remove it for pure CPU inference.

To run: cargo run --release --features ... --package hanzo-bench

Fast and easy LLM serving.

Usage: hanzo-bench [OPTIONS] <COMMAND>

Commands:
  plain        Select a plain model
  x-lora       Select an X-LoRA architecture
  lora         Select a LoRA architecture
  gguf         Select a GGUF model
  x-lora-gguf  Select a GGUF model with X-LoRA
  lora-gguf    Select a GGUF model with LoRA
  ggml         Select a GGML model
  x-lora-ggml  Select a GGML model with X-LoRA
  lora-ggml    Select a GGML model with LoRA
  help         Print this message or the help of the given subcommand(s)

Options:
  -p, --n-prompt <N_PROMPT>
          Number of prompt tokens to run [default: 512]
  -g, --n-gen <N_GEN>
          Number of generations tokens to run [default: 128]
  -c, --concurrency <CONCURRENCY>
          Number of concurrent requests to run. Default is 1
  -r, --repetitions <REPETITIONS>
          Number of times to repeat each test [default: 5]
  -n, --num-device-layers <NUM_DEVICE_LAYERS>
          Number of device layers to load and run on the device. All others will be on the CPU
  -h, --help
          Print help
  -V, --version
          Print version