Files
zeekayandHanzo Dev 02b3f00cef chore: complete brand scrub of upstream mistral.rs lineage to hanzo/hanzoai
Replace all remaining references to the upstream project/author/framework
brands with hanzo / hanzoai across source, comments, docs, examples, and
build scripts.

- Project spellings mistral.rs / mistral-rs / mistralrs -> hanzo (and the
  hanzo-* crate/binary names in Dockerfiles, Makefile, install.ps1, .gitignore).
- Author EricLBuehler / EricBuehler / EricB / Eric Buehler -> hanzoai / Hanzo AI
  (incl. Cargo `authors` and LICENSE copyright).
- URLs: github.com/EricLBuehler/mistral.rs -> github.com/hanzoai/engine;
  ericlbuehler.github.io/mistral.rs -> hanzoai.github.io/engine;
  github.com/huggingface/candle -> github.com/hanzoai/ml;
  docs.rs/candle-* -> docs.rs/hanzo-ml; ghcr.io image -> ghcr.io/hanzoai/engine.
- Env var prefix MISTRALRS_ -> HANZO_ (producers and consumers kept in sync,
  incl. the build-script-injected HANZO_GIT_REVISION).
- ROCm static lib libmistralrsrocm.a / mistralrsrocm -> libhanzorocm.a / hanzorocm.
- localStorage keys mistralrs_settings -> hanzo_settings.

Deliberately preserved: real model architectures (struct Mistral, models/mistral.rs
file, Mixtral/Llama/Gemma/Qwen/Phi), real model-org HF repo-ids (mistralai/,
meta-llama/, google/, Qwen/, microsoft/), the `use hanzo_ml as candle` code
aliases and `candle::` identifiers, and the upstream-author HF data repo-id
suffix in "hanzoai/mistralrs_tests".

Both `cargo check` and `cargo check --features vulkan` pass with 0 errors.

Co-authored-by: Hanzo Dev <[email protected]>
2026-06-01 11:53:12 -07:00

2.8 KiB

Qwen 3 Next: collection

Qwen3-Coder-Next is a coding-focused language model using a hybrid Gated Delta Network (GDN) + full attention architecture with Mixture of Experts. With only 3B activated parameters (80B total), it achieves performance comparable to models with 10-20x more active parameters. It supports a 256K context window.

Quick Start

hanzo run --isq 4 -m Qwen/Qwen3-Coder-Next

Note: hanzo can load the FP8 pre-quantized version natively! Simply replace the model ID.

GGUF quantized models are also supported:

hanzo run --format gguf -m Qwen/Qwen3-Coder-Next-GGUF -f <filename>

HTTP API

You can find a more detailed example here.

hanzo serve --isq 4 -p 1234 -m Qwen/Qwen3-Coder-Next
from openai import OpenAI

client = OpenAI(api_key="foobar", base_url="http://localhost:1234/v1/")

completion = client.chat.completions.create(
    model="default",
    messages=[
        {"role": "user", "content": "Tell me a story about the Rust type system."}
    ],
    max_tokens=256,
)
print(completion.choices[0].message.content)

Python SDK

You can find a more detailed example here.

from hanzo import Runner, Which, ChatCompletionRequest, Architecture

runner = Runner(
    which=Which.Plain(
        model_id="Qwen/Qwen3-Coder-Next",
        arch=Architecture.Qwen3Next,
    ),
    in_situ_quant="Q4K",
)

res = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="default",
        messages=[
            {"role": "user", "content": "Write a Python function to compute fibonacci numbers."}
        ],
        max_tokens=256,
        presence_penalty=1.0,
        top_p=0.1,
        temperature=0.1,
    )
)
print(res.choices[0].message.content)
print(res.usage)

Rust SDK

You can find a more detailed example here.

use anyhow::Result;
use hanzo::{
    IsqType, PagedAttentionMetaBuilder, TextMessageRole, TextMessages, TextModelBuilder,
};

#[tokio::main]
async fn main() -> Result<()> {
    let model = TextModelBuilder::new("Qwen/Qwen3-Coder-Next")
        .with_isq(IsqType::Q4K)
        .with_logging()
        .with_paged_attn(|| PagedAttentionMetaBuilder::default().build())?
        .build()
        .await?;

    let messages = TextMessages::new()
        .add_message(TextMessageRole::User, "Hello!");

    let response = model.send_chat_request(messages).await?;
    println!("{}", response.choices[0].message.content.as_ref().unwrap());

    Ok(())
}