Files
engine/docs/SUPPORTED_MODELS.md

4.8 KiB

Supported Models

Complete reference for model support in Hanzo Engine.

Model Categories

Text Models

  • Granite 4.0
  • SmolLM 3
  • DeepSeek V3
  • GPT-OSS
  • DeepSeek V2
  • Qwen 3 Next
  • Qwen 3 MoE
  • Phi 3.5 MoE
  • Qwen 3
  • GLM 4
  • GLM-4.7-Flash
  • GLM-4.7 (MoE)
  • Gemma 2
  • Qwen 2
  • Starcoder 2
  • Phi 3
  • Mixtral
  • Phi 2
  • Gemma
  • Llama
  • Mistral

Multimodal Models

  • Qwen 3-VL
  • Qwen 3-VL MoE
  • Qwen 3.5
  • Qwen 3.5 MoE
  • Gemma 4 (text, image, video, audio)
  • Gemma 3n
  • Llama 4
  • Gemma 3
  • Mistral 3
  • Phi 4 multimodal
  • Qwen 2.5-VL
  • MiniCPM-O
  • Llama 3.2 Vision
  • Qwen 2-VL
  • Idefics 3
  • Idefics 2
  • LLaVA Next
  • LLaVA
  • Phi 3V

Speech Models

  • Voxtral (ASR/speech-to-text)
  • Dia

Image Generation Models

  • FLUX

Embedding Models

  • Embedding Gemma
  • Qwen 3 Embedding

Request a new model

Supported GGUF Architectures

Plain:

  • llama
  • phi2
  • phi3
  • starcoder2
  • qwen2
  • qwen3
  • qwen3moe

With adapters:

  • llama
  • phi3

Quantization Support

Model GGUF GGML ISQ
Mistral ✅ ✅
Gemma ✅
Llama ✅ ✅ ✅
Mixtral ✅ ✅
Phi 2 ✅ ✅
Phi 3 ✅ ✅
Phi 3.5 MoE ✅
Qwen 2.5 ✅
Phi 3 Vision ✅
Idefics 2 ✅
Gemma 2 ✅
GLM4 ✅
GLM-4.7-Flash (MoE) ✅
GLM-4.7 (MoE) ✅
Starcoder 2 ✅ ✅
LLaVa Next ✅
LLaVa ✅
Llama 3.2 Vision ✅
Qwen2-VL ✅
Idefics 3 ✅
Deepseek V2 ✅
Deepseek V3 ✅
MiniCPM-O 2.6 ✅
Qwen2.5-VL ✅
Gemma 3 ✅
Mistral 3 ✅
Llama 4 ✅
Qwen 3 ✅ ✅
SmolLM3 ✅
Dia 1.6b ✅
Voxtral ✅
Gemma 3n ✅
Gemma 4 ✅
Qwen 3 VL ✅
Qwen 3 MoE ✅ ✅
Qwen 3-VL MoE ✅
Qwen 3.5 ✅
Qwen 3 Next ✅
Phi 4 Multimodal ✅
Granite 4.0 ✅
GPT-OSS ✅

Device Mapping Support

Model category Supported
Plain ✅
GGUF ✅
GGML
Multimodal Plain ✅

X-LoRA and LoRA Support

Model X-LoRA X-LoRA+GGUF X-LoRA+GGML
Mistral ✅ ✅
Gemma ✅
Llama ✅ ✅ ✅
Mixtral ✅ ✅
Phi 2 ✅
Phi 3 ✅ ✅
Phi 3.5 MoE
Qwen 2.5
Phi 3 Vision
Idefics 2
Gemma 2 ✅
GLM4 ✅
GLM-4.7-Flash (MoE)
GLM-4.7 (MoE)
Starcoder 2 ✅
LLaVa Next
LLaVa
Qwen2-VL
Idefics 3
Deepseek V2
Deepseek V3
MiniCPM-O 2.6
Qwen2.5-VL
Gemma 3
Mistral 3
Llama 4
Qwen 3
SmolLM3 ✅
Gemma 3n
Gemma 4
Voxtral
Qwen 3 VL
Qwen 3-VL MoE
Qwen 3.5
Qwen 3 Next
Phi 4 Multimodal
Llama 3.2 Vision
Granite 4.0
GPT-OSS

AnyMoE Support

Model AnyMoE
Mistral 7B ✅
Gemma ✅
Llama ✅
Mixtral
Phi 2 ✅
Phi 3 ✅
Phi 3.5 MoE
Qwen 2.5 ✅
Phi 3 Vision
Idefics 2
Gemma 2 ✅
GLM-4.7-Flash (MoE)
GLM-4.7 (MoE)
Starcoder 2 ✅
LLaVa Next ✅
LLaVa ✅
Llama 3.2 Vision
Qwen2-VL
Idefics 3 ✅
Deepseek V2
Deepseek V3
MiniCPM-O 2.6
Qwen2.5-VL
Gemma 3 ✅
Mistral 3 ✅
Llama 4
Qwen 3
SmolLM3 ✅
Gemma 3n
Gemma 4
Voxtral
Qwen 3 VL
Qwen 3-VL MoE
Qwen 3.5
Qwen 3 Next
Phi 4 Multimodal
Dia 1.6b
Granite 4.0
GPT-OSS

Using Derivative Models

Model type is auto-detected. Use flags for quantized models and adapters:

Model Type Required Arguments
Plain -m <model-id>
GGUF Quantized -m <model-id> --format gguf -f <file>
ISQ Quantized -m <model-id> --isq <level>
UQFF Quantized -m <model-id> --from-uqff <file>
LoRA -m <model-id> --lora <adapter>
X-LoRA -m <model-id> --xlora <adapter> --xlora-order <file>

Example: Zephyr GGUF model

hanzo serve -p 1234 --log output.txt --format gguf -t HuggingFaceH4/zephyr-7b-beta -m TheBloke/zephyr-7B-beta-GGUF -f zephyr-7b-beta.Q5_0.gguf

Chat Templates and Tokenizer

Hanzo Engine will attempt to automatically load a chat template and tokenizer. This enables high flexibility across models and ensures accurate and flexible chat templating. However, this behavior can be customized.