mirror of
https://github.com/zenlm/zen-docs.git
synced 2026-10-30 14:58:13 +00:00
zeekay
8ef8f3e801
Updated 8 existing stubs and added 4 new model pages: - zen-nano: 0.6B edge model specs, throughput table, benchmarks - zen-scribe: 4B writing model, multilingual, use cases - zen-omni: 30B hypermodal Thinker-Talker MoE, speech-to-speech - zen-vl: 4B/8B/30B VL family with function calling and agent variants - zen-embedding: MTEB #1 multilingual (70.58), MRL support - zen-coder: 4B/31B/480B MoE family, FIM, 358 languages - zen-designer: 235B VL MoE, document parsing, GUI interaction - zen-max: 1T+ MoE, 256K context, 200+ tool calls, heavy mode - zen-pro: 8B instruct/thinking/agent variants - zen-reranker: 8B dual-task, native 7680-dim for DSO - zen-translator: real-time translation pipeline with voice cloning - zen-guard: removed upstream model reference No upstream model names in any page (Zen MoDE branding throughout).
100 lines
2.5 KiB
Plaintext
100 lines
2.5 KiB
Plaintext
---
|
|
title: Zen Nano
|
|
description: Ultra-lightweight 0.6B edge model for mobile and IoT deployment
|
|
---
|
|
|
|
# Zen Nano
|
|
|
|
Zen Nano is an ultra-lightweight 0.6B parameter language model from the Zen MoDE family, optimized for edge devices, mobile applications, and resource-constrained environments. It delivers practical language capabilities at 44,000 tokens/sec on Apple Silicon.
|
|
|
|
## Specifications
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Parameters | 0.6B |
|
|
| Architecture | Zen MoDE (dense transformer) |
|
|
| Context Length | 40,960 tokens |
|
|
| Languages | English, Chinese |
|
|
| License | Apache 2.0 |
|
|
| Formats | SafeTensors (BF16), GGUF, MLX |
|
|
|
|
## Throughput
|
|
|
|
| Platform | Speed |
|
|
|----------|-------|
|
|
| M3 Max (MLX) | 44,000 tokens/sec |
|
|
| RTX 4090 (GGUF Q4) | 35,000 tokens/sec |
|
|
| iPhone 15 Pro | 8,000 tokens/sec |
|
|
| Raspberry Pi 5 | 2,500 tokens/sec |
|
|
|
|
## Memory Usage
|
|
|
|
| Format | VRAM/RAM |
|
|
|--------|----------|
|
|
| Q2_K | 0.3 GB |
|
|
| Q4_K_M | 0.4 GB |
|
|
| Q8_0 | 0.7 GB |
|
|
| F16 | 1.2 GB |
|
|
|
|
## Quick Start
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
"zenlm/zen-nano-0.6b",
|
|
torch_dtype="auto",
|
|
device_map="auto"
|
|
)
|
|
tokenizer = AutoTokenizer.from_pretrained("zenlm/zen-nano-0.6b")
|
|
|
|
messages = [{"role": "user", "content": "Hello!"}]
|
|
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
|
outputs = model.generate(**inputs, max_new_tokens=100)
|
|
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
|
```
|
|
|
|
## Use Cases
|
|
|
|
- Edge AI applications and embedded systems
|
|
- Mobile chatbots and offline AI assistants
|
|
- IoT device intelligence
|
|
- Real-time inference on consumer hardware
|
|
- Resource-constrained environments
|
|
|
|
## Formats
|
|
|
|
- **SafeTensors** (BF16) — native PyTorch/Transformers
|
|
- **MLX** — optimized for Apple Silicon (M1/M2/M3)
|
|
- **GGUF** — Q2_K, Q4_K_M, Q5_K_M, Q8_0, F16 for llama.cpp
|
|
|
|
### MLX (Apple Silicon)
|
|
|
|
```python
|
|
from mlx_lm import load, generate
|
|
|
|
model, tokenizer = load("zenlm/zen-nano-0.6b")
|
|
response = generate(model, tokenizer, prompt="Hello!", max_tokens=100)
|
|
```
|
|
|
|
### GGUF (llama.cpp)
|
|
|
|
```bash
|
|
./llama-cli -m zen-nano-0.6b-Q4_K_M.gguf -p "Hello!" -n 100
|
|
```
|
|
|
|
## Benchmarks
|
|
|
|
| Task | Score |
|
|
|------|-------|
|
|
| MMLU | 35.2% (5-shot) |
|
|
| GSM8K | 28.4% (8-shot CoT) |
|
|
| HumanEval | 24.1% (pass@1) |
|
|
| MATH | 18.7% (4-shot) |
|
|
|
|
## Get Started
|
|
|
|
- [HuggingFace](https://huggingface.co/zenlm/zen-nano-0.6b)
|
|
- [GitHub](https://github.com/zenlm/zen-nano)
|