mirror of
https://github.com/zenlm/gym.git
synced 2026-11-04 17:11:44 +00:00
Major updates: - Implemented BitDelta 1-bit quantization (10-25× memory reduction) - Added DeltaQuant flexible quantization (INT2/4/8, Binary, Ternary) - Created DeltaSoup for Byzantine-robust community aggregation - Configured accurate Qwen3 models for Zen family (Nano, Eco, Coder, Omni, Next) - Added GRPO/GSPO training algorithms with memory optimizations - Simplified CLI from gym-cli to gym - Updated WebUI with Zoo Labs black monochromatic theme - Integrated Unsloth optimizations for 2-3× speedup - Added comprehensive test suite for quantization methods Zoo Labs Foundation Inc. - Copyright 2025
1.1 KiB
1.1 KiB
Zen Nano Model (0.6B)
Ultra-lightweight model for edge deployment
Overview
Zen Nano is the smallest model in the Zen family, optimized for:
- Edge devices and mobile applications
- Rapid prototyping and experimentation
- Resource-constrained environments
- Educational demonstrations
Model Details
- Base Model: Qwen/Qwen3-0.6B-Instruct
- Parameters: 600M
- Context Length: 512 tokens
- Memory Required: ~2GB VRAM
Training
Quick Start
# Train with GSPO algorithm
gym train models/nano/configs/gspo_training.yaml
# Custom training
gym train models/nano/configs/gspo_training.yaml \
--output_dir saves/nano-custom \
--num_train_epochs 3
Configuration
- Algorithm: GSPO (Group Sequence Policy Optimization)
- LoRA Rank: 4
- Learning Rate: 5e-5
- Batch Size: 8
Performance
- MMLU: 45.2
- HumanEval: 28.3
- MT-Bench: 5.8
- Speed: 120 tokens/sec
Use Cases
✓ IoT devices ✓ Mobile apps ✓ Browser extensions ✓ Quick demos ✓ Learning/teaching
Part of the Zen Model Family by Zoo Labs Foundation