mirror of
https://github.com/zenlm/zen-router.git
synced 2026-10-30 14:57:44 +00:00
zeekay
f888212afb
Reads a prompt, picks the best model. 0.6B encoder-pooled router with task/route/feature heads; SFT from eval profiles + usage ledger, GRPO on quality - lambda*cost - mu*latency; serves via hanzo-engine and plugs the hanzo-router Classifier and enso Featurizer seams. Apache-2.0.
8 lines
418 B
Bash
Executable File
8 lines
418 B
Bash
Executable File
#!/usr/bin/env bash
|
|
# Quantize trained zen-router to GGUF (Q4_K_M) and MLX.
|
|
set -euo pipefail
|
|
MODEL_DIR="${1:-out/zen-router}"
|
|
python ../llama.cpp/convert_hf_to_gguf.py "$MODEL_DIR" --outfile "$MODEL_DIR/zen-router-f16.gguf"
|
|
../llama.cpp/build/bin/llama-quantize "$MODEL_DIR/zen-router-f16.gguf" "$MODEL_DIR/zen-router-Q4_K_M.gguf" Q4_K_M
|
|
python -m mlx_lm convert --hf-path "$MODEL_DIR" --mlx-path "$MODEL_DIR/mlx" -q
|