Files
engine/hanzo-cli/src/commands/distill.rs
zooqueen 14b2e6dc55 feat(train): hanzo distill — sequence-level teacher->student distillation (Phase 3)
run_distill composes the existing primitives: the teacher (loaded with no
adapters — empty target_modules is exactly the base model) samples
completions for a {"prompt"} JSONL set; the generated {prompt, completion}
dataset is written to out/distill.jsonl (kept as an inspectable artifact);
the student LoRA-fine-tunes on it via run_sft into out/adapter. No new
training machinery.

Also DRYs prompt handling onto TrainingClient (encode_prompt/decode) — the
run loop, the /v1/training sample handler, and distill now share one path.

CLI: `hanzo distill --teacher X --student Y --prompts p.jsonl
[--max-tokens/--temperature/--lora-rank/--lr/--steps/...]`.

Verified end-to-end (SmolLM2-135M as both roles): teacher generated real
completions ("The capital of France is" -> " Paris. It is the largest
city in France and"), dataset written, student trained (loss ~2.04->2.16
over 2 toy steps), PEFT adapter saved. Tests 15/15 (2 new: prompt-set
parsing, JSONL round-trip); clippy -D warnings + rustfmt clean.
2026-07-15 18:07:15 -07:00

63 lines
1.8 KiB
Rust

//! `hanzo distill` — sequence-level distillation: a teacher samples
//! completions for a prompt set, the student LoRA-fine-tunes on them
//! (`hanzo-train`'s run_distill: generate → distill.jsonl → run_sft).
use std::path::PathBuf;
use anyhow::Result;
use hanzo_train::{run_distill, DistillConfig, LoraConfig};
/// Arguments for `hanzo distill`, as parsed by the CLI.
pub struct DistillRunConfig {
pub teacher: String,
pub student: String,
pub prompts: PathBuf,
pub max_tokens: usize,
pub temperature: f64,
pub lora_rank: usize,
pub lora_alpha: Option<f64>,
pub lr: f64,
pub steps: usize,
pub batch_size: usize,
pub out: PathBuf,
pub seed: u64,
pub sample_prompt: Option<String>,
}
pub fn run_distill_cmd(cfg: DistillRunConfig) -> Result<()> {
let lora = LoraConfig {
rank: cfg.lora_rank,
alpha: cfg.lora_alpha.unwrap_or(2.0 * cfg.lora_rank as f64),
target_modules: LoraConfig::default_target_modules(),
};
let report = run_distill(&DistillConfig {
teacher: cfg.teacher,
student: cfg.student,
prompts: cfg.prompts,
max_tokens: cfg.max_tokens,
temperature: cfg.temperature,
lora,
lr: cfg.lr,
steps: cfg.steps,
batch_size: cfg.batch_size,
out: cfg.out,
seed: cfg.seed,
sample_prompt: cfg.sample_prompt,
})?;
println!(
"dataset: {} ({} examples)",
report.dataset.display(),
report.examples
);
println!("adapter: {}", report.sft.adapter.display());
println!(
"trainable params: {} steps: {} final loss: {:.4}",
report.sft.trainable_params, report.sft.steps, report.sft.final_loss
);
if let Some(sample) = report.sft.sample {
println!("sample: {sample}");
}
Ok(())
}