mirror of
https://github.com/hanzoai/gym.git
synced 2026-09-30 22:37:59 +00:00
zeekayandHanzo Dev
df832f2104
Rollouts go through a KV cache (prompt prefilled once per group, one decode step per token); cached logits match the full forward to 1e-4. GRPO runs end to end on Metal — a mixed group at step 5 gives reward 0.75, loss −0.032, gradient norm 1.19 — after the clipping reduction stopped going through f64, which Metal does not have. gym synth ports Data Designer's core: sampler, expression and LLM columns (text, structured, judge) generated per row in dependency order against any OpenAI-compatible endpoint, validators that drop or regenerate, jsonl or parquet out. gym trainjob is the Kubeflow TrainJob entry for the hosted fine-tuning API: the broker's env contract becomes a Config; METHOD=lora is served, qlora and full are refused by name. Dockerfile.cuda and image.yml build ghcr.io/hanzoai/gym on the forge runners. 84 tests. Co-authored-by: Hanzo Dev <[email protected]>
39 lines
1.1 KiB
YAML
39 lines
1.1 KiB
YAML
# gym synth examples/synth-qa.yml --preview 3
|
|
model:
|
|
endpoint: https://api.hanzo.ai/v1
|
|
api_key_env: HANZO_API_KEY
|
|
name: zen5
|
|
temperature: 0.8
|
|
concurrency: 8
|
|
rows: 20
|
|
output: out/qa.jsonl
|
|
columns:
|
|
- name: domain
|
|
sampler: category
|
|
values: [finance, health, travel]
|
|
weights: [0.5, 0.3, 0.2]
|
|
- name: topic
|
|
sampler: subcategory
|
|
parent: domain
|
|
values: { finance: [tax, loans], health: [sleep, diet], travel: [visas, flights] }
|
|
- name: question
|
|
llm: text
|
|
prompt: "Write one realistic user question about {{ topic }} in the {{ domain }} domain."
|
|
system: "You write terse, realistic questions."
|
|
- name: answer
|
|
llm: text
|
|
prompt: "Answer this question accurately:\n{{ question }}"
|
|
- name: quality
|
|
llm: judge
|
|
prompt: "Rate the answer to the question."
|
|
rubric: { 5: "correct and complete", 3: "partly correct", 1: "wrong" }
|
|
columns: [question, answer]
|
|
validators:
|
|
- column: answer
|
|
kind: regex
|
|
pattern: "\\S"
|
|
on_fail: retry:2
|
|
- column: quality
|
|
kind: expression
|
|
expression: "quality >= 3"
|