mirror of
https://github.com/zoo-apps/gym.git
synced 2026-10-30 14:57:09 +00:00
Wing LianandGitHub
a1bdf1b0c9
* use warmup_ratio as a better default than warmup steps since it's data dependent * replace remainder of warmup_steps
Qwen
TODO
Qwen2 MoE
✅ multipack ✅ qwen2_moe 4-bit QLoRA ✅ qwen2_moe 16-bit LoRA ❓ qwen2_moe 8-bit LoRA