mirror of
https://github.com/hanzo-ml/trainer.git
synced 2026-09-24 00:31:34 +00:00
615e338bf4
* feat(runtimes): Add lora recipe and config in torch.plugin. Signed-off-by: Electronic-Waste <[email protected]> * chore: add --trainer-use-lora flag and related logics. Signed-off-by: Electronic-Waste <[email protected]> * feat(lora): Add support for QLoRA. Signed-off-by: Electronic-Waste <[email protected]> * fix(lora): fix extra rdzv backend in single device mode. Signed-off-by: Electronic-Waste <[email protected]> * test(lora): Add UTs for lora/qlora in trainingruntime. Signed-off-by: Electronic-Waste <[email protected]> * fix(lora): remove extra quote symbol in lora attn module. Signed-off-by: Electronic-Waste <[email protected]> * fix(lint): fix lint error. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtime): check default GPU resources allocated in CTRs. Signed-off-by: Electronic-Waste <[email protected]> * chore(runtime): add getNumProcPerNode. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtime): remove specific vendor for GPU. Signed-off-by: Electronic-Waste <[email protected]> * test(runtime): add UTs for lora validation. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): remove extra flags for torchtune. Signed-off-by: Electronic-Waste <[email protected]> * Update pkg/runtime/framework/plugins/torch/torch.go Co-authored-by: Andrey Velichkevich <[email protected]> Signed-off-by: Shao Wang <[email protected]> * chore(runtime): add extractGPUCountFromRuntime. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): remove filtering args logics. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): use fallbackNumProcPerNode if GPU is set in runtimes. Signed-off-by: Electronic-Waste <[email protected]> * refactor(runtimes): refactor getNumProcPerNode logics. Signed-off-by: Electronic-Waste <[email protected]> * fix(lint): fix lint error. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): call getNumProcPerNode only once. Signed-off-by: Electronic-Waste <[email protected]> * test(runtime): fix UTs for torch. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): add limits to calculation. Signed-off-by: Electronic-Waste <[email protected]> * refactor(runtimes): reorg nppNode related logics. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): fix lints. Signed-off-by: Electronic-Waste <[email protected]> * fix(runtimes): update condition that use nppNode in Limits. Signed-off-by: Electronic-Waste <[email protected]> * refactor(runtimes): add torchtune.go Signed-off-by: Electronic-Waste <[email protected]> * fix(lint): fix lint error. Signed-off-by: Electronic-Waste <[email protected]> * refactor(torch): only reset nppNode when using CPU. Signed-off-by: Electronic-Waste <[email protected]> * chore(runtimes): remove getNumProcPerNodeFromCPU. Signed-off-by: Electronic-Waste <[email protected]> --------- Signed-off-by: Electronic-Waste <[email protected]> Signed-off-by: Shao Wang <[email protected]> Co-authored-by: Andrey Velichkevich <[email protected]>