mirror of
https://github.com/zenlm/gym.git
synced 2026-11-04 17:11:44 +00:00
## Major Changes - **Package Rename**: llamafactory → zoo-gym (PyPI) / gym (internal) - **Organization**: Rebranded to Zoo Labs Foundation Inc (501c3 non-profit) - **Complete Module Migration**: All src/llamafactory/ → src/gym/ - **Updated Metadata**: setup.py, pyproject.toml, README.md ## New Features - **Qwen3 Fine-tuning**: Full support with configs/qwen3_finetune.yaml - **Zen Nano Integration**: QLoRA config for 4B edge models - **Enhanced Documentation**: README-HF.md, QUICKSTART.md, LLM.md - **HuggingFace Integration**: .huggingface deployment config - **Training Scripts**: train_qwen3.py, train_qwen3_omni.py ## Technical Improvements - **Import Path Updates**: All internal references updated - **CLI Rename**: gym-cli command (backward compatible) - **Gradio WebUI**: Enhanced interface for training - **Educational Focus**: Democratizing AI education and research ## Testing - ✅ Package imports successfully - ✅ CLI functionality verified - ✅ Core dependencies resolved - 📋 CI tests pending ## Attribution Maintains original Apache 2.0 license with updated copyright to Zoo Labs Foundation Inc. All original LlamaFactory contributors acknowledged in commit history.
6.7 KiB
6.7 KiB
We provide diverse examples about fine-tuning LLMs.
Make sure to execute these commands in the LLaMA-Factory directory.
Table of Contents
- LoRA Fine-Tuning
- QLoRA Fine-Tuning
- Full-Parameter Fine-Tuning
- Merging LoRA Adapters and Quantization
- Inferring LoRA Fine-Tuned Models
- Extras
Use CUDA_VISIBLE_DEVICES (GPU) or ASCEND_RT_VISIBLE_DEVICES (NPU) to choose computing devices.
By default, LLaMA-Factory uses all visible computing devices.
Basic usage:
gym-cli train examples/train_lora/llama3_lora_sft.yaml
Advanced usage:
CUDA_VISIBLE_DEVICES=0,1 gym-cli train examples/train_lora/llama3_lora_sft.yaml \
learning_rate=1e-5 \
logging_steps=1
bash examples/train_lora/llama3_lora_sft.sh
Examples
LoRA Fine-Tuning
(Continuous) Pre-Training
gym-cli train examples/train_lora/llama3_lora_pretrain.yaml
Supervised Fine-Tuning
gym-cli train examples/train_lora/llama3_lora_sft.yaml
Multimodal Supervised Fine-Tuning
gym-cli train examples/train_lora/qwen2_5vl_lora_sft.yaml
DPO/ORPO/SimPO Training
gym-cli train examples/train_lora/llama3_lora_dpo.yaml
Multimodal DPO/ORPO/SimPO Training
gym-cli train examples/train_lora/qwen2_5vl_lora_dpo.yaml
Reward Modeling
gym-cli train examples/train_lora/llama3_lora_reward.yaml
PPO Training
gym-cli train examples/train_lora/llama3_lora_ppo.yaml
KTO Training
gym-cli train examples/train_lora/llama3_lora_kto.yaml
Preprocess Dataset
It is useful for large dataset, use tokenized_path in config to load the preprocessed dataset.
gym-cli train examples/train_lora/llama3_preprocess.yaml
Evaluating on MMLU/CMMLU/C-Eval Benchmarks
gym-cli eval examples/train_lora/llama3_lora_eval.yaml
Supervised Fine-Tuning on Multiple Nodes
FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 gym-cli train examples/train_lora/llama3_lora_sft.yaml
FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 gym-cli train examples/train_lora/llama3_lora_sft.yaml
Supervised Fine-Tuning with DeepSpeed ZeRO-3 (Weight Sharding)
FORCE_TORCHRUN=1 gym-cli train examples/train_lora/llama3_lora_sft_ds3.yaml
Supervised Fine-Tuning with Ray on 4 GPUs
USE_RAY=1 gym-cli train examples/train_lora/llama3_lora_sft_ray.yaml
QLoRA Fine-Tuning
Supervised Fine-Tuning with 4/8-bit Bitsandbytes/HQQ/EETQ Quantization (Recommended)
gym-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml
Supervised Fine-Tuning with 4-bit Bitsandbytes Quantization on Ascend NPU
gym-cli train examples/train_qlora/llama3_lora_sft_bnb_npu.yaml
Supervised Fine-Tuning with 4/8-bit GPTQ Quantization
gym-cli train examples/train_qlora/llama3_lora_sft_gptq.yaml
Supervised Fine-Tuning with 4-bit AWQ Quantization
gym-cli train examples/train_qlora/llama3_lora_sft_awq.yaml
Supervised Fine-Tuning with 2-bit AQLM Quantization
gym-cli train examples/train_qlora/llama3_lora_sft_aqlm.yaml
Full-Parameter Fine-Tuning
Supervised Fine-Tuning on Single Node
FORCE_TORCHRUN=1 gym-cli train examples/train_full/llama3_full_sft.yaml
Supervised Fine-Tuning on Multiple Nodes
FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 gym-cli train examples/train_full/llama3_full_sft.yaml
FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 gym-cli train examples/train_full/llama3_full_sft.yaml
Elastic and Fault-Tolerant Supervised Fine-Tuning on Multiple Nodes
To launch an elastic job with MAX_RESTARTS failures retries, run the following on at least MIN_NNODES nodes and at most MAX_NNODES nodes. RDZV_ID should be set as a unique job id (shared by all nodes participating in the job). See also torchrun.
FORCE_TORCHRUN=1 MIN_NNODES=1 MAX_NNODES=3 MAX_RESTARTS=3 RDZV_ID=gym MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 gym-cli train examples/train_full/llama3_full_sft.yaml
Multimodal Supervised Fine-Tuning
FORCE_TORCHRUN=1 gym-cli train examples/train_full/qwen2_5vl_full_sft.yaml
Merging LoRA Adapters and Quantization
Merge LoRA Adapters
Note: DO NOT use quantized model or quantization_bit when merging LoRA adapters.
gym-cli export examples/merge_lora/llama3_lora_sft.yaml
Quantizing Model using AutoGPTQ
gym-cli export examples/merge_lora/llama3_gptq.yaml
Save Ollama modelfile
gym-cli export examples/merge_lora/llama3_full_sft.yaml
Inferring LoRA Fine-Tuned Models
Evaluation using vLLM's Multi-GPU Inference
python scripts/vllm_infer.py --model_name_or_path meta-llama/Meta-Llama-3-8B-Instruct --template llama3 --dataset alpaca_en_demo
python scripts/eval_bleu_rouge.py generated_predictions.jsonl
Use CLI ChatBox
gym-cli chat examples/inference/llama3_lora_sft.yaml
Use Web UI ChatBox
gym-cli webchat examples/inference/llama3_lora_sft.yaml
Launch OpenAI-style API
gym-cli api examples/inference/llama3_lora_sft.yaml
Extras
Full-Parameter Fine-Tuning using GaLore
gym-cli train examples/extras/galore/llama3_full_sft.yaml
Full-Parameter Fine-Tuning using APOLLO
gym-cli train examples/extras/apollo/llama3_full_sft.yaml
Full-Parameter Fine-Tuning using BAdam
gym-cli train examples/extras/badam/llama3_full_sft.yaml
Full-Parameter Fine-Tuning using Adam-mini
gym-cli train examples/extras/adam_mini/qwen2_full_sft.yaml
Full-Parameter Fine-Tuning using Muon
gym-cli train examples/extras/muon/qwen2_full_sft.yaml
LoRA+ Fine-Tuning
gym-cli train examples/extras/loraplus/llama3_lora_sft.yaml
PiSSA Fine-Tuning
gym-cli train examples/extras/pissa/llama3_lora_sft.yaml
Mixture-of-Depths Fine-Tuning
gym-cli train examples/extras/mod/llama3_full_sft.yaml
LLaMA-Pro Fine-Tuning
bash examples/extras/llama_pro/expand.sh
gym-cli train examples/extras/llama_pro/llama3_freeze_sft.yaml
FSDP+QLoRA Fine-Tuning
bash examples/extras/fsdp_qlora/train.sh
OFT Fine-Tuning
gym-cli train examples/extras/oft/llama3_oft_sft.yaml
QOFT Fine-Tuning
gym-cli train examples/extras/qoft/llama3_oft_sft_bnb_npu.yaml