Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>Fine tuning LLM study - Experimental Setup</title> | |
| <style> | |
| body { | |
| font-family: Arial, sans-serif; | |
| line-height: 1.6; | |
| margin: 20px; | |
| } | |
| h1, h2 { | |
| color: #333; | |
| } | |
| h2 { | |
| margin-top: 30px; | |
| } | |
| ul { | |
| list-style-type: disc; | |
| margin-left: 20px; | |
| } | |
| p { | |
| margin-bottom: 15px; | |
| } | |
| table{ | |
| border-collapse: collapse; | |
| width: 95%; | |
| border: 2px solid #2c3e50; | |
| } | |
| tr{ | |
| border-bottom: 2px solid #b60e0e; | |
| } | |
| td{ | |
| width: 15%; | |
| vertical-align: top; | |
| border: 2px solid #3498db; | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <h2>The very basics</h2> | |
| <p> | |
| There are various open source libraries available to fine tune LLM's. In the following experimental setup, we will utilize <br> | |
| mlx_lm, its blazing fast on a mac, as it uses native apple silicon hardware. Under the PEFT (Parameter Efficient Fine Tuning) bucket, | |
| it offers LORA, DORA and Full fine tuning options. (Pls refert to --fine-tune-type {lora,dora,full} option). | |
| Several other libraries like Unsloth, Hugging face transformers, Llama-factory etc can be used as well.<br> | |
| <b>Absolute Golden Rule of Thumb:</b> Decide to finetune a LLM on how model behaves, use RAG on what it knows. | |
| </p> | |
| <h2>So, here is a very basic run down of the three fine tuning types</h2> | |
| <p> | |
| <ol> | |
| <li>lora (Low-Rank Adaptation)Status: This is the default mode.Behavior: It injects small low-rank adapter matrices while freezing the base model. If you pass a quantized base model (e.g., 4-bit), mlx_lm will automatically perform QLoRA instead of standard LoRA to save massive amounts of memory.</li> | |
| <li>dora (Weight-Decomposition Low-Rank Adaptation)Status: Fully supported PEFT method in mlx_lm.Behavior: DoRA is an advanced PEFT styled fine-tuning method that decomposes the weights into magnitude and direction components. It trains similarly to LoRA but mathematically matches full fine-tuning performance much more closely.</li> | |
| <li>full (Full Fine-Tuning / FFT)Status: Supported.Behavior: This is not a PEFT method. It unfreezes all parameters of the model and modifies every single weight directly. It requires vastly more Unified Memory (RAM) on your Mac, but it completely eliminates the restrictions of adapters when trying to teach the model deep, complex factual knowledge.</li> | |
| </ol> | |
| </p> | |
| <hr> | |
| <p> | |
| <ol> | |
| <li> | |
| Create the following data sets for training the model with data.<br> | |
| <pre>data | |
| ├── train.jsonl | |
| └── valid.jsonl | |
| </pre> | |
| </li> | |
| <li> | |
| Choose a type of fine tuning needed, LORA, DORA or FULL. Execute the following or create a script for it.<br> | |
| <pre> | |
| mlx_lm.lora \ | |
| --model mlx-community/Llama-3.2-3B-Instruct-4bit \ | |
| --data data \ | |
| --train \ | |
| --iters 100 \ | |
| --batch-size 2 \ | |
| --adapter-path adapters | |
| </pre> | |
| Give the data directory where training and validation data is available and give the path where new adapters needs to be created. | |
| </li> | |
| <li> | |
| If everything goes well, you should see something like this. | |
| <pre> | |
| Loading pretrained model | |
| Fetching 6 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 9245.34it/s] | |
| Download complete: : | 0.00B | |
| Reconstruction complete: | | 0.00B / 0.00B | |
| Loading datasets | |
| Training | |
| Trainable parameters: 0.216% (6.947M/3212.750M) | |
| Starting training..., iters: 100 | |
| Calculating loss...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3.06it/s] | |
| Iter 1: Val loss 4.726, Val took 0.334s | |
| Iter 10: Train loss 3.008, Learning Rate 1.000e-05, It/sec 1.737, Tokens/sec 147.120, Trained Tokens 847, Peak mem 2.565 GB | |
| Iter 20: Train loss 0.978, Learning Rate 1.000e-05, It/sec 1.732, Tokens/sec 147.028, Trained Tokens 1696, Peak mem 2.565 GB | |
| Iter 30: Train loss 0.626, Learning Rate 1.000e-05, It/sec 1.721, Tokens/sec 145.270, Trained Tokens 2540, Peak mem 2.565 GB | |
| Iter 40: Train loss 0.551, Learning Rate 1.000e-05, It/sec 1.704, Tokens/sec 144.336, Trained Tokens 3387, Peak mem 2.565 GB | |
| Iter 50: Train loss 0.472, Learning Rate 1.000e-05, It/sec 1.686, Tokens/sec 143.173, Trained Tokens 4236, Peak mem 2.565 GB | |
| Iter 60: Train loss 0.347, Learning Rate 1.000e-05, It/sec 1.669, Tokens/sec 140.866, Trained Tokens 5080, Peak mem 2.565 GB | |
| Iter 70: Train loss 0.229, Learning Rate 1.000e-05, It/sec 1.669, Tokens/sec 141.694, Trained Tokens 5929, Peak mem 2.565 GB | |
| Iter 80: Train loss 0.148, Learning Rate 1.000e-05, It/sec 1.684, Tokens/sec 142.670, Trained Tokens 6776, Peak mem 2.565 GB | |
| Iter 90: Train loss 0.093, Learning Rate 1.000e-05, It/sec 1.679, Tokens/sec 141.744, Trained Tokens 7620, Peak mem 2.565 GB | |
| Calculating loss...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3.04it/s] | |
| Iter 100: Val loss 2.496, Val took 0.332s | |
| Iter 100: Train loss 0.085, Learning Rate 1.000e-05, It/sec 1.670, Tokens/sec 141.798, Trained Tokens 8469, Peak mem 2.565 GB | |
| Iter 100: Saved adapter weights to adapters/adapters.safetensors and adapters/0000100_adapters.safetensors. | |
| Saved final weights to adapters/adapters.safetensors. | |
| </pre> | |
| </li> | |
| </ol> | |
| </p> | |
| </body> | |
| </html> |