| cd /root/ai-tuning |
|
|
| cat > README.md <<'EOF' |
| # 🚀 Fine-Tuned DevOps LLM |
|
|
| A complete end-to-end project for fine-tuning **Qwen2.5-3B-Instruct** for DevOps and Cloud engineering tasks using **LoRA**, converting the model to **GGUF**, quantizing it to **Q4_K_M**, and deploying it with **Ollama inside Docker**. |
|
|
| --- |
|
|
| ## 📌 Project Overview |
|
|
| The complete pipeline: |
|
|
| ```text |
| DevOps Dataset |
| ↓ |
| JSONL Dataset |
| ↓ |
| Train / Validation Split |
| ↓ |
| Qwen2.5-3B-Instruct |
| ↓ |
| LoRA Fine-Tuning |
| ↓ |
| LoRA Adapter |
| ↓ |
| Merge Adapter + Base Model |
| ↓ |
| Merged Model |
| ↓ |
| GGUF F16 |
| ↓ |
| Q4_K_M Quantization |
| ↓ |
| Ollama |
| ↓ |
| devops-qwen:latest |
| ↓ |
| Benchmark |
| ↓ |
| RAG + n8n / FastAPI |
| ↓ |
| DevOps AI Assistant |
| |
| |
| |
| |
| |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf auth whoami |
| user: PrithviRana |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf repo create devops-qwen --type model |
| usage: hf <command> [<args>] |
| hf: error: unrecognized arguments: --type model |
| (.venv310) [root@e2e-77-184 ai-tuning]# |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf repo create devops-qwen |
| Successfully created PrithviRana/devops-qwen on the Hub. |
| Your repo is now available at https://huggingface.co/PrithviRana/devops-qwen |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf upload \ |
| > PrithviRana/devops-qwen \ |
| > /root/ai-tuning/qwen-devops-q4_k_m.gguf \ |
| > qwen-devops-q4_k_m.gguf |
| Processing Files (1 / 1) : 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.93GB / 1.93GB, 41.3MB/s |
| New Data Upload : 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 277MB / 277MB, 7.98MB/s |
| ...g/qwen-devops-q4_k_m.gguf: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.93GB / 1.93GB |
| https://huggingface.co/PrithviRana/devops-qwen/blob/main/qwen-devops-q4_k_m.gguf |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf repo-files PrithviRana/devops-qwen |
| usage: hf <command> [<args>] repo-files [-h] {delete} ... |
| hf <command> [<args>] repo-files: error: argument {delete}: invalid choice: 'PrithviRana/devops-qwen' (choose from 'delete') |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf repo-files PrithviRana/devops-qwen |
| usage: hf <command> [<args>] repo-files [-h] {delete} ... |
| hf <command> [<args>] repo-files: error: argument {delete}: invalid choice: 'PrithviRana/devops-qwen' (choose from 'delete') |
| (.venv310) [root@e2e-77-184 ai-tuning]# hf download PrithviRana/devops-qwen \ |
| > qwen-devops-q4_k_m.gguf \ |
| > --local-dir /tmp/devops-test |
| Downloading 'qwen-devops-q4_k_m.gguf' to '/tmp/devops-test/.cache/huggingface/download/PRb-k0RfBjIZ_D61s_zXLFcUvb0=.6740758595869f0abd012e19a8e6ed73124f92a9c7a8435dd6ca17050c0f52d0.incomplete' |
| qwen-devops-q4_k_m.gguf: 0%| | 0.00/1.93G [00:00<?, ?B/s]^CCancellation requested; stopping current tasks. |
| Traceback (most recent call last): |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 633, in xet_get |
| download_files( |
| KeyboardInterrupt |
| |
| During handling of the above exception, another exception occurred: |
| |
| Traceback (most recent call last): |
| File "/root/ai-tuning/.venv310/bin/hf", line 6, in <module> |
| sys.exit(main()) |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/cli/hf.py", line 59, in main |
| service.run() |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/cli/download.py", line 136, in run |
| print(self._download()) # Print path to downloaded files |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/cli/download.py", line 149, in _download |
| return hf_hub_download( |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn |
| return fn(*args, **kwargs) |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 994, in hf_hub_download |
| return _hf_hub_download_to_local_dir( |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1304, in _hf_hub_download_to_local_dir |
| _download_to_tmp_and_move( |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1727, in _download_to_tmp_and_move |
| xet_get( |
| File "/root/ai-tuning/.venv310/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 633, in xet_get |
| download_files( |
| KeyboardInterrupt |
| qwen-devops-q4_k_m.gguf: 0%| |
| |
