| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-0.5B-Instruct |
| library_name: gguf |
| language: |
| - en |
| tags: |
| - lora |
| - gguf |
| - llama.cpp |
| - wllama |
| - ffmpeg |
| - git |
| - imagemagick |
| - modelxare |
| --- |
| |
| # Command add-ons for Qwen2.5 0.5B |
|
|
| Three small LoRA adapters that turn a plain-language request into a command line, one tool each. |
| They run in the browser with [wllama](https://github.com/ngxson/wllama) or natively with llama.cpp, |
| on top of one shared base model, and are published as signed releases in |
| [modelXare](https://modelxare.com) by weXare. |
|
|
| | File | Tool | Example request | Answer | |
| |---|---|---|---| |
| | `git.gguf` | git | undo my last commit but keep the changes | `git reset HEAD~1` | |
| | `ffmpeg.gguf` | ffmpeg | extract the audio from talk.mp4 as an mp3 | `ffmpeg -i talk.mp4 -vn talk.mp3` | |
| | `imagemagick.gguf` | ImageMagick | make a 200px thumbnail of photo.jpg | `magick photo.jpg -thumbnail 200x output.jpg` | |
|
|
| Each file is 5.9 MB (f16, rank 8, 16 layers). |
|
|
| ## Base model |
|
|
| They apply only to |
| [`qwen2.5-0.5b-instruct-q4_k_m.gguf`](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct-GGUF/blob/9217f5db79a29953eb74d5343926648285ec7e67/qwen2.5-0.5b-instruct-q4_k_m.gguf) |
| (491 MB, SHA-256 `74a4da8c9fdbcd15bd1f6d01d621410d31c6fc00986f5eb687824e7b93d7a9db`). One copy of the |
| base serves all three, and switching between them needs no reload. |
|
|
| ## Prompt |
|
|
| Each adapter was trained with one fixed system prompt and the request as the user message, using the |
| base model's own chat template. Use the same system prompt; the modelXare releases carry it in their |
| `command-generation` operation. ffmpeg and ImageMagick answer `UNSUPPORTED` / `unsupported_request` |
| when the tool can't do what was asked. |
|
|
| ```sh |
| llama-cli -m qwen2.5-0.5b-instruct-q4_k_m.gguf --lora git.gguf -cnv \ |
| -sys "You are a Git command translator. Read the user's natural-language request and reply with ONLY the exact git CLI command(s) that fulfill it. Use a single line when one command is enough; use newlines for a short sequence. Do not wrap commands in markdown, quotes, or explanation." |
| ``` |
|
|
| ## Training |
|
|
| LoRA (QLoRA on `mlx-community/Qwen2.5-0.5B-Instruct-4bit`) with mlx-lm: rank 8, 16 layers, learning |
| rate 1e-4, batch 4, completion-only loss, early stopping on validation loss. Converted to PEFT layout |
| and then to GGUF with llama.cpp's `convert_lora_to_gguf.py`. |
|
|
| | Adapter | Training rows | Best checkpoint (val loss) | |
| |---|---|---| |
| | git | 4,592 | 0.047 | |
| | ffmpeg | 2,504 | 0.169 (iter 300) | |
| | ImageMagick | 2,505 | 0.100 | |
|
|
| The git data comes from templates with slot pools and typing noise. The ffmpeg and ImageMagick data |
| was generated by a larger language model from a task description, then checked by script. |
|
|
| ## Evaluation |
|
|
| 211 held-out requests per tool, greedy decoding, through native llama.cpp (node-llama-cpp) with the |
| q4_K_M base. Many test rows give an output file name the request never mentioned, so the second |
| column also accepts a different output file name when the request names none. |
|
|
| | Adapter | Exact match | Exact match, any unnamed output name | Base model alone (exact) | |
| |---|---|---|---| |
| | git | 83% | 86% | 30% (first 60 rows) | |
| | ImageMagick | 65% | 80% | 2% (first 60 rows) | |
| | ffmpeg | 35% | 68% | 0% (first 60 rows) | |
|
|
| ## Limits |
|
|
| - Commands are suggestions: read them before running them. They can be wrong, and some (`git reset |
| --hard`, `git push --force`, overwriting output files) destroy data. |
| - English requests only. One tool per adapter; requests that need several tools are out of scope. |
| - Unusual flags, filters and long multi-step requests are the weakest cases, ffmpeg most of all. |
| Known misses: "discard my changes to README.md" gave `git checkout -- . README.md` (which discards |
| every file's changes), and trimming or GIF requests often get the wrong ffmpeg flags. |
| - ImageMagick answers sometimes write over the input file (`magick photo.jpg -rotate 90 photo.jpg`). |
|
|
| ## License |
|
|
| Apache-2.0, like the base model. Qwen2.5-0.5B-Instruct is by the Qwen Team, Alibaba Cloud. |
|
|
