smart-commit-1.5b-v0.1

A fine-tune of Qwen/Qwen2.5-Coder-1.5B-Instruct that writes git commit messages in the repository's style, for smart-commit, a CLI that runs a small model locally (Intel NPU, GPU or CPU). Version 0.1: the first published model, while the training recipe still changes.

It is trained on smart-commit's own prompt, byte for byte: a system prompt with the rules (subject of at most 72 columns, imperative mood, Conventional Commits when the repository uses them) and a user prompt with the changed files, the compacted diff and up to 5 recent subjects of the repository. Other prompts are untested.

Files

File	What	SHA-256
model.safetensors (+ config and tokenizer)	merged weights, bf16	003e62ae5e309d9eafd20e6a2b0d57b186043da0ee5c17384563c68752864a60
smart-commit-1.5b-v0.1-q4_k_m.gguf	Q4_K_M for llama.cpp, 986 MB	7b8f456ff62c050e43ebdb90593960990e490e1dd7c2635652a82929a040d9f7

Use with smart-commit

hf download LucasBRT/smart-commit-1.5b-v0.1 smart-commit-1.5b-v0.1-q4_k_m.gguf --local-dir ~/models
SMART_COMMIT_GGUF=~/models/smart-commit-1.5b-v0.1-q4_k_m.gguf smart-commit

It runs on smart-commit's llama.cpp devices (CPU and Vulkan GPUs). For the OpenVINO devices (Intel NPU and GPU), the safetensors can be exported with smart-commit's export scripts; that export is not validated yet for this model.

Evaluation

eval-v1: 500 commits from the test split of CommitChronicle (repositories never seen in training), 250 following Conventional Commits and 250 free-form; eval-short: 197 commits from young repositories (1–5 earlier subjects). Greedy decoding, smart-commit at db389c2, Vulkan, with its retry and file-list fallback.

Model	type_match	scope_known	first_try	fallback	word_f1	word_f1 (free)	fallback (young repos)
Qwen2.5-Coder-1.5B-Instruct, upstream Q4_K_M	0.456	0.580	0.908	0.068	0.245	0.143	0.102
same, re-quantized like this model (control)	0.408	0.516	0.928	0.004	0.245	0.139	0.051
smart-commit-1.5b-v0.1	0.644	0.852	0.948	0.016	0.362	0.257	0.030

• type_match: same Conventional type as the real commit. The base model answers fix to most changes; answering fix always would score 0.41.
• scope_known: no scope, or one the repository already used.
• first_try: the first reply was usable; fallback: no reply was, and smart-commit wrote the message from the file list.
• word_f1: word overlap with the real subject, a rough proxy for content.

All the numbers, the learning curve behind the choice of 40k examples and the method are in training/HYPOTHESES.md (H3). For reference, 795 s for the 500 commits of eval-v1 on an Intel Arc iGPU (Core Ultra 5 225H), retries included. See also smart-commit-0.5b-v0.1: the 0.5B version matches it on type and scope at about 2.5× the speed.

Training

• Data: 39,985 commits from the train split of CommitChronicle (revision 5fd076e), each with at least 10 earlier subjects of its repository; prompts built by smart-commit itself, the real subject as the target (examples file sha256 7495c85514a53985…).
• Method: LoRA r=16, alpha=32, dropout 0.05 on all linear layers, lr 2e-4 cosine, warmup 3%, batch 4 × 4, 1 epoch, seed 0, loss on the reply only; 4-bit NF4 base with fp16 compute (QLoRA) on a Kaggle Tesla T4, 394 minutes. torch 2.11.0, transformers 5.20.1, peft 0.21.2.
• Export: merged into the bf16 base and quantized to Q4_K_M with llama.cpp b11381 (training/export_gguf.py --merged).

Training on a GPU is not bit-for-bit reproducible; generation is (greedy). The recipe, the data selection and the scripts are in training/.

Limitations

• English messages only (the training data is English).
• Young repositories: with fewer than 10 earlier commits it sometimes repeats or paraphrases one of them, or the reply is unusable and smart-commit falls back to the file list (0.030 of eval-short).
• It can invent details the diff does not show (a version, a file's purpose), and it does not know when the staged changes are several commits.
• Trained on prompts without smart-commit's map of where each scope is used, added later; with that map in the prompt it was not re-evaluated on eval-v1.

Licence and attribution

The weights are released under Apache-2.0, like the base model. CommitChronicle (JetBrains Research) has no single licence: it gathers commits from repositories under MIT, Apache-2.0 and BSD-3-Clause, and asks that uses follow their terms, attribution included. The dataset records the repository and licence of every commit (repo and license fields), which is the provenance of the training data.