README.md

---
license: cc-by-nc-4.0
base_model: LiquidAI/LFM2.5-1.2B-Thinking
library_name: peft
language:
- en
tags:
- kodcode
- lora
- negative-result
- human-eval-plus
- code
---

# LFM2.5-1.2B-Thinking — KodCode LoRA fine-tune (documented negative result)

LoRA fine-tune (r=32, α=64, attention + MLP modules, 1 epoch, LR 1e-4, bf16, 4096-token packing)
of [LiquidAI/LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) on
[KodCode-V1-SFT-R1](https://huggingface.co/datasets/KodCode/KodCode-V1-SFT-R1) — 70% code-only
`r1_solution` targets, 30% chain-of-thought — formatted for the LFM2.5 chat template
([dataset](https://huggingface.co/datasets/enseven/kodcode-lfm2.5)).

## Evaluation result: the fine-tune made the model worse at code

Sealed HumanEval+ benchmark (128 never-before-used tasks, greedy decoding, one sample per task,
EvalPlus v0.1.10, pinned llama.cpp runtime, restricted offline containers):

| Row | plus pass@1 (scored) | plus rate (absence = 0) | tokens / task | extraction yield |
|---|---:|---:|---:|---:|
| Base model (BF16 GGUF) | **0.5545** (56/101) | **0.4375** | ≈1,654 | 0.789 |
| This adapter (merged BF16) | 0.4048 (51/126) | 0.3984 | ≈148 | 0.984 |
| Q8_0 of this adapter | 0.4173 (53/127) | 0.4141 | ≈128 | 0.992 |
| Q6_K of this adapter | 0.3889 (49/126) | 0.3828 | ≈151 | 0.984 |

Paired on the 99 tasks scored for both models: the fine-tune passes 49 versus the base model's 56
(**−7.1 pp**; 16 gained, 23 lost). "Absence = 0" counts unextractable outputs as failures; the
scorer's pass@1 silently drops them from the denominator, which flatters a broken model — both
views are reported.

**Mechanism.** The fine-tune learned to skip the base model's reasoning traces: output length
collapsed ~11× (≈1,654 → ≈148 tokens per task), which is also why extraction yield and decode
speed improved. Those discarded reasoning tokens were doing the work. A 33-task pilot before the
sealed run showed the same direction.

## Recommendation

- For deployment on HumanEval-style code generation, use
  [the base model](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking).
- GGUF artifacts used in this evaluation (including the base-model baseline):
  [enseven/lfm-2.5-think-code-GGUF](https://huggingface.co/enseven/lfm-2.5-think-code-GGUF).
- This adapter is published as a documented negative result. Do not build on it expecting coding
  gains. The next iteration would preserve the reasoning path and execution-verify training
  targets — see the repository's evaluation notes.

## Provenance

- Evaluated adapter revision: `e01354fecc52e62b9ca86399da10e9e40ebf51e9` (unchanged since training).
- Benchmark: EvalPlus HumanEval+ v0.1.10; frozen prompt `lfm-code-v1`; per-task extraction-failure
  policy (failures score zero, never repaired or retried); all generation and scoring inside
  restricted offline containers.
- Full evidence chain (frozen suites, raw generations, extraction and scoring runs, manifests):
  [GitHub — lfm2.5-finetune-code](https://github.com/ensevengg/lfm2.5-finetune-code)
  (`reports/e3/`, `reports/e4/`).
- Attached summary reports: [pilot-report.json](pilot-report.json),
  [e4-report.json](e4-report.json).

## License

CC BY-NC 4.0 — inherited from the KodCode-V1-SFT-R1 training data. The MIT license in the GitHub
repository applies to its code, not to this adapter.