---
base_model: Qwen/Qwen2.5-Coder-14B
library_name: peft
license: apache-2.0
pipeline_tag: text-generation
tags:
- lora
- code-generation
- sft
- plan-then-code
---

# plan-then-code-lora-14b

LoRA adapter for [Qwen/Qwen2.5-Coder-14B](https://huggingface.co/Qwen/Qwen2.5-Coder-14B), trained to **"plan before coding"** — first emit an I/O contract + 3–8 steps of pseudocode, then the code.

On 324 held-out algorithm problems, pass@1 improves from **36.7% to 45.7% (+9.0pp, McNemar p<0.001)**, compile rate from 99.4% to **100%**.

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen2.5-Coder-14B"
adapter = "byesher/plan-then-code-lora-14b"   # or a local adapter path

tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter)

messages = [{"role": "user", "content": "【题目】\n{题目}\n\n请先写出分步伪代码（算法计划），再写出完整可运行的 Python 程序。"}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=2048)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

## Results

### Three metrics (324 held-out algorithm problems)

| Setting | pass@1 | compile | case_pass |
|---|---|---|---|
| Bare Coder, chat prompt | 6.2% (20/324) | 26.2% (85/324) | 6.6% (256/3894) |
| Bare Coder, comment-completion | 36.7% (119/324) | 99.4% (322/324) | 34.1% (1327/3894) |
| **SFT (plan-then-code)** | **45.7% (148/324)** | **100% (324/324)** | **36.7% (1429/3894)** |

Relative to the "comment-completion" baseline, SFT improves pass@1 by **+9.0pp** (McNemar test, p < 0.001).

### Failure breakdown (324 problems, by failure reason)

| Setting | Passed | Compile error | Wrong output (mismatch) | Runtime error | Timeout |
|---|---|---|---|---|---|
| Bare Coder, chat prompt | 20 | 239 | 41 | 22 | 2 |
| Bare Coder, comment-completion | 119 | 2 | 172 | 26 | 5 |
| **SFT (plan-then-code)** | **148** | **0** | 167 | 8 | 1 |

## Training

- **Base model**: Qwen2.5-Coder-14B (base)
- **LoRA**: r=8, alpha=16, lr=1e-4, 1 epoch, bf16
- **Training data**: 2,372 samples (closed-loop verification, 81.4% acceptance)
- **Generation**: single-turn, one call per problem (plan + code together)

## Repository

- Code & full documentation: [byesher/plan-then-code](https://github.com/byesher/plan-then-code)

## Framework versions

- PEFT 0.21.2