| --- |
| base_model: Qwen/Qwen2.5-Coder-14B |
| library_name: peft |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| tags: |
| - lora |
| - code-generation |
| - sft |
| - plan-then-code |
| --- |
| |
| # plan-then-code-lora-14b |
|
|
| LoRA adapter for [Qwen/Qwen2.5-Coder-14B](https://huggingface.co/Qwen/Qwen2.5-Coder-14B), trained to **"plan before coding"** — first emit an I/O contract + 3–8 steps of pseudocode, then the code. |
|
|
| On 324 held-out algorithm problems, pass@1 improves from **36.7% to 45.7% (+9.0pp, McNemar p<0.001)**, compile rate from 99.4% to **100%**. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| base = "Qwen/Qwen2.5-Coder-14B" |
| adapter = "byesher/plan-then-code-lora-14b" # or a local adapter path |
| |
| tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True) |
| model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True) |
| model = PeftModel.from_pretrained(model, adapter) |
| |
| messages = [{"role": "user", "content": "【题目】\n{题目}\n\n请先写出分步伪代码（算法计划），再写出完整可运行的 Python 程序。"}] |
| text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tok(text, return_tensors="pt").to(model.device) |
| out = model.generate(**inputs, max_new_tokens=2048) |
| print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ## Results |
|
|
| ### Three metrics (324 held-out algorithm problems) |
|
|
| | Setting | pass@1 | compile | case_pass | |
| |---|---|---|---| |
| | Bare Coder, chat prompt | 6.2% (20/324) | 26.2% (85/324) | 6.6% (256/3894) | |
| | Bare Coder, comment-completion | 36.7% (119/324) | 99.4% (322/324) | 34.1% (1327/3894) | |
| | **SFT (plan-then-code)** | **45.7% (148/324)** | **100% (324/324)** | **36.7% (1429/3894)** | |
| |
| Relative to the "comment-completion" baseline, SFT improves pass@1 by **+9.0pp** (McNemar test, p < 0.001). |
| |
| ### Failure breakdown (324 problems, by failure reason) |
| |
| | Setting | Passed | Compile error | Wrong output (mismatch) | Runtime error | Timeout | |
| |---|---|---|---|---|---| |
| | Bare Coder, chat prompt | 20 | 239 | 41 | 22 | 2 | |
| | Bare Coder, comment-completion | 119 | 2 | 172 | 26 | 5 | |
| | **SFT (plan-then-code)** | **148** | **0** | 167 | 8 | 1 | |
| |
| ## Training |
| |
| - **Base model**: Qwen2.5-Coder-14B (base) |
| - **LoRA**: r=8, alpha=16, lr=1e-4, 1 epoch, bf16 |
| - **Training data**: 2,372 samples (closed-loop verification, 81.4% acceptance) |
| - **Generation**: single-turn, one call per problem (plan + code together) |
| |
| ## Repository |
| |
| - Code & full documentation: [byesher/plan-then-code](https://github.com/byesher/plan-then-code) |
| |
| ## Framework versions |
| |
| - PEFT 0.21.2 |
| |
