metadata
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
library_name: peft
datasets:
- b-mc2/sql-create-context
tags:
- text-to-sql
- qlora
- lora
qwen2.5-coder-3b-text2sql-qlora
QLoRA adapter for Qwen/Qwen2.5-Coder-3B-Instruct, fine-tuned to turn a table schema and a question into a SQL query.
Code and full evaluation: https://github.com/VarshithaChappidi/text2sql-qlora
Results (300 held-out examples)
| Model | Strict exact match | Normalized exact match |
|---|---|---|
| Base, prompted | 0.023 | 0.570 |
| This adapter | 0.767 | 0.773 |
Normalized match unifies quote style, drops wrapping quotes, treats INNER JOIN as JOIN, and ignores AND order. It still rejects some valid SQL written differently, so it is a lower bound.
Usage
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
base_id = "Qwen/Qwen2.5-Coder-3B-Instruct"
tok = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(base, "Varshitha2603/qwen2.5-coder-3b-text2sql-qlora")
schema = "CREATE TABLE singer (name VARCHAR, age INT)"
question = "How many singers are older than 30?"
msgs = [
{"role": "system", "content": "You are a SQL expert. Given a table schema and a question, reply with only the SQL query."},
{"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Training
- Data: 5,000 examples from
b-mc2/sql-create-context(300 other examples held out for testing) - QLoRA: 4-bit NF4, rank 16, alpha 32, dropout 0.05, all attention and MLP projections (29.9M trainable parameters)
- 1 epoch, learning rate 2e-4 with cosine schedule, effective batch size 16, loss on answer tokens only
- Free Colab T4 GPU, about 21 minutes. Loss fell from 0.212 (step 25) to 0.046 (step 300).
Limitations
- Evaluated by exact match, not execution, because the dataset has no table contents.
- Trained and tested only on single-table-style questions from one dataset, plus a few Spider-style joins. Multi-table joins are a known weak spot.
- Gold labels contain ambiguity and typos. Single training run, no variance estimate.
- The adapter inherits the license terms of the base model and the dataset. Check both before commercial use.