bimal2004

Update README.md

687b8c2 verified 8 days ago

3.96 kB

---
base_model: Qwen/Qwen2.5-1.5B-Instruct
library_name: peft
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- text-to-sql
- sql
- grpo
- reinforcement-learning
- lora
- trl
- unsloth
---
# SQL-GRPO 1.5B

A LoRA adapter for **Qwen2.5-1.5B-Instruct**, trained with **GRPO** (Group Relative Policy Optimization) to turn natural-language questions into SQLite queries. The reward comes from **executing the generated SQL** against the real database: no human-labeled feedback, the database is the judge.

🚀 **[Live demo](https://huggingface.co/spaces/bimal2004/sql-grpo-demo)** · 💻 **[Code](https://github.com/bimal1023/sql-grpo)**

## Results

Spider dev set (1,034 questions on databases not seen during training), execution accuracy:

| Model | Accuracy | Valid SQL | Correct format |
| --- | --- | --- | --- |
| Qwen2.5-1.5B-Instruct (untrained) | 40.72% | 58.41% | 76.89% |
| **This model (GRPO)** | **57.74%** | **82.40%** | **100.00%** |
| gpt-4.1-mini (prompted, for reference) | 74.37% | 98.74% | 100.00% |

GRPO closed about half the gap to gpt-4.1-mini, with gains on every question type: aggregation +18.3 points, GROUP BY +15.5, nested queries +11.9, JOINs +10.0.

## How to use

```python
import re
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "bimal2004/sql-grpo-1.5b")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
schema = """CREATE TABLE singer (
  Singer_ID int, Name text, Country text, Age int,
  PRIMARY KEY (Singer_ID)
)"""
question = "How many singers are from France?"
messages = [
    {"role": "system", "content": (
        "You are a SQLite expert. Given a database schema and a question, "
        "write one SQLite query that answers it. "
        "Put only the query inside <sql></sql> tags.")},
    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
answer = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
sql = re.search(r"<sql>(.*?)</sql>", answer, re.DOTALL).group(1).strip()
print(sql)
```

**Prompt format matters:** the model was trained with exactly this system prompt, with the schema given as `CREATE TABLE` statements, and answers inside `<sql></sql>` tags.

## Training details

- **Method:** GRPO via Hugging Face TRL `GRPOTrainer` + Unsloth, LoRA rank 32 (about 3% of parameters trained)
- **Data:** Spider train split (6,289 questions after filtering very long schemas)
- **Reward:** +1.0 if the query returns the same rows as the gold SQL, +0.2 if it runs without error, +0.1 for correct `<sql>` formatting, −0.1 for overly long output. Queries run on a read-only connection with a 2-second timeout.
- **Setup:** 400 steps, 8 sampled answers per question, learning rate 5e-6, on a single free Kaggle T4 GPU

## Limitations

- Trained and evaluated only on **Spider** (SQLite); performance on other SQL dialects or very large real-world schemas is untested.
- The most common remaining error is **schema linking**: referencing columns that don't exist (about a third of failures). JOINs and nested queries remain the hardest cases (~40% accuracy).
- Some correct answers on borderline questions can flip under different numerical precision or batching. Evaluate on many questions, not single examples.
- Always review generated SQL before running it on important data.

## Credits

Built by [Bimal Kumal](https://bimalkumal7.com.np). Base model: [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct). Data: [Spider](https://yale-lily.github.io/spider) (Yu et al., 2018).

	---
	base_model: Qwen/Qwen2.5-1.5B-Instruct
	library_name: peft
	license: apache-2.0
	language:
	- en
	pipeline_tag: text-generation
	tags:
	- text-to-sql
	- sql
	- grpo
	- reinforcement-learning
	- lora
	- trl
	- unsloth
	---

	# SQL-GRPO 1.5B

	A LoRA adapter for Qwen2.5-1.5B-Instruct, trained with GRPO (Group Relative Policy Optimization) to turn natural-language questions into SQLite queries. The reward comes from executing the generated SQL against the real database: no human-labeled feedback, the database is the judge.

	🚀 [Live demo](https://huggingface.co/spaces/bimal2004/sql-grpo-demo) · 💻 [Code](https://github.com/bimal1023/sql-grpo)

	## Results

	Spider dev set (1,034 questions on databases not seen during training), execution accuracy:

	\| Model \| Accuracy \| Valid SQL \| Correct format \|
	\| --- \| --- \| --- \| --- \|
	\| Qwen2.5-1.5B-Instruct (untrained) \| 40.72% \| 58.41% \| 76.89% \|
	\| This model (GRPO) \| 57.74% \| 82.40% \| 100.00% \|
	\| gpt-4.1-mini (prompted, for reference) \| 74.37% \| 98.74% \| 100.00% \|

	GRPO closed about half the gap to gpt-4.1-mini, with gains on every question type: aggregation +18.3 points, GROUP BY +15.5, nested queries +11.9, JOINs +10.0.

	## How to use

	```python
	import re
	from transformers import AutoModelForCausalLM, AutoTokenizer
	from peft import PeftModel

	base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype="auto")
	model = PeftModel.from_pretrained(base, "bimal2004/sql-grpo-1.5b")
	tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")

	schema = """CREATE TABLE singer (
	Singer_ID int, Name text, Country text, Age int,
	PRIMARY KEY (Singer_ID)
	)"""
	question = "How many singers are from France?"

	messages = [
	{"role": "system", "content": (
	"You are a SQLite expert. Given a database schema and a question, "
	"write one SQLite query that answers it. "
	"Put only the query inside <sql></sql> tags.")},
	{"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
	]
	text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
	inputs = tokenizer(text, return_tensors="pt").to(model.device)
	out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
	answer = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

	sql = re.search(r"<sql>(.*?)</sql>", answer, re.DOTALL).group(1).strip()
	print(sql)
	```

	Prompt format matters: the model was trained with exactly this system prompt, with the schema given as `CREATE TABLE` statements, and answers inside `<sql></sql>` tags.

	## Training details

	- Method: GRPO via Hugging Face TRL `GRPOTrainer` + Unsloth, LoRA rank 32 (about 3% of parameters trained)
	- Data: Spider train split (6,289 questions after filtering very long schemas)
	- Reward: +1.0 if the query returns the same rows as the gold SQL, +0.2 if it runs without error, +0.1 for correct `<sql>` formatting, −0.1 for overly long output. Queries run on a read-only connection with a 2-second timeout.
	- Setup: 400 steps, 8 sampled answers per question, learning rate 5e-6, on a single free Kaggle T4 GPU

	## Limitations

	- Trained and evaluated only on Spider (SQLite); performance on other SQL dialects or very large real-world schemas is untested.
	- The most common remaining error is schema linking: referencing columns that don't exist (about a third of failures). JOINs and nested queries remain the hardest cases (~40% accuracy).
	- Some correct answers on borderline questions can flip under different numerical precision or batching. Evaluate on many questions, not single examples.
	- Always review generated SQL before running it on important data.

	## Credits

	Built by [Bimal Kumal](https://bimalkumal7.com.np). Base model: [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct). Data: [Spider](https://yale-lily.github.io/spider) (Yu et al., 2018).