---
license: llama3.2
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: peft
pipeline_tag: text-generation
language:
- en
datasets:
- gretelai/synthetic_text_to_sql
tags:
- text-to-sql
- sql
- qlora
- lora
- peft
- llama
- llama-3.2
---

# Llama-3.2-3B-Text2SQL-QLoRA

**Built with Llama.** A LoRA adapter for Llama-3.2-3B-Instruct that turns a question and a database schema into a SQLite query. It was trained with QLoRA (4-bit) on 600 examples in about 16 minutes on a free Colab T4 GPU.

- Code, training scripts and evaluation: https://github.com/Sam-2610/LocalText2SQL
- This repository contains only the adapter (about 97 MB). You need the base model to use it.

## Results
Scored by **execution accuracy**: the correct and the predicted query are both run on a SQLite database built from the schema and sample rows, and the returned rows are compared. Test set: 150 questions from the dataset's test split (50 easy, 50 medium, 50 hard), none of which were used for training.

| | Untuned model | This adapter |
|---|---|---|
| **Execution accuracy (all)** | 53% (79/150) | **69% (104/150)** |
| Easy | 78% | 86% |
| Medium | 50% | 78% |
| Hard | 30% | 44% |
| Valid SQL | 87% | 99% |

Of the questions where the two models differ, the adapter fixed 28 and broke 3. Reloading this published adapter in a fresh session reproduced the 69% result exactly.

## How to use
Tested with transformers 5.22.2, peft 0.21.2 and trl 1.17.1 on a Colab T4. 4-bit loading needs an NVIDIA GPU.

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

adapter = "Sam-2610/Llama-3.2-3B-Text2SQL-QLoRA"
base_id = "unsloth/Llama-3.2-3B-Instruct"  # ungated mirror; meta-llama/Llama-3.2-3B-Instruct also works

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)
tokenizer = AutoTokenizer.from_pretrained(adapter)
model = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb,
                                             device_map="auto", dtype=torch.float16)
model = PeftModel.from_pretrained(model, adapter)
model.eval()

schema = ("CREATE TABLE employees (id INT, name TEXT, department TEXT, salary REAL); "
          "INSERT INTO employees VALUES (1,'Asha','Sales',50000),(2,'Ravi','Sales',60000),(3,'Meera','IT',70000);")
question = "Who earns the most?"

messages = [
    {"role": "system", "content": "You are an expert in SQLite. Given a database schema, write one SQLite query "
                                  "that answers the question. Output only the SQL query."},
    {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
Use the same system prompt and message format as above, because the adapter was trained with it.

## Training
- **Data:** 600 examples (200 easy, 200 medium, 200 hard) from [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql). Every example runs on SQLite and returns real rows. Easy is basic SQL, medium is aggregation and single joins, hard is subqueries, window functions, multiple joins and set operations.
- **Method:** QLoRA. 4-bit NF4 base model with double quantization, LoRA r=16, alpha 32, dropout 0.05 on the attention and MLP projection layers (24.3M trainable parameters, 0.75%). Only the SQL answer contributes to the loss.
- **Settings:** 3 epochs, learning rate 2e-4 with a cosine schedule, effective batch size 16, fp16, max length 1024.
- **Run:** 15 min 32 s on a free Colab T4. Training loss fell from 0.51 to 0.08.

## Limitations
- The test questions come from the same synthetic dataset as the training data, so the results show the model learned this style of question. They do not show it is a general SQL expert. Public benchmarks such as Spider or BIRD would be a harder test.
- Hard questions are still only 44% correct, and each difficulty group has only 50 test questions, so the per-group numbers are rough.
- Trained and tested for SQLite and short schemas. It can produce wrong or invalid queries, so check generated SQL before running it, and never run it with write access on data you care about.
- Not evaluated for any use outside this demo.

## License and credits
- Built with Llama. Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved. Use of this adapter is subject to that license and Meta's Acceptable Use Policy.
- Training data: gretelai/synthetic_text_to_sql by Gretel.ai, licensed under Apache 2.0.