| --- |
| base_model: Qwen/Qwen2.5-Coder-3B-Instruct |
| library_name: peft |
| datasets: |
| - b-mc2/sql-create-context |
| tags: |
| - text-to-sql |
| - qlora |
| - lora |
| --- |
| |
| # qwen2.5-coder-3b-text2sql-qlora |
|
|
| QLoRA adapter for `Qwen/Qwen2.5-Coder-3B-Instruct`, fine-tuned to turn a table schema and a question into a SQL query. |
|
|
| Code and full evaluation: https://github.com/VarshithaChappidi/text2sql-qlora |
|
|
| ## Results (300 held-out examples) |
| | Model | Strict exact match | Normalized exact match | |
| |---|---|---| |
| | Base, prompted | 0.023 | 0.570 | |
| | This adapter | 0.767 | 0.773 | |
|
|
| Normalized match unifies quote style, drops wrapping quotes, treats `INNER JOIN` as `JOIN`, and ignores `AND` order. It still rejects some valid SQL written differently, so it is a lower bound. |
|
|
| ## Usage |
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| from peft import PeftModel |
| |
| base_id = "Qwen/Qwen2.5-Coder-3B-Instruct" |
| tok = AutoTokenizer.from_pretrained(base_id) |
| base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.float16, device_map="auto") |
| model = PeftModel.from_pretrained(base, "Varshitha2603/qwen2.5-coder-3b-text2sql-qlora") |
| |
| schema = "CREATE TABLE singer (name VARCHAR, age INT)" |
| question = "How many singers are older than 30?" |
| msgs = [ |
| {"role": "system", "content": "You are a SQL expert. Given a table schema and a question, reply with only the SQL query."}, |
| {"role": "user", "content": f"Schema:\n{schema}\n\nQuestion: {question}"}, |
| ] |
| prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) |
| inputs = tok(prompt, return_tensors="pt").to(model.device) |
| out = model.generate(**inputs, max_new_tokens=128, do_sample=False) |
| print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ## Training |
| - Data: 5,000 examples from `b-mc2/sql-create-context` (300 other examples held out for testing) |
| - QLoRA: 4-bit NF4, rank 16, alpha 32, dropout 0.05, all attention and MLP projections (29.9M trainable parameters) |
| - 1 epoch, learning rate 2e-4 with cosine schedule, effective batch size 16, loss on answer tokens only |
| - Free Colab T4 GPU, about 21 minutes. Loss fell from 0.212 (step 25) to 0.046 (step 300). |
|
|
| ## Limitations |
| - Evaluated by exact match, not execution, because the dataset has no table contents. |
| - Trained and tested only on single-table-style questions from one dataset, plus a few Spider-style joins. Multi-table joins are a known weak spot. |
| - Gold labels contain ambiguity and typos. Single training run, no variance estimate. |
| - The adapter inherits the license terms of the base model and the dataset. Check both before commercial use. |
