PEFT
Safetensors
lora
qwen3_5
text-to-sql
sql
sqlite
Privoid-ai's picture
Card: state how much of the training pool was used
b0dd27c verified
|
Raw History Blame Contribute Delete
3.22 kB
---
base_model: Qwen/Qwen3.5-2B
library_name: peft
license: cc-by-sa-4.0
datasets:
- xlangai/spider
- gretelai/synthetic_text_to_sql
tags: [lora, qwen3_5, text-to-sql, sql, sqlite]
---
# Privoid SQL (Qwen3.5-2B)
A LoRA fine-tune of [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) (revision `15852e8c`) by Privoid. Give it a SQLite schema and a question; it replies with one SQLite query. GGUF files are in [Privoid-ai/Privoid-SQL-Qwen3.5-2B-GGUF](https://huggingface.co/Privoid-ai/Privoid-SQL-Qwen3.5-2B-GGUF).
## Benchmark: Spider dev, execution accuracy
![Execution accuracy on Spider dev](benchmark/benchmark.png)
| Model | Correct results | Query ran without error |
|---|---|---|
| Qwen3.5-2B | 53.5% (553/1034) | 82.1% |
| Privoid SQL | 72.0% (744/1034) | 93.7% |
- Every question in the Spider dev set whose reference query runs, on the real SQLite databases. None of these databases were used in training.
- A query is correct when its result rows match the reference query's rows. Order counts only when the reference uses `ORDER BY`.
- Both models get the same prompt and greedy decoding, with thinking off.
- Fine-tuning fixed 254 questions that the base model got wrong and broke 63 that it got right.
- This check is simpler than the official Spider test-suite evaluation, so these numbers are not directly comparable with the Spider leaderboard.
Per-question results are in [benchmark/results.json](benchmark/results.json).
## Prompt format
System prompt:
```text
You are Privoid SQL. Given a SQLite database schema and a question, reply with one SQLite query that answers it, and nothing else.
```
User message:
```text
Database schema:
<CREATE TABLE statements, optionally with a few example rows in /* */ comments>
Question: <question>
```
Use thinking off (`enable_thinking=False`). With llama.cpp: `llama-server -m Privoid-SQL-Qwen3.5-2B-Q8_0.gguf --jinja`, then send `chat_template_kwargs: {"enable_thinking": false}`.
## Training
- LoRA r=16, alpha=32 on the linear-attention, attention and MLP layers, on one NVIDIA L4.
- 240 updates in 39.3 minutes, stopped by time limit. Training loss 0.589 to 0.152.
- Training pool: 6918 rows from Spider train (with schemas and two example rows per table) and 3150 from Gretel synthetic text-to-SQL, kept only if they run in SQLite. The time limit allowed 240 of 518 batches, so about 46% of this pool was seen, once.
## Credits and licence
- Base model: [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B), Apache-2.0.
- [Spider](https://yale-lily.github.io/spider) (Yu et al., EMNLP 2018), CC BY-SA 4.0. Databases from the pinned re-host [HAL-9001/spider-databases](https://huggingface.co/datasets/HAL-9001/spider-databases).
- [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql), Apache-2.0.
- Because the training data includes Spider, this model is released under CC BY-SA 4.0.
## Limitations
- SQLite only, one query per answer. It has not been tested on other SQL dialects or on very large schemas.
- It can return queries that run but answer a slightly different question. Check results before relying on them, and run generated SQL with read-only database access.