--- base_model: Qwen/Qwen3.5-2B library_name: peft license: cc-by-sa-4.0 datasets: - xlangai/spider - gretelai/synthetic_text_to_sql tags: [lora, qwen3_5, text-to-sql, sql, sqlite] --- # Privoid SQL (Qwen3.5-2B) A LoRA fine-tune of [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B) (revision `15852e8c`) by Privoid. Give it a SQLite schema and a question; it replies with one SQLite query. GGUF files are in [Privoid-ai/Privoid-SQL-Qwen3.5-2B-GGUF](https://huggingface.co/Privoid-ai/Privoid-SQL-Qwen3.5-2B-GGUF). ## Benchmark: Spider dev, execution accuracy ![Execution accuracy on Spider dev](benchmark/benchmark.png) | Model | Correct results | Query ran without error | |---|---|---| | Qwen3.5-2B | 53.5% (553/1034) | 82.1% | | Privoid SQL | 72.0% (744/1034) | 93.7% | - Every question in the Spider dev set whose reference query runs, on the real SQLite databases. None of these databases were used in training. - A query is correct when its result rows match the reference query's rows. Order counts only when the reference uses `ORDER BY`. - Both models get the same prompt and greedy decoding, with thinking off. - Fine-tuning fixed 254 questions that the base model got wrong and broke 63 that it got right. - This check is simpler than the official Spider test-suite evaluation, so these numbers are not directly comparable with the Spider leaderboard. Per-question results are in [benchmark/results.json](benchmark/results.json). ## Prompt format System prompt: ```text You are Privoid SQL. Given a SQLite database schema and a question, reply with one SQLite query that answers it, and nothing else. ``` User message: ```text Database schema: Question: ``` Use thinking off (`enable_thinking=False`). With llama.cpp: `llama-server -m Privoid-SQL-Qwen3.5-2B-Q8_0.gguf --jinja`, then send `chat_template_kwargs: {"enable_thinking": false}`. ## Training - LoRA r=16, alpha=32 on the linear-attention, attention and MLP layers, on one NVIDIA L4. - 240 updates in 39.3 minutes, stopped by time limit. Training loss 0.589 to 0.152. - Training pool: 6918 rows from Spider train (with schemas and two example rows per table) and 3150 from Gretel synthetic text-to-SQL, kept only if they run in SQLite. The time limit allowed 240 of 518 batches, so about 46% of this pool was seen, once. ## Credits and licence - Base model: [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B), Apache-2.0. - [Spider](https://yale-lily.github.io/spider) (Yu et al., EMNLP 2018), CC BY-SA 4.0. Databases from the pinned re-host [HAL-9001/spider-databases](https://huggingface.co/datasets/HAL-9001/spider-databases). - [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql), Apache-2.0. - Because the training data includes Spider, this model is released under CC BY-SA 4.0. ## Limitations - SQLite only, one query per answer. It has not been tested on other SQL dialects or on very large schemas. - It can return queries that run but answer a slightly different question. Check results before relying on them, and run generated SQL with read-only database access.