README.md

---
title: LocalSQL
sdk: gradio
app_file: app.py
license: apache-2.0
colorFrom: blue
colorTo: green
pinned: true
---

# LocalSQL

Ask a SQLite database questions in plain English.

LocalSQL is a small, open text-to-SQL tool built around a fine-tuned
Qwen2.5-Coder-7B model. It reasons step by step over the schema, writes the SQL,
and runs it read-only against your database. The default model,
[`jk200201/qwen2.5-coder-7b-bird-cot`](https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-cot),
scores **52.1%** greedy and **58.5%** with self-consistency (K=8) on the BIRD
dev split, matching DeepSeek V4-Pro (58.7%, 1.6T params) at roughly 0.4% of the
size.

The product goal is simple:

- **Local mode:** run the model on your own GPU, point it at your own `.sqlite`
  file, and keep your data on your machine.
- **Demo mode:** run the same model in a hosted Gradio app against sample data,
  or upload your own `.sqlite` file to try it without any setup.

```bash
python3 -m src.text2sql --db chinook.sqlite --q "Which 5 artists have the most albums?"
```

```text
SQL
SELECT ar.Name, COUNT(al.AlbumId) AS album_count
FROM Artist ar
JOIN Album al ON ar.ArtistId = al.ArtistId
GROUP BY ar.ArtistId
ORDER BY album_count DESC
LIMIT 5;

Results
| Name         | album_count |
|--------------|-------------|
| Iron Maiden  | 21          |
| Led Zeppelin | 14          |
```

## Why This Exists

Most text-to-SQL tools are cloud wrappers: send schema plus question to a large
API model, pay per request, and trust the remote provider with your database
structure. LocalSQL explores the opposite path: a specialized 7B model that runs
locally and still competes with much larger frontier models on SQL tasks. With
self-consistency it reaches the accuracy of a 1.6T frontier model while keeping
your schema and rows on your own machine.

## Models

| Model | Best For | Notes |
|---|---|---|
| `jk200201/qwen2.5-coder-7b-bird-cot` | General SQLite questions, messy schemas | Default. Merged CoT model. 52.1% greedy / 58.5% self-consistency on BIRD dev. |
| `jk200201/qwen2.5-coder-7b-bird-cot-GGUF` | Running on a laptop (Ollama / llama.cpp) | Quantized builds (Q4_K_M is 4.4 GB). `ollama run muence/bird-cot`. |

The model was trained on top of `Qwen/Qwen2.5-Coder-7B-Instruct`.

## Install

```bash
git clone https://github.com/jenishk20/finetuning-text-to-sql
cd finetuning-text-to-sql
pip install -r requirements.txt
```

You need a CUDA GPU for 4-bit inference. A hosted demo should use a GPU-backed
Hugging Face Space or an equivalent GPU service.

## CLI Usage

```bash
# Generate SQL and execute it read-only against a SQLite database.
python3 -m src.text2sql \
  --db mydata.sqlite \
  --q "top 5 customers by revenue"

# Generate SQL only (no execution).
python3 -m src.text2sql \
  --db mydata.sqlite \
  --q "total revenue per month in 2023" \
  --no-exec

# Add a domain hint for questions that need external knowledge.
python3 -m src.text2sql \
  --db mydata.sqlite \
  --evidence "revenue = price * quantity" \
  --q "revenue by month"

# Skip reasoning and use the direct-SQL prompt (faster, usually less accurate).
python3 -m src.text2sql --db mydata.sqlite --q "top 5 customers" --direct
```

As a library:

```python
from src.text2sql import load_model, predict

model, tokenizer = load_model()  # loads the merged CoT model
result = predict("mydata.sqlite", "top 5 customers by revenue", model, tokenizer)
print(result["sql"])
print(result["reasoning"])  # the step-by-step chain of thought
print(result["rows"])
```

## Hosted Demo

This repo includes a Gradio app:

```bash
python3 app.py
```

The public Space runs against a generated sample SQLite database, and you can
upload your own `.sqlite` file to try it on real data. Uploaded files are used
only for the current query and are not stored. For fully private use, run the
CLI locally so your schema and rows never leave your machine.

## Research Snapshot

The training recipe is the interesting part. The current best model is a
chain-of-thought SFT: distil step-by-step SQL reasoning from a strong teacher on
the BIRD train split, keep only the correct chains, and fine-tune the 7B on
them. It is then evaluated on the held-out BIRD dev split.

BIRD dev (execution accuracy):

| Model | Params | Result Accuracy |
|---|---:|---:|
| Qwen2.5-Coder-7B base | 7B | 27.0% |
| LocalSQL, greedy | 7B | 52.1% |
| LocalSQL, self-consistency (K=8) | 7B | 58.5% |
| DeepSeek V4-Pro | 1.6T | 58.7% |
| GLM 5.2 | 744B | 63.0% |

For the broader experiment log, see `docs/research/`.

> **Note on an earlier number.** A previous version of this project reported
> 78.2% on Spider dev for a DPO model. That evaluation was later found to be
> train/test contaminated (the DPO preference pairs were built from the same
> dev split used for scoring), so that number has been retired. The BIRD results
> above use a clean train/dev split.

## Roadmap

- Keep the hosted Space warm and let users upload their own SQLite files.
- High-accuracy mode using Best-of-N execution self-consistency in the demo.
- Schema search/linking for large databases.
- Postgres and MySQL connectors.
- On-policy RL (GRPO / rejection fine-tuning) to push past 58.5%.

## Citation

```bibtex
@misc{kothari2026localsql,
  author = {Kothari, Jenish},
  title  = {LocalSQL: A Local Chain-of-Thought Text-to-SQL Model},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/jk200201/qwen2.5-coder-7b-bird-cot}},
}
```