qwen3.5-9b-text2sql-grpo

An agentic text-to-SQL model: given a natural-language question and a SQLite database, it explores the schema with tools, runs candidate queries, observes real execution feedback, and submits a final SQL query.

Trained with agentic GRPO directly from vanilla Qwen/Qwen3.5-9B โ€” no supervised finetuning stage, no teacher traces. 200 GRPO steps in an execution-grounded environment (verifiers / prime-rl): the reward executes the submitted query against the real database and compares result sets to gold (1.0 match / 0.1 runs-but-wrong / 0.0 error / โˆ’1.0 write attempt). Training prompts come from the BIRD train split, filtered to those the base policy solves 1โ€“7 times out of 8 (prompts with no gradient signal are dropped).

Results (BIRD dev, 1,534 examples, execution accuracy)

setting EX
agentic, greedy 63.4%
agentic, exec-consistency voting @ k=8 / k=32 (temp 1.0) 67.0% / 67.4%
single-turn (no tools), greedy 58.2%
base Qwen/Qwen3.5-9B zero-shot agentic, for reference 49.7%

Exec-consistency voting: sample k rollouts, group the submitted queries by the result set they return, exclude empty results, take the largest group (majority over result sets โ€” never over SQL text, and without de-duplicating verbatim repeats).

Usage

The model calls tools in Qwen3.5's XML dialect โ€” serve with vLLM using --enable-auto-tool-choice --tool-call-parser qwen3_xml (the hermes parser silently parses nothing).

It expects the agentic environment's tool set: inspect_schema, sample_rows, run_sql (read-only), submit. The training/eval harness, environment definition, and a ready-to-use CLI (scripts/ask.py, agentic and single-turn modes, works against any SQLite file) live in the training repository: https://github.com/oliverkinch/text2sql-env

It also works as a plain single-turn SQL generator (schema in the prompt, SQL out) at the lower accuracy listed above.

Limitations

  • Tuned and measured on SQLite only.
  • Rewarded only for read queries; it was penalized for writes, but do not point it at a database you cannot afford to mutate without a read-only connection (the training harness enforced one).
  • BIRD dev was used for model selection across the project's history; treat dev numbers as development metrics, not test claims.
Downloads last month
21
Safetensors
Model size
10B params
Tensor type
BF16
ยท
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for oliverkinch/qwen3.5-9b-text2sql-grpo

Finetuned
Qwen/Qwen3.5-9B
Finetuned
(1,426)
this model