Instructions to use ximosss/ReViSQL-Qwen3-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ximosss/ReViSQL-Qwen3-4B with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507") model = PeftModel.from_pretrained(base_model, "ximosss/ReViSQL-Qwen3-4B") - Notebooks
- Google Colab
- Kaggle
Upload validation-selected step 1300 LoRA adapter
Browse files- README.md +71 -0
- adapter_config.json +19 -0
- adapter_model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: Qwen/Qwen3-4B-Instruct-2507
|
| 3 |
+
library_name: peft
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
tags:
|
| 7 |
+
- text-to-sql
|
| 8 |
+
- reinforcement-learning
|
| 9 |
+
- rlvr
|
| 10 |
+
- lora
|
| 11 |
+
- bird
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# ReViSQL-Qwen3-4B
|
| 15 |
+
|
| 16 |
+
LoRA adapter trained directly from
|
| 17 |
+
[`Qwen/Qwen3-4B-Instruct-2507`](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
|
| 18 |
+
with RLVR on expert-verified BIRD-Platinum data. This is the validation-selected
|
| 19 |
+
step 1,300 adapter from the
|
| 20 |
+
[`bird-text2sql-rl`](https://github.com/ximosss/bird-text2sql-rl) project.
|
| 21 |
+
|
| 22 |
+
## Results
|
| 23 |
+
|
| 24 |
+
| Dataset / decoding | Exact execution accuracy |
|
| 25 |
+
| --- | ---: |
|
| 26 |
+
| Arcwise-Plat-SQL, greedy | 376/498 = 75.50% |
|
| 27 |
+
| Arcwise-Plat-SQL, SC-16 | 414/498 = 83.13% |
|
| 28 |
+
| Arcwise-Plat, greedy | 404/498 = 81.12% |
|
| 29 |
+
| BIRD Full Dev, greedy | 961/1,534 = 62.65% |
|
| 30 |
+
| BIRD Mini-Dev, greedy | 298/500 = 59.60% |
|
| 31 |
+
|
| 32 |
+
Greedy uses temperature 0. SC-16 samples 16 candidates at temperature 1 and
|
| 33 |
+
selects the majority execution result. The model is evaluated as a SQL agent
|
| 34 |
+
with up to five rounds of read-only database interaction; see the source
|
| 35 |
+
repository for the exact prompt, environment, and evaluation contract.
|
| 36 |
+
|
| 37 |
+
## Loading the adapter
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
from peft import PeftModel
|
| 41 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 42 |
+
|
| 43 |
+
base_model_id = "Qwen/Qwen3-4B-Instruct-2507"
|
| 44 |
+
adapter_id = "ximosss/ReViSQL-Qwen3-4B"
|
| 45 |
+
|
| 46 |
+
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
| 47 |
+
base_model = AutoModelForCausalLM.from_pretrained(
|
| 48 |
+
base_model_id,
|
| 49 |
+
torch_dtype="auto",
|
| 50 |
+
device_map="auto",
|
| 51 |
+
)
|
| 52 |
+
model = PeftModel.from_pretrained(base_model, adapter_id)
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
The adapter alone does not reproduce the benchmark protocol. Use the database
|
| 56 |
+
tool environment and prompts in the source repository for evaluation.
|
| 57 |
+
|
| 58 |
+
## Training details
|
| 59 |
+
|
| 60 |
+
- Objective: CISPO with execution, VeriEQL, and evidence-process rewards
|
| 61 |
+
- LoRA: rank 32, alpha 64; attention and MLP linear layers
|
| 62 |
+
- Learning rate: `1e-5`
|
| 63 |
+
- Rollout temperature: `0.8`
|
| 64 |
+
- Training batch / group size: 64 / 16
|
| 65 |
+
|
| 66 |
+
## References
|
| 67 |
+
|
| 68 |
+
- Yuxuan Zhu, Tengjun Jin, Yoojin Choi, and Daniel Kang. 2026.
|
| 69 |
+
[Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering](https://arxiv.org/abs/2603.20004).
|
| 70 |
+
- Yuxuan Zhu, Tengjun Jin, Yoojin Choi, and Daniel Kang. 2026.
|
| 71 |
+
[Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQL](https://thinkingmachines.ai/news/putting-task-expertise-into-rl/).
|
adapter_config.json
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"peft_type": "LORA",
|
| 3 |
+
"task_type": "CAUSAL_LM",
|
| 4 |
+
"base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
|
| 5 |
+
"r": 32,
|
| 6 |
+
"lora_alpha": 64.0,
|
| 7 |
+
"lora_dropout": 0.0,
|
| 8 |
+
"bias": "none",
|
| 9 |
+
"target_modules": [
|
| 10 |
+
"down_proj",
|
| 11 |
+
"gate_proj",
|
| 12 |
+
"k_proj",
|
| 13 |
+
"o_proj",
|
| 14 |
+
"q_proj",
|
| 15 |
+
"up_proj",
|
| 16 |
+
"v_proj"
|
| 17 |
+
],
|
| 18 |
+
"modules_to_save": null
|
| 19 |
+
}
|
adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fbf9f12b9b7368e27bb7fa8b384aad6fd6037857337e2b6f9f28af54ed1ec9ea
|
| 3 |
+
size 264300328
|