ximosss commited on
Commit
b91c06c
·
verified ·
1 Parent(s): bb2fb8d

Upload validation-selected step 1300 LoRA adapter

Browse files
Files changed (3) hide show
  1. README.md +71 -0
  2. adapter_config.json +19 -0
  3. adapter_model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen3-4B-Instruct-2507
3
+ library_name: peft
4
+ license: apache-2.0
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - text-to-sql
8
+ - reinforcement-learning
9
+ - rlvr
10
+ - lora
11
+ - bird
12
+ ---
13
+
14
+ # ReViSQL-Qwen3-4B
15
+
16
+ LoRA adapter trained directly from
17
+ [`Qwen/Qwen3-4B-Instruct-2507`](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
18
+ with RLVR on expert-verified BIRD-Platinum data. This is the validation-selected
19
+ step 1,300 adapter from the
20
+ [`bird-text2sql-rl`](https://github.com/ximosss/bird-text2sql-rl) project.
21
+
22
+ ## Results
23
+
24
+ | Dataset / decoding | Exact execution accuracy |
25
+ | --- | ---: |
26
+ | Arcwise-Plat-SQL, greedy | 376/498 = 75.50% |
27
+ | Arcwise-Plat-SQL, SC-16 | 414/498 = 83.13% |
28
+ | Arcwise-Plat, greedy | 404/498 = 81.12% |
29
+ | BIRD Full Dev, greedy | 961/1,534 = 62.65% |
30
+ | BIRD Mini-Dev, greedy | 298/500 = 59.60% |
31
+
32
+ Greedy uses temperature 0. SC-16 samples 16 candidates at temperature 1 and
33
+ selects the majority execution result. The model is evaluated as a SQL agent
34
+ with up to five rounds of read-only database interaction; see the source
35
+ repository for the exact prompt, environment, and evaluation contract.
36
+
37
+ ## Loading the adapter
38
+
39
+ ```python
40
+ from peft import PeftModel
41
+ from transformers import AutoModelForCausalLM, AutoTokenizer
42
+
43
+ base_model_id = "Qwen/Qwen3-4B-Instruct-2507"
44
+ adapter_id = "ximosss/ReViSQL-Qwen3-4B"
45
+
46
+ tokenizer = AutoTokenizer.from_pretrained(base_model_id)
47
+ base_model = AutoModelForCausalLM.from_pretrained(
48
+ base_model_id,
49
+ torch_dtype="auto",
50
+ device_map="auto",
51
+ )
52
+ model = PeftModel.from_pretrained(base_model, adapter_id)
53
+ ```
54
+
55
+ The adapter alone does not reproduce the benchmark protocol. Use the database
56
+ tool environment and prompts in the source repository for evaluation.
57
+
58
+ ## Training details
59
+
60
+ - Objective: CISPO with execution, VeriEQL, and evidence-process rewards
61
+ - LoRA: rank 32, alpha 64; attention and MLP linear layers
62
+ - Learning rate: `1e-5`
63
+ - Rollout temperature: `0.8`
64
+ - Training batch / group size: 64 / 16
65
+
66
+ ## References
67
+
68
+ - Yuxuan Zhu, Tengjun Jin, Yoojin Choi, and Daniel Kang. 2026.
69
+ [Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineering](https://arxiv.org/abs/2603.20004).
70
+ - Yuxuan Zhu, Tengjun Jin, Yoojin Choi, and Daniel Kang. 2026.
71
+ [Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQL](https://thinkingmachines.ai/news/putting-task-expertise-into-rl/).
adapter_config.json ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "peft_type": "LORA",
3
+ "task_type": "CAUSAL_LM",
4
+ "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
5
+ "r": 32,
6
+ "lora_alpha": 64.0,
7
+ "lora_dropout": 0.0,
8
+ "bias": "none",
9
+ "target_modules": [
10
+ "down_proj",
11
+ "gate_proj",
12
+ "k_proj",
13
+ "o_proj",
14
+ "q_proj",
15
+ "up_proj",
16
+ "v_proj"
17
+ ],
18
+ "modules_to_save": null
19
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fbf9f12b9b7368e27bb7fa8b384aad6fd6037857337e2b6f9f28af54ed1ec9ea
3
+ size 264300328