ggcristian commited on
Commit
013ed78
·
1 Parent(s): c2d12f6

Hide readme

Browse files
Files changed (2) hide show
  1. INTRO.md +102 -0
  2. README.md +1 -102
INTRO.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Qwen3.6-27B-RL-Verif
2
+
3
+ A finetune of Qwen3.6-27B specialized for SystemVerilog test-stimulus and test-checking
4
+
5
+ Training was done in two stages:
6
+
7
+ - Supervised Finetuning (SFT) on 70k examples distilled from Gemma-4-31B
8
+ - Reinforcement Learning (RL) on 12k SystemVerilog environments
9
+
10
+ A smaller 9B variant is trained similarly.
11
+
12
+ # CVDP Results
13
+
14
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/64da082a79c87e13ca232042/MZwGhx1CoxyJ7F1rUHZCL.png" width=800px />
15
+
16
+ Full results on the [NVIDIA CVDP benchmark](https://github.com/NVlabs/cvdp_benchmark).
17
+
18
+ | | GLM-5.2 | Kimi-K2.5 | R1-0528 | Gemma-4-31B | Qwen3.6-27B | **Our 27B model** |
19
+ |:----|--:|--:|--:|--:|--:|--:|
20
+ | **CVDP Design Verification** | | | | | | |
21
+ | `cid012*` Testbench Stimulus Gen. | 34.03% | 22.09% | 22.99% | 24.78% | 19.40% | **39.40%** |
22
+ | `cid013*` Testbench Checker Gen. | 13.96% | 23.40% | 18.87% | 17.74% | 15.09% | **31.32%** |
23
+ | `cid014*` Assertion Generation | 33.13% | 36.12% | 22.69% | 33.13% | 22.39% | **39.70%** |
24
+ | `cid016` Debugging / Bug Fixing | 44.57% | **64.57%** | 54.29% | 63.43% | 58.86% | 56.57% |
25
+ | **CVDP Code Generation** | | | | | | |
26
+ | `cid002` Code Completion | **48.51%** | 44.04% | 34.47% | 39.57% | 32.55% | 29.57% |
27
+ | `cid003` Natural Language Spec to Code | 52.82% | **56.15%** | 41.28% | 48.21% | 37.18% | 50.00% |
28
+ | `cid004` Code Modification | 50.91% | 50.55% | 37.45% | **54.55%** | 35.27% | 41.45% |
29
+ | `cid007` Code Improvement (Linting/QoR) | 51.50% | 59.50% | 39.50% | **64.50%** | 50.00% | 40.50% |
30
+ | **CVDP Code Comprehension** | | | | | | |
31
+ | `cid006` RTL ↔ Specification | 69.17% | 70.53% | **72.77%** | 67.66% | 68.25% | 72.69% |
32
+ | `cid008` Testbench ↔ Test Plan | 61.83% | **66.69%** | 45.65% | 63.10% | 62.06% | 58.08% |
33
+
34
+ Note that those categories with asterisks in the title were converted from Commercial tools (XCelium) to OSS tools (Verilator)
35
+
36
+ # Deployment
37
+
38
+ ## Serving with vLLM
39
+
40
+ Install [vLLM](https://github.com/vllm-project/vllm) `>= 0.23`, then serve the model:
41
+
42
+ ```bash
43
+ vllm serve HPAI-BSC/Qwen3.6-27B-RL-Verif \
44
+ --trust-remote-code \
45
+ --tensor-parallel-size 1 \
46
+ --reasoning-parser qwen3 \
47
+ --language-model-only \
48
+ --enable-auto-tool-choice \
49
+ --tool-call-parser qwen3_coder
50
+ ```
51
+
52
+ ## MareNostrum 5
53
+
54
+ From a login node, submit [`launch.slurm`](launch.slurm) with `sbatch`:
55
+
56
+ ```bash
57
+ sbatch launch.slurm
58
+ ```
59
+
60
+ Once the model is loaded, the job output prints the connection details:
61
+
62
+ ```
63
+ ============================================
64
+ vLLM Server is UP and READY!
65
+ ============================================
66
+ Job ID: 43755135
67
+ Node IP: 10.2.101.26
68
+ vLLM Port: 41211
69
+
70
+ To test from a login node:
71
+ curl http://10.2.101.26:41211/v1/models
72
+
73
+ To connect from your laptop (replace bscXXXXXX with your BSC username):
74
+ ssh -N -L 8000:10.2.101.26:41211 bscXXXXXX@glogin1.bsc.es
75
+
76
+ Then test from your laptop:
77
+ curl http://localhost:8000/v1/models
78
+
79
+ Server will keep running until time limit or scancel 43755135
80
+ ============================================
81
+ ```
82
+
83
+ Open the SSH tunnel it gives you (forwards local port `8000` to the compute node):
84
+
85
+ ```bash
86
+ ssh -N -L 8000:10.2.101.26:41211 bscXXXXXX@glogin1.bsc.es
87
+ ```
88
+
89
+ Then send a request from your laptop:
90
+
91
+ ```bash
92
+ curl http://localhost:8000/v1/chat/completions \
93
+ -H "Content-Type: application/json" \
94
+ -d '{
95
+ "model": "Qwen3.6-27B-RL-Verif",
96
+ "messages": [{"role": "user", "content": "Hello world"}]
97
+ }'
98
+ ```
99
+
100
+ # Contact
101
+
102
+ For questions, reach out to [cristian.gutierrez@bsc.es](mailto:cristian.gutierrez@bsc.es).
README.md CHANGED
@@ -1,102 +1 @@
1
- # Qwen3.6-27B-RL-Verif
2
-
3
- A finetune of Qwen3.6-27B specialized for SystemVerilog test-stimulus and test-checking
4
-
5
- Training was done in two stages:
6
-
7
- - Supervised Finetuning (SFT) on 70k examples distilled from Gemma-4-31B
8
- - Reinforcement Learning (RL) on 12k SystemVerilog environments
9
-
10
- A smaller 9B variant is trained similarly.
11
-
12
- # CVDP Results
13
-
14
- <img src="https://cdn-uploads.huggingface.co/production/uploads/64da082a79c87e13ca232042/MZwGhx1CoxyJ7F1rUHZCL.png" width=800px />
15
-
16
- Full results on the [NVIDIA CVDP benchmark](https://github.com/NVlabs/cvdp_benchmark).
17
-
18
- | | GLM-5.2 | Kimi-K2.5 | R1-0528 | Gemma-4-31B | Qwen3.6-27B | **Our 27B model** |
19
- |:----|--:|--:|--:|--:|--:|--:|
20
- | **CVDP Design Verification** | | | | | | |
21
- | `cid012*` Testbench Stimulus Gen. | 34.03% | 22.09% | 22.99% | 24.78% | 19.40% | **39.40%** |
22
- | `cid013*` Testbench Checker Gen. | 13.96% | 23.40% | 18.87% | 17.74% | 15.09% | **31.32%** |
23
- | `cid014*` Assertion Generation | 33.13% | 36.12% | 22.69% | 33.13% | 22.39% | **39.70%** |
24
- | `cid016` Debugging / Bug Fixing | 44.57% | **64.57%** | 54.29% | 63.43% | 58.86% | 56.57% |
25
- | **CVDP Code Generation** | | | | | | |
26
- | `cid002` Code Completion | **48.51%** | 44.04% | 34.47% | 39.57% | 32.55% | 29.57% |
27
- | `cid003` Natural Language Spec to Code | 52.82% | **56.15%** | 41.28% | 48.21% | 37.18% | 50.00% |
28
- | `cid004` Code Modification | 50.91% | 50.55% | 37.45% | **54.55%** | 35.27% | 41.45% |
29
- | `cid007` Code Improvement (Linting/QoR) | 51.50% | 59.50% | 39.50% | **64.50%** | 50.00% | 40.50% |
30
- | **CVDP Code Comprehension** | | | | | | |
31
- | `cid006` RTL ↔ Specification | 69.17% | 70.53% | **72.77%** | 67.66% | 68.25% | 72.69% |
32
- | `cid008` Testbench ↔ Test Plan | 61.83% | **66.69%** | 45.65% | 63.10% | 62.06% | 58.08% |
33
-
34
- Note that those categories with asterisks in the title were converted from Commercial tools (XCelium) to OSS tools (Verilator)
35
-
36
- # Deployment
37
-
38
- ## Serving with vLLM
39
-
40
- Install [vLLM](https://github.com/vllm-project/vllm) `>= 0.23`, then serve the model:
41
-
42
- ```bash
43
- vllm serve HPAI-BSC/Qwen3.6-27B-RL-Verif \
44
- --trust-remote-code \
45
- --tensor-parallel-size 1 \
46
- --reasoning-parser qwen3 \
47
- --language-model-only \
48
- --enable-auto-tool-choice \
49
- --tool-call-parser qwen3_coder
50
- ```
51
-
52
- ## MareNostrum 5
53
-
54
- From a login node, submit [`launch.slurm`](launch.slurm) with `sbatch`:
55
-
56
- ```bash
57
- sbatch launch.slurm
58
- ```
59
-
60
- Once the model is loaded, the job output prints the connection details:
61
-
62
- ```
63
- ============================================
64
- vLLM Server is UP and READY!
65
- ============================================
66
- Job ID: 43755135
67
- Node IP: 10.2.101.26
68
- vLLM Port: 41211
69
-
70
- To test from a login node:
71
- curl http://10.2.101.26:41211/v1/models
72
-
73
- To connect from your laptop (replace bscXXXXXX with your BSC username):
74
- ssh -N -L 8000:10.2.101.26:41211 bscXXXXXX@glogin1.bsc.es
75
-
76
- Then test from your laptop:
77
- curl http://localhost:8000/v1/models
78
-
79
- Server will keep running until time limit or scancel 43755135
80
- ============================================
81
- ```
82
-
83
- Open the SSH tunnel it gives you (forwards local port `8000` to the compute node):
84
-
85
- ```bash
86
- ssh -N -L 8000:10.2.101.26:41211 bscXXXXXX@glogin1.bsc.es
87
- ```
88
-
89
- Then send a request from your laptop:
90
-
91
- ```bash
92
- curl http://localhost:8000/v1/chat/completions \
93
- -H "Content-Type: application/json" \
94
- -d '{
95
- "model": "Qwen3.6-27B-RL-Verif",
96
- "messages": [{"role": "user", "content": "Hello world"}]
97
- }'
98
- ```
99
-
100
- # Contact
101
-
102
- For questions, reach out to [cristian.gutierrez@bsc.es](mailto:cristian.gutierrez@bsc.es).
 
1
+ See [`INTRO.md`](/INTRO.md)