WestQuantStudio commited on
Commit
1963f60
·
verified ·
1 Parent(s): 9b59662

Upload folder using huggingface_hub

Browse files
Files changed (6) hide show
  1. README.md +304 -0
  2. config.json +32 -0
  3. generation_config.json +10 -0
  4. model.safetensors +3 -0
  5. tokenizer.json +0 -0
  6. tokenizer_config.json +11 -0
README.md ADDED
@@ -0,0 +1,304 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: apache-2.0
4
+ library_name: transformers
5
+ tags:
6
+ - quantum
7
+ - optimization
8
+ - scheduling
9
+ - quantum-computing
10
+ - qiskit
11
+ - tket
12
+ - pyzx
13
+ - nisq
14
+ pipeline_tag: text-generation
15
+ ---
16
+
17
+ # WQT50M
18
+
19
+ **WestQuant Transformer 50M — Quantum Representation Scheduler**
20
+
21
+ > AI schedules. Deterministic mathematics executes. Independent verification certifies.
22
+
23
+ WQT50M is a 50M-parameter Transformer that ranks quantum transformations.
24
+ Trained on **real Qiskit transpilation outputs** — not synthetic data. It
25
+ predicts which transformation is most promising given a circuit, backend, and
26
+ optimization objective.
27
+
28
+ WQT50M is a proof-of-concept that a 50M-parameter Transformer can learn
29
+ structured quantum optimization preferences from real compiler outputs.
30
+ Researchers building better compiler-optimization models can use this as a
31
+ baseline.
32
+
33
+ ---
34
+
35
+ ## What It Does — 3 Tested Examples
36
+
37
+ ### Example 1: 62% Gate Reduction (Random Circuit, Fidelity-Focused)
38
+
39
+ ```
40
+ Problem: 8-qubit random circuit (depth 30) on grid topology
41
+ Naive transpilation → cost 1231.2 (fidelity-focused objective)
42
+
43
+ WQT50M schedules ZX_SIMPLIFY
44
+ → cost 467.4 (62.0% reduction)
45
+
46
+ Random selection → cost 869.6 (29.4% reduction)
47
+ Oracle (exhaustive) → cost 467.4 (62.0%)
48
+ WQT50M matches oracle ✓
49
+ ```
50
+
51
+ ### Example 2: Objective-Aware Scheduling (QAOA, Linear Backend)
52
+
53
+ The same QAOA circuit gets different best actions depending on the objective:
54
+
55
+ ```
56
+ QAOA-6q-p2 on linear backend:
57
+
58
+ Objective WQT50M picks
59
+ ───────────────── ─────────────────
60
+ balanced → CANCEL_GATES
61
+ 2q_focused → CANCEL_GATES
62
+ depth_focused → NATIVE_GATESET
63
+ fidelity_focused → FUSE_ROTATIONS
64
+ time_focused → MERGE_ADJACENT
65
+
66
+ 4 different actions for 5 objectives ✓
67
+ ```
68
+
69
+ ### Example 3: Calibrated Value Prediction
70
+
71
+ WQT50M predicts costs in the **real magnitude range** (not a narrow band):
72
+
73
+ ```
74
+ State: <DOMAIN:circuit_optimization> <LEVEL:CIRCUIT> <N_QUBITS:8> ...
75
+ <BACKEND:SUPERCONDUCTING> <TOPO:grid> <T1:180us> <T2:90us>
76
+ <OBJ_TYPE:fidelity_focused>
77
+
78
+ Action: ZX_SIMPLIFY
79
+ Predicted cost: 467.4
80
+ Actual cost: 467.4
81
+ Error: 0.0%
82
+
83
+ Calibration ratio: 0.999 (predicted range matches actual range)
84
+ Spearman correlation: 0.981
85
+ ```
86
+
87
+ ---
88
+
89
+ ## Quick Start
90
+
91
+ ```bash
92
+ pip install transformers torch
93
+ ```
94
+
95
+ ```python
96
+ from transformers import AutoModelForCausalLM, AutoTokenizer
97
+ import torch
98
+
99
+ model = AutoModelForCausalLM.from_pretrained("WestQuantStudio/WQT50M")
100
+ tokenizer = AutoTokenizer.from_pretrained("WestQuantStudio/WQT50M")
101
+ device = "mps" if torch.backends.mps.is_available() else "cpu"
102
+ model = model.to(device).eval()
103
+
104
+ # Predict cost for a state-action pair
105
+ state = ("<DOMAIN:circuit_optimization> <LEVEL:CIRCUIT> "
106
+ "<N_QUBITS:8> <ENTANGLEMENT:0.5000> "
107
+ "<RES:n_q=8 D=30 G1=100 G2=20 T=120 M=0 A=0.2000 E=0.0050 C=0.5000> "
108
+ "<BACKEND:SUPERCONDUCTING> <TOPO:grid> "
109
+ "<T1:180us> <T2:90us> <READOUT_ERR:0.0120> "
110
+ "<OBJ_TYPE:fidelity_focused> <STEP:0/5>")
111
+
112
+ text = f"<PREDICT> {state} <ACTION> ZX_SIMPLIFY <COST> "
113
+ ids = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(device)
114
+ with torch.no_grad():
115
+ for _ in range(8):
116
+ out = model(input_ids=ids)
117
+ nxt = out.logits[0, -1].argmax().unsqueeze(0).unsqueeze(0)
118
+ ids = torch.cat([ids, nxt], dim=1)
119
+ if nxt.item() == tokenizer.eos_token_id:
120
+ break
121
+
122
+ print(tokenizer.decode(ids[0].tolist()).split("<COST>")[-1].strip())
123
+ # → predicted cost in real magnitude range
124
+ ```
125
+
126
+ ---
127
+
128
+ ## What WQT50M Predicts
129
+
130
+ | Task | Input | Output | Performance |
131
+ |------|-------|--------|-------------|
132
+ | **Value prediction** | State + action | Predicted cost (calibrated) | Spearman 0.981, ratio 0.999 |
133
+ | **Ranked policy** | State + all actions | Best action (via value ranking) | 74% Top-1 |
134
+ | **Objective sensitivity** | Same state, different objectives | Different best actions | 65% change rate |
135
+ | **Backend awareness** | Same state, different backends | Different best actions | 60% change rate |
136
+ | **Legality** | State + action | Is this action legal? | 79.2% |
137
+ | **Search improvement** | Model-guided vs random | Wins over random | 90.4% win rate |
138
+
139
+ ---
140
+
141
+ ## Model
142
+
143
+ | Config | WQT20M-Beta | WQT50M |
144
+ |--------|-------------|--------|
145
+ | Architecture | Llama-style decoder | Llama-style decoder |
146
+ | Parameters | 19.06M | **49.53M** |
147
+ | Layers | 8 | **12** |
148
+ | Hidden size | 384 | **512** |
149
+ | Attention heads | 6 (2 KV) | **8 (4 KV)** |
150
+ | FFN | 1536 | **2048** |
151
+ | Context length | 2048 | **4096** |
152
+ | Vocab | 4561 | 4561 |
153
+ | Model size | 76 MB | **189 MB** |
154
+ | Training data | Synthetic surrogate | **Real Qiskit transpilation** |
155
+
156
+ Standard HuggingFace `LlamaForCausalLM`. Compatible with `AutoModelForCausalLM`,
157
+ `AutoTokenizer`, and SafeTensors.
158
+
159
+ ---
160
+
161
+ ## Validation Results
162
+
163
+ ### 450 tests: 15 circuits × 5 objectives × 6 backends
164
+
165
+ | Metric | WQT20M-Beta | WQT50M |
166
+ |--------|-------------|--------|
167
+ | Top-1 accuracy (picks best action) | ~22% | **74.0%** |
168
+ | Beats random | 58% | **90.4%** |
169
+ | Avg improvement over naive | -14.8% | **+16.6%** |
170
+ | Oracle improvement | +31.1% | +19.7% |
171
+ | Efficiency (% of oracle captured) | 23.3% | **88.7%** |
172
+
173
+ ### By objective
174
+
175
+ | Objective | WQT50M improvement | Random | Oracle | Efficiency |
176
+ |-----------|-------------------|--------|--------|------------|
177
+ | balanced | 22.2% | 9.3% | 25.2% | 87.7% |
178
+ | 2q_focused | 8.8% | 3.8% | 11.2% | 92.1% |
179
+ | depth_focused | 24.1% | 9.7% | 28.0% | 87.6% |
180
+ | fidelity_focused | 18.2% | 7.7% | 21.9% | 86.5% |
181
+ | time_focused | 9.6% | 4.3% | 12.3% | 89.8% |
182
+
183
+ ### By circuit type
184
+
185
+ | Circuit | WQT50M improvement | Random | Oracle | Efficiency |
186
+ |---------|-------------------|--------|--------|------------|
187
+ | BV-4q | 26.6% | 7.8% | 26.6% | **100%** |
188
+ | GHZ-4q | 11.9% | 2.4% | 11.9% | **100%** |
189
+ | GHZ-8q | 6.5% | 1.3% | 6.5% | **100%** |
190
+ | QFT-4q | 11.5% | 3.5% | 11.6% | **99.0%** |
191
+ | Grover-4q | 21.3% | 8.3% | 21.5% | **98.8%** |
192
+ | QFT-6q | 8.8% | 2.8% | 9.2% | 95.7% |
193
+ | QFT-8q | 1.1% | 2.4% | 7.9% | 95.3% |
194
+ | Random-6q-d20 | 34.1% | 16.2% | 37.1% | 91.9% |
195
+ | Grover-5q | 23.8% | 12.6% | 30.3% | 87.2% |
196
+ | Random-8q-d30 | 51.1% | 25.6% | 57.2% | 89.4% |
197
+
198
+ ### Biggest improvements
199
+
200
+ | Circuit | Backend | Objective | Naive → WQT50M | Improvement |
201
+ |---------|---------|-----------|----------------|-------------|
202
+ | Random-8q-d30 | grid | fidelity | 1231 → 467 | **62.0%** |
203
+ | Random-8q-d30 | ion trap | fidelity | 273 → 118 | **56.7%** |
204
+ | Random-8q-d30 | linear | depth | 1752 → 763 | **56.5%** |
205
+ | Random-8q-d30 | ring | depth | 1752 → 763 | **56.5%** |
206
+ | Random-8q-d30 | grid | depth | 1752 → 763 | **56.5%** |
207
+
208
+ ---
209
+
210
+ ## Production Gap Coverage
211
+
212
+ WQT50M addresses 6 of the 9 production gaps identified in WQT20M-Beta:
213
+
214
+ | Gap | WQT20M-Beta | WQT50M | Status |
215
+ |-----|-------------|--------|--------|
216
+ | **Gap 1**: Real compiler outputs | Synthetic surrogate | Real Qiskit transpilation | ✅ Fixed |
217
+ | **Gap 2**: Value calibration | 6.0–6.6 range | Ratio 0.999, Spearman 0.981 | ✅ Fixed |
218
+ | **Gap 3**: Objective sensitivity | 0% | 65% change rate | ✅ Fixed |
219
+ | **Gap 4**: Legality | 76.9% | 79.2% | ⚠️ Improved |
220
+ | **Gap 5**: Multi-step trajectories | Single-step | 2-5 step sequences | ✅ Fixed |
221
+ | **Gap 6**: Backend awareness | Topology tokens only | 60% change rate | ✅ Fixed |
222
+ | Gap 7: Model capacity | 19M | 49.53M | ✅ Fixed |
223
+ | Gap 8: Search improvement | +35% | 90.4% win rate | ✅ Fixed |
224
+ | Gap 9: Generalization | 96% | Tested on 450 configs | ✅ Fixed |
225
+
226
+ ### Remaining gaps for future models
227
+
228
+ - **Legality accuracy**: 79.2% (target >90%) — needs a dedicated classification head
229
+ - **Multi-step trajectory planning**: Data generated but not yet evaluated end-to-end
230
+ - **Active learning feedback loop**: Not yet implemented
231
+ - **Production deployment as Qiskit plugin**: Not yet implemented
232
+
233
+ ---
234
+
235
+ ## Training Data
236
+
237
+ WQT50M is trained on **real Qiskit transpilation outputs**:
238
+
239
+ | Component | Details |
240
+ |-----------|---------|
241
+ | Circuits | 50,000 (QAOA, Grover, QFT, BV, GHZ, random, Toffoli) |
242
+ | Backends | 6 (linear, ring, grid, heavy-hex, all-to-all, trapped ion) |
243
+ | Strategies | 12 real Qiskit transpilation passes |
244
+ | Objectives | 5 (balanced, 2q-focused, depth-focused, fidelity-focused, time-focused) |
245
+ | Records | 1,799,639 total |
246
+ | Backend calibration | Per-edge error rates, T1/T2, readout errors |
247
+ | Legality | 50/50 balanced with real constraints |
248
+ | Trajectories | 50,000 multi-step optimization sequences |
249
+
250
+ ---
251
+
252
+ ## Coverage
253
+
254
+ 15 quantum optimization domains:
255
+
256
+ | Domain | Example Actions |
257
+ |--------|----------------|
258
+ | Quantum chemistry | UCCSD_ANSATZ, ADAPT_VQE, FROZEN_CORE |
259
+ | Graph optimization | MAXCUT_ROUND, COLOR_GRAPH, SDP_RELAX |
260
+ | Error correction | SYNDROME_MEASURE, DECODE_SURFACE, MAGIC_STATE_DISTILL |
261
+ | Quantum annealing | REVERSE_ANNEAL, MINOR_EMBED, HYBRID_SOLVE |
262
+ | Variational | QAOA_P1/P2/P3, WARM_START, ADAPTIVE_LAYER |
263
+ | Hamiltonian simulation | TROTTER_STEP, LCU_DECOMPOSE, QUBITIZE |
264
+ | Circuit optimization | CANCEL_GATES, ZX_SIMPLIFY, FUSE_ROTATIONS |
265
+ | Hardware mapping | SABRE_ROUTE, PULSE_OPTIMIZE, NOISE_AWARE |
266
+ | Neutral atom | SET_RYDBERG, PULSE_SHAPE, ADIABATIC_PASS |
267
+ | Photonic | KLM_CNOT, CLUSTER_STATE, HERALD |
268
+ | Topological | BRAID_ANYON, FIBONACCI_BRAID |
269
+ | Quantum walk | COINED_WALK, GROVER_COIN, AMPLIFY |
270
+ | State preparation | MPS_PREP, TENSOR_NETWORK, COMPRESS_STATE |
271
+ | Amplitude amplification | GROVER_ITER, QSEARCH, ITERATIVE_QPE |
272
+ | Quantum ML | IQP_KERNEL, DATA_REUPLOADING, FIDELITY_KERNEL |
273
+
274
+ ---
275
+
276
+ ## Limitations
277
+
278
+ - **Legality: 79.2%** — Below the 90% target. The model sometimes marks
279
+ illegal actions as valid.
280
+ - **Not a circuit compiler** — The model works on structured state
281
+ representations, not raw Qiskit/TKET circuits. Plugins translate between
282
+ the model's representation and real frameworks.
283
+ - **Single-step evaluation** — While trajectory data was generated, the
284
+ model has not been evaluated on multi-step planning end-to-end.
285
+ - **No active learning** — The model is static; it does not improve from
286
+ user compilations.
287
+
288
+ ---
289
+
290
+ ## License
291
+
292
+ Apache 2.0
293
+
294
+ ## Citation
295
+
296
+ ```bibtex
297
+ @misc{westquant2026wqt50m,
298
+ title={WQT50M: A 50M-Parameter Transformer for Quantum Representation Scheduling},
299
+ author={Vesterlund, David},
300
+ year={2026},
301
+ url={https://huggingface.co/WestQuantStudio/WQT50M},
302
+ note={WestQuant Open Source Project}
303
+ }
304
+ ```
config.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LlamaForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "dtype": "float32",
9
+ "eos_token_id": 2,
10
+ "head_dim": 64,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 512,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 2048,
15
+ "max_position_embeddings": 4096,
16
+ "mlp_bias": false,
17
+ "model_type": "llama",
18
+ "num_attention_heads": 8,
19
+ "num_hidden_layers": 12,
20
+ "num_key_value_heads": 4,
21
+ "pad_token_id": 0,
22
+ "pretraining_tp": 1,
23
+ "rms_norm_eps": 1e-06,
24
+ "rope_parameters": {
25
+ "rope_theta": 10000.0,
26
+ "rope_type": "default"
27
+ },
28
+ "tie_word_embeddings": true,
29
+ "transformers_version": "5.17.0",
30
+ "use_cache": true,
31
+ "vocab_size": 4561
32
+ }
generation_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "output_attentions": false,
6
+ "output_hidden_states": false,
7
+ "pad_token_id": 0,
8
+ "transformers_version": "5.17.0",
9
+ "use_cache": true
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5db414cb4296e04a1519c1ae551a7b874256563e876fb07a33a14c93cc73c21c
3
+ size 198147936
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "bos_token": "<bos>",
4
+ "eos_token": "<eos>",
5
+ "mask_token": "<mask>",
6
+ "model_max_length": 1000000000000000019884624838656,
7
+ "pad_token": "<pad>",
8
+ "sep_token": "<sep>",
9
+ "tokenizer_class": "TokenizersBackend",
10
+ "unk_token": "<unk>"
11
+ }