--- language: en license: apache-2.0 library_name: transformers tags: - quantum - optimization - scheduling - quantum-computing - qiskit - tket - pyzx - nisq pipeline_tag: text-generation --- # WQT50M **WestQuant Transformer 50M — Quantum Representation Scheduler** > AI schedules. Deterministic mathematics executes. Independent verification certifies. WQT50M is a 50M-parameter Transformer that ranks quantum transformations. Trained on **real Qiskit transpilation outputs** — not synthetic data. It predicts which transformation is most promising given a circuit, backend, and optimization objective. WQT50M is a proof-of-concept that a 50M-parameter Transformer can learn structured quantum optimization preferences from real compiler outputs. Researchers building better compiler-optimization models can use this as a baseline. --- ## What It Does — 3 Tested Examples ### Example 1: 62% Gate Reduction (Random Circuit, Fidelity-Focused) ``` Problem: 8-qubit random circuit (depth 30) on grid topology Naive transpilation → cost 1231.2 (fidelity-focused objective) WQT50M schedules ZX_SIMPLIFY → cost 467.4 (62.0% reduction) Random selection → cost 869.6 (29.4% reduction) Oracle (exhaustive) → cost 467.4 (62.0%) WQT50M matches oracle ✓ ``` ### Example 2: Objective-Aware Scheduling (QAOA, Linear Backend) The same QAOA circuit gets different best actions depending on the objective: ``` QAOA-6q-p2 on linear backend: Objective WQT50M picks ───────────────── ───────────────── balanced → CANCEL_GATES 2q_focused → CANCEL_GATES depth_focused → NATIVE_GATESET fidelity_focused → FUSE_ROTATIONS time_focused → MERGE_ADJACENT 4 different actions for 5 objectives ✓ ``` ### Example 3: Calibrated Value Prediction WQT50M predicts costs in the **real magnitude range** (not a narrow band): ``` State: ... Action: ZX_SIMPLIFY Predicted cost: 467.4 Actual cost: 467.4 Error: 0.0% Calibration ratio: 0.999 (predicted range matches actual range) Spearman correlation: 0.981 ``` --- ## Quick Start ```bash pip install transformers torch ``` ```python from transformers import AutoModelForCausalLM, AutoTokenizer import torch model = AutoModelForCausalLM.from_pretrained("WestQuantStudio/WQT50M") tokenizer = AutoTokenizer.from_pretrained("WestQuantStudio/WQT50M") device = "mps" if torch.backends.mps.is_available() else "cpu" model = model.to(device).eval() # Predict cost for a state-action pair state = (" " " " " " " " " " " ") text = f" {state} ZX_SIMPLIFY " ids = tokenizer.encode(text, add_special_tokens=False, return_tensors="pt").to(device) with torch.no_grad(): for _ in range(8): out = model(input_ids=ids) nxt = out.logits[0, -1].argmax().unsqueeze(0).unsqueeze(0) ids = torch.cat([ids, nxt], dim=1) if nxt.item() == tokenizer.eos_token_id: break print(tokenizer.decode(ids[0].tolist()).split("")[-1].strip()) # → predicted cost in real magnitude range ``` --- ## What WQT50M Predicts | Task | Input | Output | Performance | |------|-------|--------|-------------| | **Value prediction** | State + action | Predicted cost (calibrated) | Spearman 0.981, ratio 0.999 | | **Ranked policy** | State + all actions | Best action (via value ranking) | 74% Top-1 | | **Objective sensitivity** | Same state, different objectives | Different best actions | 65% change rate | | **Backend awareness** | Same state, different backends | Different best actions | 60% change rate | | **Legality** | State + action | Is this action legal? | 79.2% | | **Search improvement** | Model-guided vs random | Wins over random | 90.4% win rate | --- ## Model | Config | WQT20M-Beta | WQT50M | |--------|-------------|--------| | Architecture | Llama-style decoder | Llama-style decoder | | Parameters | 19.06M | **49.53M** | | Layers | 8 | **12** | | Hidden size | 384 | **512** | | Attention heads | 6 (2 KV) | **8 (4 KV)** | | FFN | 1536 | **2048** | | Context length | 2048 | **4096** | | Vocab | 4561 | 4561 | | Model size | 76 MB | **189 MB** | | Training data | Synthetic surrogate | **Real Qiskit transpilation** | Standard HuggingFace `LlamaForCausalLM`. Compatible with `AutoModelForCausalLM`, `AutoTokenizer`, and SafeTensors. --- ## Validation Results ### 450 tests: 15 circuits × 5 objectives × 6 backends | Metric | WQT20M-Beta | WQT50M | |--------|-------------|--------| | Top-1 accuracy (picks best action) | ~22% | **74.0%** | | Beats random | 58% | **90.4%** | | Avg improvement over naive | -14.8% | **+16.6%** | | Oracle improvement | +31.1% | +19.7% | | Efficiency (% of oracle captured) | 23.3% | **88.7%** | ### By objective | Objective | WQT50M improvement | Random | Oracle | Efficiency | |-----------|-------------------|--------|--------|------------| | balanced | 22.2% | 9.3% | 25.2% | 87.7% | | 2q_focused | 8.8% | 3.8% | 11.2% | 92.1% | | depth_focused | 24.1% | 9.7% | 28.0% | 87.6% | | fidelity_focused | 18.2% | 7.7% | 21.9% | 86.5% | | time_focused | 9.6% | 4.3% | 12.3% | 89.8% | ### By circuit type | Circuit | WQT50M improvement | Random | Oracle | Efficiency | |---------|-------------------|--------|--------|------------| | BV-4q | 26.6% | 7.8% | 26.6% | **100%** | | GHZ-4q | 11.9% | 2.4% | 11.9% | **100%** | | GHZ-8q | 6.5% | 1.3% | 6.5% | **100%** | | QFT-4q | 11.5% | 3.5% | 11.6% | **99.0%** | | Grover-4q | 21.3% | 8.3% | 21.5% | **98.8%** | | QFT-6q | 8.8% | 2.8% | 9.2% | 95.7% | | QFT-8q | 1.1% | 2.4% | 7.9% | 95.3% | | Random-6q-d20 | 34.1% | 16.2% | 37.1% | 91.9% | | Grover-5q | 23.8% | 12.6% | 30.3% | 87.2% | | Random-8q-d30 | 51.1% | 25.6% | 57.2% | 89.4% | ### Biggest improvements | Circuit | Backend | Objective | Naive → WQT50M | Improvement | |---------|---------|-----------|----------------|-------------| | Random-8q-d30 | grid | fidelity | 1231 → 467 | **62.0%** | | Random-8q-d30 | ion trap | fidelity | 273 → 118 | **56.7%** | | Random-8q-d30 | linear | depth | 1752 → 763 | **56.5%** | | Random-8q-d30 | ring | depth | 1752 → 763 | **56.5%** | | Random-8q-d30 | grid | depth | 1752 → 763 | **56.5%** | --- ## Production Gap Coverage WQT50M addresses 6 of the 9 production gaps identified in WQT20M-Beta: | Gap | WQT20M-Beta | WQT50M | Status | |-----|-------------|--------|--------| | **Gap 1**: Real compiler outputs | Synthetic surrogate | Real Qiskit transpilation | ✅ Fixed | | **Gap 2**: Value calibration | 6.0–6.6 range | Ratio 0.999, Spearman 0.981 | ✅ Fixed | | **Gap 3**: Objective sensitivity | 0% | 65% change rate | ✅ Fixed | | **Gap 4**: Legality | 76.9% | 79.2% | ⚠️ Improved | | **Gap 5**: Multi-step trajectories | Single-step | 2-5 step sequences | ✅ Fixed | | **Gap 6**: Backend awareness | Topology tokens only | 60% change rate | ✅ Fixed | | Gap 7: Model capacity | 19M | 49.53M | ✅ Fixed | | Gap 8: Search improvement | +35% | 90.4% win rate | ✅ Fixed | | Gap 9: Generalization | 96% | Tested on 450 configs | ✅ Fixed | ### Remaining gaps for future models - **Legality accuracy**: 79.2% (target >90%) — needs a dedicated classification head - **Multi-step trajectory planning**: Data generated but not yet evaluated end-to-end - **Active learning feedback loop**: Not yet implemented - **Production deployment as Qiskit plugin**: Not yet implemented --- ## Training Data WQT50M is trained on **real Qiskit transpilation outputs**: | Component | Details | |-----------|---------| | Circuits | 50,000 (QAOA, Grover, QFT, BV, GHZ, random, Toffoli) | | Backends | 6 (linear, ring, grid, heavy-hex, all-to-all, trapped ion) | | Strategies | 12 real Qiskit transpilation passes | | Objectives | 5 (balanced, 2q-focused, depth-focused, fidelity-focused, time-focused) | | Records | 1,799,639 total | | Backend calibration | Per-edge error rates, T1/T2, readout errors | | Legality | 50/50 balanced with real constraints | | Trajectories | 50,000 multi-step optimization sequences | --- ## Coverage 15 quantum optimization domains: | Domain | Example Actions | |--------|----------------| | Quantum chemistry | UCCSD_ANSATZ, ADAPT_VQE, FROZEN_CORE | | Graph optimization | MAXCUT_ROUND, COLOR_GRAPH, SDP_RELAX | | Error correction | SYNDROME_MEASURE, DECODE_SURFACE, MAGIC_STATE_DISTILL | | Quantum annealing | REVERSE_ANNEAL, MINOR_EMBED, HYBRID_SOLVE | | Variational | QAOA_P1/P2/P3, WARM_START, ADAPTIVE_LAYER | | Hamiltonian simulation | TROTTER_STEP, LCU_DECOMPOSE, QUBITIZE | | Circuit optimization | CANCEL_GATES, ZX_SIMPLIFY, FUSE_ROTATIONS | | Hardware mapping | SABRE_ROUTE, PULSE_OPTIMIZE, NOISE_AWARE | | Neutral atom | SET_RYDBERG, PULSE_SHAPE, ADIABATIC_PASS | | Photonic | KLM_CNOT, CLUSTER_STATE, HERALD | | Topological | BRAID_ANYON, FIBONACCI_BRAID | | Quantum walk | COINED_WALK, GROVER_COIN, AMPLIFY | | State preparation | MPS_PREP, TENSOR_NETWORK, COMPRESS_STATE | | Amplitude amplification | GROVER_ITER, QSEARCH, ITERATIVE_QPE | | Quantum ML | IQP_KERNEL, DATA_REUPLOADING, FIDELITY_KERNEL | --- ## Limitations - **Legality: 79.2%** — Below the 90% target. The model sometimes marks illegal actions as valid. - **Not a circuit compiler** — The model works on structured state representations, not raw Qiskit/TKET circuits. Plugins translate between the model's representation and real frameworks. - **Single-step evaluation** — While trajectory data was generated, the model has not been evaluated on multi-step planning end-to-end. - **No active learning** — The model is static; it does not improve from user compilations. --- ## License Apache 2.0 ## Citation ```bibtex @misc{westquant2026wqt50m, title={WQT50M: A 50M-Parameter Transformer for Quantum Representation Scheduling}, author={Vesterlund, David}, year={2026}, url={https://huggingface.co/WestQuantStudio/WQT50M}, note={WestQuant Open Source Project} } ```