WQT50M-NG: WestQuant Transformer for Quantum Mathematical Optimization

Model Description

WQT50M-NG (Next-Generation) is a 51.21M-parameter Transformer model trained to predict the effect of quantum circuit transformations. It replaces the original WQT50M which used log-likelihood scoring and performed worse than random on the Paper A benchmark.

Key Improvements over WQT50M

1. Value prediction head (MSE loss) instead of log-likelihood scoring
2. Concrete action parameters (n_qubits, circuit_family, seed, basis_gates, ...) instead of abstract names
3. Verification-aware training (4-class head: exact, property_verified, unknown, invalid)
4. Seed sensitivity (opt_seed included in action parameters)
5. Action diversity (no single generator > 25% of training data)

Architecture

• Backbone: Llama-style Transformer (12 layers, 512 hidden, 8 heads, 4 KV heads)
• Value head: 9 metric deltas (G, G2, D, D2, T, TD, P, M, Cs)
• Verification head: 4 classes (exact, property_verified, unknown, invalid)
• Auxiliary LM head: next-token prediction (weight 0.1)
• Parameters: 51.21M

Training Data

• Dataset: WQT-Math-10M (11,707,371 records)
• Generators: 9 generator families
• Representations: circuit, zx, clifford_t, hamiltonian
• Splits: structural by semantic_root_id (train 80%, val 5%, iid_test 5%, ood_test 10%)

Generator	Records	Representation
legacy_projection	300K	circuit
circuit_transform	1.5M	circuit
equivalence	849K	circuit
zx_calculus	261K	zx
clifford_t	600K	clifford_t
hamiltonian	1.5M	hamiltonian
trajectory	5.0M	circuit
representation_switch	300K	multi
negative	1.4M	circuit

Training Results

Loss	Final Value
Total	0.0082
Value (MSE)	0.0000
Verification (CE)	0.0000
Aux LM (CE)	0.0814

Evaluation

Generator	Value MSE	Verification Accuracy
legacy_projection	0.000001	100%
circuit_transform	0.001151	100%
trajectory	0.000247	100%
hamiltonian	0.000062	33%
zx_calculus	0.001818	33%
clifford_t	0.045511	100%
representation_switch	0.030079	100%
negative	0.025129	0%
equivalence	0.082358	0%

Usage

import torch
from safetensors.torch import load_file
import json

# Load config
with open('config.json') as f:
    config = json.load(f)

# Load model weights
state_dict = load_file('model.safetensors')

# The model predicts:
# - value_pred: [9] metric deltas (G, G2, D, D2, T, TD, P, M, Cs)
# - verification_logits: [4] verification class logits
# - lm_logits: [seq_len, vocab_size] auxiliary LM logits

Intended Use

This model is designed for:

• Predicting the effect of quantum circuit transformations
• Ranking transformation actions by predicted quality
• Verification status prediction
• Research on quantum compiler optimization

Limitations

• Verification accuracy is low for equivalence and negative examples (0%)
• Clifford+T representation has higher prediction error
• Trained on synthetic data, not real QPU outputs
• No timeout/resource-awareness (future work)

Citation

@misc{wqt50m_ng_2026,
  title={WQT50M-NG: Value-Prediction Transformer for Quantum Mathematical Optimization},
  author={WestQuant},
  year={2026},
  url={https://huggingface.co/WestQuantStudio/WQT50M-NG}
}

License

Apache 2.0