Upload folder using huggingface_hub

README.md

@@ -19,9 +19,14 @@ question here is narrower. Matching is near-optimal when its noise model is righ
 only place a learned decoder can win is where that model is wrong. This measures how much it
 wins by, and at what point it stops.
 
-Three architectures, all within 1.2x of each other on parameter count and given identical
+Four architectures, all within 1.17x of each other on parameter count and given identical
 optimiser, schedule and step budget. Comparing a big model to a small one, or one tuned
-harder than another, measures capacity or patience rather than architecture.
+harder than another, measures capacity or patience rather than architecture. The fourth,
+a recurrent transformer, is a smaller-scale attempt at the specific idea behind Google
+DeepMind's AlphaQubit (Bausch et al., *Nature*, Nov 2024): self-attention within each round
+for correlations spanning the whole lattice, plus a per-stabiliser recurrent state carried
+across rounds by physical coordinate rather than array position (a real bug, caught and
+fixed before it shipped - the coordinate fix has its own test in the repo).
 
 ## Using a checkpoint
 
@@ -99,9 +104,12 @@ One row is not a tie: at distance 3 and p=0.01 the MLP beats matching outright w
 non-overlapping intervals. That was not the expected outcome in the regime designed to
 favour matching.
 
-The architecture ordering also flips with distance. The plain MLP is best at distance 3, the
-graph network at distance 5. The graph prior costs more than it returns until the code is
-large enough for the structure to carry information.
+The architecture ordering also flips with distance among the first three. The plain MLP is
+best at distance 3, the graph network at distance 5. The graph prior costs more than it
+returns until the code is large enough for the structure to carry information. The
+transformer sits behind all three at both distances in this control regime - attention's
+extra reach doesn't pay for itself when matching already has the right noise model, which
+turns out not to be where it matters (see below).
 
 ## Real device: Google Sycamore
 
@@ -111,14 +119,27 @@ and the comparison.
 
 | d | rounds | trivial | pymatching | correlated | best neural | vs pymatching |
 |---|---|---|---|---|---|---|
-| 3 | 5 | 0.29340 | 0.14790 | 0.13760 | 0.15720 | indistinguishable |
-| 3 | 25 | 0.49010 | 0.43020 | 0.42410 | 0.48920 | mwpm |
+| 3 | 5 | 0.29340 | 0.14790 | 0.13760 | 0.14090 | indistinguishable |
+| 3 | 25 | 0.49010 | 0.43020 | 0.42410 | 0.48640 | mwpm |
 | 5 | 5 | 0.38690 | 0.14730 | 0.12750 | 0.26430 | mwpm |
+| 5 | 25 | 0.50520 | 0.43180 | 0.40650 | 0.50520 | mwpm |
 
-These lose, and the 25-round row loses badly enough to be worth stating plainly: 0.489
-against a trivial floor of 0.490 means the model learned essentially nothing. Each experiment
-has 50,000 shots, so 35,000 to train on, against a 200-detector input with a base flip rate
-near 50 percent. That is not enough supervision, and no amount of architecture fixes it.
+Mostly these lose, and one row loses badly enough to be worth stating plainly: at d=5, 25
+rounds the transformer came back at 0.5052, identical to the trivial floor to four decimal
+places. It learned nothing. That configuration was never evaluated with the earlier three
+architectures at all, so there's no prior number to compare against - it's simply the
+hardest one tested, largest distance and most rounds together, and it lost completely.
+
+The d=3, 25-round row is unchanged from before: 0.486 against a trivial floor of 0.490,
+still essentially nothing learned. Each experiment has 50,000 shots, 35,000 to train on,
+against a 200+ detector input with a base flip rate near 50 percent. That is not enough
+supervision, and no architecture fixes a training-data shortage - the transformer doesn't
+either.
+
+The one real exception is d=3, 5 rounds, where the transformer wins outright: 0.1409 against
+pymatching's 0.1479, and against the previous best neural result there, ConvGRU's 0.1572.
+First time any decoder in this repo beats a classical baseline on real hardware. Small and
+specific, not a general result, but real.
 
 It is also the most practically useful thing in this repo. Simulated syndromes are free.
 Hardware shots are not, and the shot budget is what actually limits learned decoders on real
@@ -126,11 +147,14 @@ machines.
 
 ## Scope
 
-Distance 3 and 5, one code, one crosstalk model. The architectures are standard; the
-measurement is the part that is mine. State of the art is AlphaQubit from Google DeepMind,
-which used far more compute and real device data, and nothing here competes with it.
-Regimes A and B are simulated, so leakage and calibration drift are absent. The distance-5
-models remain budget-limited.
+Distance 3 and 5, one code, one crosstalk model. Three of the four architectures are
+standard; the fourth is a smaller-scale attempt at Google DeepMind's AlphaQubit
+(*Nature*, Nov 2024) specifically, and the honest result is mixed - one real win on real
+hardware at the smallest configuration tested, losses everywhere else including one total
+failure. AlphaQubit itself used far more compute and much more real device data than this
+repo has access to; nothing here claims to match it, only to test whether its core
+architectural idea helps at this scale. Regimes A and B are simulated, so leakage and
+calibration drift are absent. The distance-5 models remain budget-limited.
 
 ## License
 

RESULTS.json

@@ -8,6 +8,7 @@
       "mlp": 0.000916,
       "convgru": 0.001246,
       "gnn": 0.001298,
+      "transformer": 0.0016566666666666667,
       "best_neural": "mlp",
       "verdict": "indistinguishable",
       "ratio_mwpm_over_neural": 0.7860262008733625
@@ -20,6 +21,7 @@
       "mlp": 0.006174,
       "convgru": 0.006862,
       "gnn": 0.008296,
+      "transformer": 0.010183333333333334,
       "best_neural": "mlp",
       "verdict": "indistinguishable",
       "ratio_mwpm_over_neural": 1.0382248137350178
@@ -32,6 +34,7 @@
       "mlp": 0.016088,
       "convgru": 0.017218,
       "gnn": 0.01842,
+      "transformer": 0.026606666666666667,
       "best_neural": "mlp",
       "verdict": "indistinguishable",
       "ratio_mwpm_over_neural": 1.03682293583275
@@ -44,6 +47,7 @@
       "mlp": 0.055618,
       "convgru": 0.05758,
       "gnn": 0.062736,
+      "transformer": 0.06750666666666667,
       "best_neural": "mlp",
       "verdict": "neural",
       "ratio_mwpm_over_neural": 1.088876281625477
@@ -56,6 +60,7 @@
       "mlp": 0.003266,
       "convgru": 0.003134,
       "gnn": 0.00262,
+      "transformer": 0.007563333333333333,
       "best_neural": "gnn",
       "verdict": "mwpm",
       "ratio_mwpm_over_neural": 0.03625954198473283
@@ -68,6 +73,7 @@
       "mlp": 0.019996,
       "convgru": 0.016982,
       "gnn": 0.011112,
+      "transformer": 0.04353666666666667,
       "best_neural": "gnn",
       "verdict": "mwpm",
       "ratio_mwpm_over_neural": 0.3041756659467243
@@ -80,6 +86,7 @@
       "mlp": 0.05319,
       "convgru": 0.044748,
       "gnn": 0.037362,
+      "transformer": 0.08963,
       "best_neural": "gnn",
       "verdict": "mwpm",
       "ratio_mwpm_over_neural": 0.37511067819450467
@@ -92,6 +99,7 @@
       "mlp": 0.197032,
       "convgru": 0.173618,
       "gnn": 0.154774,
+      "transformer": 0.23445,
       "best_neural": "gnn",
       "verdict": "mwpm",
       "ratio_mwpm_over_neural": 0.52470383584708
@@ -109,6 +117,7 @@
       "mlp": 0.006174,
       "convgru": 0.00699,
       "gnn": 0.008296,
+      "transformer": 0.010183333333333334,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "indistinguishable",
       "ratio_misspecified_over_neural": 1.0557175251052804,
@@ -126,6 +135,7 @@
       "mlp": 0.015356,
       "convgru": 0.016322,
       "gnn": 0.016928,
+      "transformer": 0.019866666666666668,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "neural",
       "ratio_misspecified_over_neural": 1.846444386559,
@@ -143,6 +153,7 @@
       "mlp": 0.00591,
       "convgru": 0.006378,
       "gnn": 0.007654,
+      "transformer": 0.01044,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "neural",
       "ratio_misspecified_over_neural": 1.1099830795262267,
@@ -160,6 +171,7 @@
       "mlp": 0.02006,
       "convgru": 0.020522,
       "gnn": 0.021386,
+      "transformer": 0.02555,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "neural",
       "ratio_misspecified_over_neural": 1.9687936191425721,
@@ -177,6 +189,7 @@
       "mlp": 0.006458,
       "convgru": 0.006998,
       "gnn": 0.007782,
+      "transformer": 0.011526666666666666,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "neural",
       "ratio_misspecified_over_neural": 1.0870238463920718,
@@ -194,6 +207,7 @@
       "mlp": 0.011954,
       "convgru": 0.012774,
       "gnn": 0.012948,
+      "transformer": 0.01827,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "neural",
       "ratio_misspecified_over_neural": 1.6747532206792706,
@@ -211,6 +225,7 @@
       "mlp": 0.019996,
       "convgru": 0.017774,
       "gnn": 0.011112,
+      "transformer": 0.04353666666666667,
       "best_neural": "gnn",
       "verdict_vs_misspecified": "mwpm",
       "ratio_misspecified_over_neural": 0.3003959683225342,
@@ -228,6 +243,7 @@
       "mlp": 0.037846,
       "convgru": 0.036114,
       "gnn": 0.039408,
+      "transformer": 0.08692666666666667,
       "best_neural": "convgru",
       "verdict_vs_misspecified": "mwpm",
       "ratio_misspecified_over_neural": 0.607576009303871,
@@ -245,6 +261,7 @@
       "mlp": 0.017908,
       "convgru": 0.014108,
       "gnn": 0.010516,
+      "transformer": 0.027483333333333332,
       "best_neural": "gnn",
       "verdict_vs_misspecified": "mwpm",
       "ratio_misspecified_over_neural": 0.3187523773297832,
@@ -262,6 +279,7 @@
       "mlp": 0.041584,
       "convgru": 0.048738,
       "gnn": 0.063192,
+      "transformer": 0.11524333333333334,
       "best_neural": "mlp",
       "verdict_vs_misspecified": "mwpm",
       "ratio_misspecified_over_neural": 0.8598018468641785,
@@ -279,6 +297,7 @@
       "mlp": 0.018724,
       "convgru": 0.015072,
       "gnn": 0.01668,
+      "transformer": 0.031906666666666667,
       "best_neural": "convgru",
       "verdict_vs_misspecified": "mwpm",
       "ratio_misspecified_over_neural": 0.23633227176220806,
@@ -296,6 +315,7 @@
       "mlp": 0.030922,
       "convgru": 0.028544,
       "gnn": 0.032574,
+      "transformer": 0.06131,
       "best_neural": "convgru",
       "verdict_vs_misspecified": "mwpm",
       "ratio_misspecified_over_neural": 0.5039938340807175,
@@ -307,7 +327,7 @@
     {
       "d": 3,
       "rounds": 5,
-      "best_neural": "convgru",
+      "best_neural": "transformer",
       "mlp": 0.1828,
       "convgru": 0.1572,
       "gnn": 0.1593,
@@ -315,12 +335,13 @@
       "correlated_matching": 0.1376,
       "tensor_network": 0.1252,
       "trivial": 0.2934,
+      "transformer": 0.1409,
       "verdict_vs_pymatching": "indistinguishable"
     },
     {
       "d": 3,
       "rounds": 25,
-      "best_neural": "convgru",
+      "best_neural": "transformer",
       "mlp": 0.4908,
       "convgru": 0.4892,
       "gnn": 0.4898,
@@ -328,6 +349,7 @@
       "correlated_matching": 0.4241,
       "tensor_network": 0.4052,
       "trivial": 0.4901,
+      "transformer": 0.4864,
       "verdict_vs_pymatching": "mwpm"
     },
     {
@@ -341,13 +363,25 @@
       "correlated_matching": 0.1275,
       "tensor_network": 0.106,
       "trivial": 0.3869,
+      "transformer": 0.3892,
+      "verdict_vs_pymatching": "mwpm"
+    },
+    {
+      "d": 5,
+      "rounds": 25,
+      "best_neural": "transformer",
+      "transformer": 0.5052,
+      "pymatching": 0.4318,
+      "correlated_matching": 0.4065,
+      "tensor_network": 0.3804,
+      "trivial": 0.5052,
       "verdict_vs_pymatching": "mwpm"
     }
   ],
   "notes": [],
   "completeness": {
-    "regime_a": "24/24",
-    "regime_b": "72/72",
-    "device": "21/24"
+    "regime_a": "32/32",
+    "regime_b": "84/84",
+    "device": "29 recorded"
   }
 }

checkpoints/transformer_A_d3_r3_p0.001.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:268f4768d631de28e7d1074607c5a6dfe80c6b522bd3af32b3df111aac4b1c74
+size 630117

checkpoints/transformer_A_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57bb128ac4b7a16d3c3a5b9371a82f6db23b09777e9d29af3881aba530810375
+size 630117

checkpoints/transformer_A_d3_r3_p0.005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63a706a09ca18270792a64d943f6ac2e1631fb82d423ce2c6d478b3d1e6fcf1f
+size 630117

checkpoints/transformer_A_d3_r3_p0.01.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd34a184fdaa20485a7e07bb603feb1de9c5c094fe4011ec2c11e853cbe1c63a
+size 630074

checkpoints/transformer_A_d5_r5_p0.001.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8920a7d07f012279aedd0e45ae583e7f22b2d9ef9bcec6ecbdb6522e5c2aeeb3
+size 636709

checkpoints/transformer_A_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dca08e7cf310fc6466ed09f1a4441cd8c7f57f78713ef6f7346b6d425955fc4
+size 636709

checkpoints/transformer_A_d5_r5_p0.005.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd67c27cff21ce02bdce3a0bc140a672277da6e2e0c9b9e7bbf5f56d7e1cabed
+size 636709

checkpoints/transformer_A_d5_r5_p0.01.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:653713453fd73f7b2db324e25acfa12af83d93f19dc220dd7f093d7fc0a32e26
+size 636666

checkpoints/transformer_s0_x0.0005_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a691c060c8e9528d6db1a3576eb6fbca2cfbdd26347d8a3acae054f3d00d11d
+size 630504

checkpoints/transformer_s0_x0.0005_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:092ed5b22091c5a94241ebe637432a9f854d080b819b2d240948e53c0b1fd4ae
+size 637096

checkpoints/transformer_s0_x0_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed8067828fa0890971019620f508635fdddd290fb9d3771db022f2656337e557
+size 630289

checkpoints/transformer_s0_x0_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc373e157e3aed61e82af269ef46e3bb25bb587e8fd48b7d062965c0cec85aea
+size 636881

checkpoints/transformer_s1.75_x0.0015_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ed6f78a19dc299f7ab68a4ed268459bc70d516d7b541828fe3fbbf1e88c2b32
+size 630633

checkpoints/transformer_s1.75_x0.0015_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98900647cd5ecfe241428258ad477fbf5d12c2c5e6fe74b8ded06acd9c127aa4
+size 637225

checkpoints/transformer_s1.75_x0_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b7dc5de7226a78d115d3f41e687291e8db5153cc9a6dc29196a03436f8e7643
+size 630418

checkpoints/transformer_s1.75_x0_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b85bebcbdb9a42c3b88624d537387610676dee2edce96f7e38eac81523e8e33
+size 637010

checkpoints/transformer_s1_x0.0005_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cfb5071be568f0efc19589a58b85dcaf6c6b45d99bb4289138bbccbf78f6b70
+size 630504

checkpoints/transformer_s1_x0.0005_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d351ea3a2630eec2e3b13310f7f6518ece0b07de2159532ea8f1c86c60421fc
+size 637096

checkpoints/transformer_s1_x0_d3_r3_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8b721c709ac623c3e2d471496f745771f98953e24607ac03588880b92b88c34
+size 630289

checkpoints/transformer_s1_x0_d5_r5_p0.003.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b3483126845b6fed0bfa8bdc54b7e4fd1ba4edc47482b4b76e1a994831ae0ba
+size 636881