QEC Decoder: 4 submissions from sdoygb, the board shows only the 2nd (0.0127) — later entries missing
#2
by sdoygb - opened
Hi organizers — I want to flag a leaderboard inconsistency on the Season 2 QEC board.
Participant: sdoygb. I have made four submissions on this board. The leaderboard currently shows one entry:
0.0127, method "Tesseract decoder (det_beam=5, merge_errors) on the public decomposed DEM", rank 7.
That corresponds to my second submission.
The four submissions, in order (times approximate, KST):
| # | what was submitted | note |
|---|---|---|
| 1 | MWPM (pymatching 2.4.0) on the public DEM | 0.0127 — equal to the MWPM baseline anchor |
| 2 | Tesseract decode | displayed now as 0.0127 |
| 3 | Tesseract (det_beam=5, det_penalty=0.5, merge_errors off) | file final2_qec_predictions.json, 150,001 B, sha256 08b5678cb5c17f54; local self-test 0.00770 (20,000 shots) / 0.00750 (8,000 shots) |
| 4 | same 10,000 predictions as #3, pretty-printed | file final3_qec_predictions_pretty.json, 180,002 B, sha256 413e1e3abfc30d44 (parses fine, 10,000 keys) |
Two things look wrong to me:
- #2 is labelled "Tesseract" but scores exactly 0.0127, which is the MWPM baseline value. A Tesseract decode of this DEM gives ~0.0077 in our local tests, so the scored entry and the method label do not seem to come from the same file.
- #3 and #4 are not shown at all. Under the stated rule ("the leaderboard shows your best score, not the most recent"), a 0.0077 entry should have replaced 0.0127.
Could you please:
- check the submission log for participant sdoygb on the QEC board: which of the four were received, and what LER each scored;
- if #3/#4 were received, re-score them against the private ground truth and display the best;
- tell me whether a later, worse-scoring submission is supposed to leave an earlier better one in place (that would explain #2 being shown, but not why #3/#4 are absent).
The queue counter shows {"submitted": 16, "ranked": 9} for the QEC board right now, which suggests several received entries are not ranked — mine may be among them.
I can provide the exact files (same sha256) on request.
Thanks!