Publish verified AbstractGym checkpoints and model cards

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 AbstractGym contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,64 @@
+---
+license: mit
+language: en
+library_name: pytorch
+tags:
+- abstractgym
+- research
+- transformers
+---
+
+# AbstractGym tiny controllers
+
+Twelve independently trained small Transformers: role-mapped step and direct-answer controls (A6), a separate role-mapped full-trace model (X4), and a raw-symbol step ablation (X6). Seeds 0, 1, 2 for each family. These are full model weights, not adapters. Each folder has model.safetensors, config.json, training.json, results.json, and a conversion verification record.
+
+## Input representation and split
+
+The step, direct, and trace controllers receive symbols already mapped to roles. The harness supplies this abstraction; perfect step execution does not demonstrate learned raw-symbol binding. **Exception: raw-step (X6)** receives printable raw-symbol IDs and a supplied symbol map; it was explicitly designed to remove role preprocessing.
+
+A6 development uses 367 trajectories on uvwxy, depths 0–4. Step training deduplicates the prefix states to 36 role observations; direct training uses all 367 membership examples with balanced label sampling. Trace training uses the same development trajectories with balanced membership sampling. Raw-step trains on the 36 raw observations of the development symbol map. The original held-out set has 92 cases across two different alphabets at depths 0/1/2/3/4/8/16. Step models also have 32 extra cases at depths 32/64 and an exhaustive 36-state check. Raw-step evaluates 108 states across the three supplied maps.
+
+## Architecture and optimization
+
+All: two Transformer layers, width 64, four heads, feed-forward width 128, GELU, dropout zero, pre-norm, sinusoidal positions (maximum 256), FP32. Step/direct use bidirectional attention and CLS classification; trace/raw-step use causal attention and autoregressive outputs. AdamW LR .001, weight decay .01, clipping 1, cosine decay to 1e-5. Step: 800 full-batch updates on 36 states. Direct/trace: 1,500 updates, balanced batch 64. Raw-step: 800 full-batch updates on 36 states. Exact per-seed metadata and vocabularies are supplied.
+
+## Exact held-out results
+
+| Family | Seed | Correct / cases |
+| --- | --- | --- |
+| step | 0 | 92/92 |
+| step | 1 | 92/92 |
+| step | 2 | 92/92 |
+| direct | 0 | 76/92 |
+| direct | 1 | 80/92 |
+| direct | 2 | 76/92 |
+| trace | 0 | 60/92 |
+| trace | 1 | 60/92 |
+| trace | 2 | 60/92 |
+| raw-step | 0 | 0/92 |
+| raw-step | 1 | 0/92 |
+| raw-step | 2 | 0/92 |
+
+Step/raw-step scores require complete live execution; direct scores are binary membership; trace scores require the entire generated action sequence. These are different tasks and interfaces. All failures remain in the denominator. Results are copied from `docs/results/2026-10-03-a6`, X4 metadata, and `docs/results/2026-10-04-x6`. Detailed auxiliary results remain in each folder.
+
+## Conversion and loading
+
+Original local state dictionaries were checked against each run’s frozen.json, the tiny runs’ original checkpoint hash manifest. Tensor values remain bitwise identical after safetensors serialization. Reloading into the original architecture reproduced every saved evaluation prediction, including teacher-forced trace outputs, corrupted-history continuations, exhaustive states, and extra-depth step trajectories. No training or evaluation selection was performed during conversion. See conversion_verification.json for exact counts and hashes.
+
+Clone [the public code on main](https://github.com/flavianv/abstractgym-public/tree/main), install PyTorch and safetensors, and set `PYTHONPATH=src:scripts`:
+
+```python
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+from abstractgym.a6_model import TinyTransformer
+path = hf_hub_download("flavianv/abstractgym-tiny-controllers", "step/seed-0/model.safetensors")
+model = TinyTransformer()
+model.load_state_dict(load_file(path), strict=True)
+model.eval()
+```
+
+For direct use `TinyTransformer(outputs=2)`; for trace import `TraceTransformer` from `run_a6_tiny_trace`; for raw-step import `RawTransformer` from `run_a6_x6`. Use the matching encoder and vocabulary recorded in config.json and implemented in the public source. These are custom PyTorch models, not root-level AutoModel checkpoints.
+
+## Limitations and license
+
+Three seeds per family. The role-mapped models depend on privileged preprocessing, and the raw-symbol ablation does not transfer reliably. Synthetic finite tests, bounded positions, and an external state machine do not prove general computation. MIT license. No pickle files or optimizer states are published. Code: [flavianv/abstractgym-public, main](https://github.com/flavianv/abstractgym-public/tree/main).

direct/seed-0/config.json

@@ -0,0 +1,36 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "direct",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "abstractgym.a6_model.TinyTransformer",
+  "outputs": 2,
+  "position_limit": 256,
+  "seed": 0,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "CENTER",
+    "END",
+    "EMPTY",
+    "DIRECT"
+  ],
+  "width": 64
+}

direct/seed-0/conversion_verification.json

@@ -0,0 +1,12 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "direct",
+  "published_sha256": "387fc671b5ee2669d371f3f10094fba55c8c5f7e31d027065d2c8dcc977c455a",
+  "saved_predictions_reproduced": {
+    "held_out": 92
+  },
+  "seed": 0,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "39bf28b49785115db3ca0b58a584230aff6db09361ee79619ba6e7fca8bcb8cd",
+  "torch_version": "2.14.1"
+}

direct/seed-0/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:387fc671b5ee2669d371f3f10094fba55c8c5f7e31d027065d2c8dcc977c455a
+size 340216

direct/seed-0/results.json

@@ -0,0 +1,8 @@
+{
+  "direct_answer": [
+    76,
+    92
+  ],
+  "direct_answer_model": "separate matched architecture; end-to-end development-label training",
+  "seed": 0
+}

direct/seed-0/training.json

@@ -0,0 +1,63 @@
+{
+  "plan": {
+    "architecture": {
+      "attention": "bidirectional encoder, CLS readout",
+      "dropout": 0,
+      "feedforward": 128,
+      "heads": 4,
+      "layers": 2,
+      "position": "fixed sinusoidal",
+      "width": 64
+    },
+    "device": "cpu",
+    "dtype": "float32",
+    "git_revision": "39bca8d1f7dd0324010f76d6eb44a68ea719d572",
+    "hardware": "arm",
+    "optimizer": {
+      "clip_norm": 1.0,
+      "lr": 0.001,
+      "name": "AdamW",
+      "schedule": "cosine to 1e-5",
+      "weight_decay": 0.01
+    },
+    "packages": {
+      "torch": "2.14.1"
+    },
+    "platform": "macOS-26.7-arm64-arm-64bit",
+    "python": "3.12.13",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_tiny.py": "62a632c65003a10c5f35b1a586f7a62cfae3605db0c39d9ae48914367ea7aa1e",
+      "src/abstractgym/a6.py": "975b3d76fc9503e11e8f0ad8c57aa27162250298d4316d949145e6df80dbbfe2",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/brackets.py": "9fa8b835c0aac8fa26177b032f1fc07e1bb0964f7ec01fd0f83ad11b74e0550f"
+    },
+    "threads": 2,
+    "training": {
+      "deduplicated_step_examples": 36,
+      "direct_examples": 367,
+      "direct_positive": 31,
+      "direct_sampling": "balanced labels, replacement; separate matched model",
+      "max_depth": 4,
+      "raw_step_examples": 2305,
+      "symbols": "uvwxy",
+      "trajectories": 367
+    },
+    "training_hash": "8fa8728efdda11e741e8ec16c2e0267cf0fd321e849119caea177055d15e5df0"
+  },
+  "training": {
+    "batch_size": 64,
+    "final_loss": 0.005661265458911657,
+    "parameters": 67970,
+    "seconds": 5.530814791000012,
+    "steps": 1500,
+    "train_accuracy": [
+      366,
+      367
+    ]
+  }
+}

direct/seed-1/config.json

@@ -0,0 +1,36 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "direct",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "abstractgym.a6_model.TinyTransformer",
+  "outputs": 2,
+  "position_limit": 256,
+  "seed": 1,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "CENTER",
+    "END",
+    "EMPTY",
+    "DIRECT"
+  ],
+  "width": 64
+}

direct/seed-1/conversion_verification.json

@@ -0,0 +1,12 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "direct",
+  "published_sha256": "540f302b732386c378e521a5ff110a28fd056686ba65d23416cf0031017a54ae",
+  "saved_predictions_reproduced": {
+    "held_out": 92
+  },
+  "seed": 1,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "07445f6621896c71053a5e64ba61b78c430954a840575bba113e7aeaef462a43",
+  "torch_version": "2.14.1"
+}

direct/seed-1/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:540f302b732386c378e521a5ff110a28fd056686ba65d23416cf0031017a54ae
+size 340216

direct/seed-1/results.json

@@ -0,0 +1,8 @@
+{
+  "direct_answer": [
+    80,
+    92
+  ],
+  "direct_answer_model": "separate matched architecture; end-to-end development-label training",
+  "seed": 1
+}

direct/seed-1/training.json

@@ -0,0 +1,63 @@
+{
+  "plan": {
+    "architecture": {
+      "attention": "bidirectional encoder, CLS readout",
+      "dropout": 0,
+      "feedforward": 128,
+      "heads": 4,
+      "layers": 2,
+      "position": "fixed sinusoidal",
+      "width": 64
+    },
+    "device": "cpu",
+    "dtype": "float32",
+    "git_revision": "39bca8d1f7dd0324010f76d6eb44a68ea719d572",
+    "hardware": "arm",
+    "optimizer": {
+      "clip_norm": 1.0,
+      "lr": 0.001,
+      "name": "AdamW",
+      "schedule": "cosine to 1e-5",
+      "weight_decay": 0.01
+    },
+    "packages": {
+      "torch": "2.14.1"
+    },
+    "platform": "macOS-26.7-arm64-arm-64bit",
+    "python": "3.12.13",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_tiny.py": "62a632c65003a10c5f35b1a586f7a62cfae3605db0c39d9ae48914367ea7aa1e",
+      "src/abstractgym/a6.py": "975b3d76fc9503e11e8f0ad8c57aa27162250298d4316d949145e6df80dbbfe2",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/brackets.py": "9fa8b835c0aac8fa26177b032f1fc07e1bb0964f7ec01fd0f83ad11b74e0550f"
+    },
+    "threads": 2,
+    "training": {
+      "deduplicated_step_examples": 36,
+      "direct_examples": 367,
+      "direct_positive": 31,
+      "direct_sampling": "balanced labels, replacement; separate matched model",
+      "max_depth": 4,
+      "raw_step_examples": 2305,
+      "symbols": "uvwxy",
+      "trajectories": 367
+    },
+    "training_hash": "8fa8728efdda11e741e8ec16c2e0267cf0fd321e849119caea177055d15e5df0"
+  },
+  "training": {
+    "batch_size": 64,
+    "final_loss": 0.00023508977028541267,
+    "parameters": 67970,
+    "seconds": 5.903513042001578,
+    "steps": 1500,
+    "train_accuracy": [
+      367,
+      367
+    ]
+  }
+}

direct/seed-2/config.json

@@ -0,0 +1,36 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "direct",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "abstractgym.a6_model.TinyTransformer",
+  "outputs": 2,
+  "position_limit": 256,
+  "seed": 2,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "CENTER",
+    "END",
+    "EMPTY",
+    "DIRECT"
+  ],
+  "width": 64
+}

direct/seed-2/conversion_verification.json

@@ -0,0 +1,12 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "direct",
+  "published_sha256": "cf8c716286a9d5abc063a423e3c44c379e1b7e132b28108a6a51a683c0e6ae3f",
+  "saved_predictions_reproduced": {
+    "held_out": 92
+  },
+  "seed": 2,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "5e14beeb2d5e742967d56c86677e37a4b0b8d2a1097d3c1d50f78d71fe29a782",
+  "torch_version": "2.14.1"
+}

direct/seed-2/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf8c716286a9d5abc063a423e3c44c379e1b7e132b28108a6a51a683c0e6ae3f
+size 340216

direct/seed-2/results.json

@@ -0,0 +1,8 @@
+{
+  "direct_answer": [
+    76,
+    92
+  ],
+  "direct_answer_model": "separate matched architecture; end-to-end development-label training",
+  "seed": 2
+}

direct/seed-2/training.json

@@ -0,0 +1,63 @@
+{
+  "plan": {
+    "architecture": {
+      "attention": "bidirectional encoder, CLS readout",
+      "dropout": 0,
+      "feedforward": 128,
+      "heads": 4,
+      "layers": 2,
+      "position": "fixed sinusoidal",
+      "width": 64
+    },
+    "device": "cpu",
+    "dtype": "float32",
+    "git_revision": "39bca8d1f7dd0324010f76d6eb44a68ea719d572",
+    "hardware": "arm",
+    "optimizer": {
+      "clip_norm": 1.0,
+      "lr": 0.001,
+      "name": "AdamW",
+      "schedule": "cosine to 1e-5",
+      "weight_decay": 0.01
+    },
+    "packages": {
+      "torch": "2.14.1"
+    },
+    "platform": "macOS-26.7-arm64-arm-64bit",
+    "python": "3.12.13",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_tiny.py": "62a632c65003a10c5f35b1a586f7a62cfae3605db0c39d9ae48914367ea7aa1e",
+      "src/abstractgym/a6.py": "975b3d76fc9503e11e8f0ad8c57aa27162250298d4316d949145e6df80dbbfe2",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/brackets.py": "9fa8b835c0aac8fa26177b032f1fc07e1bb0964f7ec01fd0f83ad11b74e0550f"
+    },
+    "threads": 2,
+    "training": {
+      "deduplicated_step_examples": 36,
+      "direct_examples": 367,
+      "direct_positive": 31,
+      "direct_sampling": "balanced labels, replacement; separate matched model",
+      "max_depth": 4,
+      "raw_step_examples": 2305,
+      "symbols": "uvwxy",
+      "trajectories": 367
+    },
+    "training_hash": "8fa8728efdda11e741e8ec16c2e0267cf0fd321e849119caea177055d15e5df0"
+  },
+  "training": {
+    "batch_size": 64,
+    "final_loss": 0.00016993776080198586,
+    "parameters": 67970,
+    "seconds": 5.7849729169975035,
+    "steps": 1500,
+    "train_accuracy": [
+      367,
+      367
+    ]
+  }
+}

manifest.json

@@ -0,0 +1,250 @@
+{
+  "LICENSE": {
+    "bytes": 1081,
+    "sha256": "36a8634aa63459be88ffbc508c73cb06fa8cfb8d050db23f0e7fd01f55ec3abb"
+  },
+  "README.md": {
+    "bytes": 4658,
+    "sha256": "20853e0339b311e29032cf8a620e511c2b81f00344853f5790c2df0fa2694dfe"
+  },
+  "direct/seed-0/config.json": {
+    "bytes": 549,
+    "sha256": "dcba247b7ba2d2b0c8772e9b4361c83953393915a1b865380f85f07ff872c06e"
+  },
+  "direct/seed-0/conversion_verification.json": {
+    "bytes": 382,
+    "sha256": "cf10ca3f07575269d0ead4ae48cf2c5616de439aea32ae3e7df21976d1a608ab"
+  },
+  "direct/seed-0/model.safetensors": {
+    "bytes": 340216,
+    "sha256": "387fc671b5ee2669d371f3f10094fba55c8c5f7e31d027065d2c8dcc977c455a"
+  },
+  "direct/seed-0/results.json": {
+    "bytes": 154,
+    "sha256": "69d311d44e0b33220d53b6c0ca918f90a190a97d49d7f5204380cb965475b8e8"
+  },
+  "direct/seed-0/training.json": {
+    "bytes": 1768,
+    "sha256": "7c77c85e0c32ad147ac294df837d014eba5ea70d590831e8fc908c77449059da"
+  },
+  "direct/seed-1/config.json": {
+    "bytes": 549,
+    "sha256": "53a9951238e89616cd290e76fe18284c34fe93f327030686a5df15893d871bcc"
+  },
+  "direct/seed-1/conversion_verification.json": {
+    "bytes": 382,
+    "sha256": "81d3787bc5f7b1b724202ba2df40ebc62edbd9dcb73a4c146b394b6c3150e46d"
+  },
+  "direct/seed-1/model.safetensors": {
+    "bytes": 340216,
+    "sha256": "540f302b732386c378e521a5ff110a28fd056686ba65d23416cf0031017a54ae"
+  },
+  "direct/seed-1/results.json": {
+    "bytes": 154,
+    "sha256": "a4824c54db8b4f93cf0d23db2ef813758dc7d60780a02f0b53b5223f86e6124b"
+  },
+  "direct/seed-1/training.json": {
+    "bytes": 1770,
+    "sha256": "c0a1329d04e5ecabdd4d773ed8dca3ffb2f24d0a15880bd16ea2aab3dc58534c"
+  },
+  "direct/seed-2/config.json": {
+    "bytes": 549,
+    "sha256": "33844a3e45bf7b19f265ba2d94752c0abf06dfbf174ef0681eb14cbf5e75dc37"
+  },
+  "direct/seed-2/conversion_verification.json": {
+    "bytes": 382,
+    "sha256": "23363eb35375ff6fab17673a2f855ff86c13cf88a611c976c7adf4ed4ac08770"
+  },
+  "direct/seed-2/model.safetensors": {
+    "bytes": 340216,
+    "sha256": "cf8c716286a9d5abc063a423e3c44c379e1b7e132b28108a6a51a683c0e6ae3f"
+  },
+  "direct/seed-2/results.json": {
+    "bytes": 154,
+    "sha256": "6603ce84ea30162446f5d7237a689224efc0f5b7bedbfb4555839f776980bb7c"
+  },
+  "direct/seed-2/training.json": {
+    "bytes": 1771,
+    "sha256": "da825931beab887b09652975f0b5ca15adc15e2059641870e9335a0b0615d8fa"
+  },
+  "raw-step/seed-0/config.json": {
+    "bytes": 1399,
+    "sha256": "8bd04099a71d6ca57d29c38e569ba5aa3ee66b6b648e492cfa2e6c2ad2112d78"
+  },
+  "raw-step/seed-0/conversion_verification.json": {
+    "bytes": 407,
+    "sha256": "03a5f3006fbc8c9e0c0c602bc3e251819c7ae49e2f57a19089fd0f36ba10b1ad"
+  },
+  "raw-step/seed-0/model.safetensors": {
+    "bytes": 392360,
+    "sha256": "88a2b23e78cbd8ef1c0f09bd9972029827dc9529d49284327b5f83318827f21d"
+  },
+  "raw-step/seed-0/results.json": {
+    "bytes": 4543,
+    "sha256": "27921d9996d95393d8cef298e06f2749c0732ae31febc5d09fb7abc33257c98e"
+  },
+  "raw-step/seed-0/training.json": {
+    "bytes": 2480,
+    "sha256": "89ac1dfbd738c8f575f00832270ff9bee8ddd8046bf17a2563c6abf628cb7dab"
+  },
+  "raw-step/seed-1/config.json": {
+    "bytes": 1399,
+    "sha256": "bb4442cbae671722b0bae353850795bdfb51ea9940a6d875421044e8d9f7a5c9"
+  },
+  "raw-step/seed-1/conversion_verification.json": {
+    "bytes": 407,
+    "sha256": "fa2739568210c1fbe541c02c3ede130fbabe093649e4aeaaf32fffb7fae3ba96"
+  },
+  "raw-step/seed-1/model.safetensors": {
+    "bytes": 392360,
+    "sha256": "c0d4940618fce954f253a157d7167b27fabd561982dc73d174be29ab02c1278a"
+  },
+  "raw-step/seed-1/results.json": {
+    "bytes": 4543,
+    "sha256": "58f3793ed88faa59c31a2a84986b39d69a537fa4306d6c9ddadc02f90ac79ab3"
+  },
+  "raw-step/seed-1/training.json": {
+    "bytes": 2480,
+    "sha256": "9559146efcde113032b8d4352e8d8b3118b909bcee7ef95ac2a1c76367733fa9"
+  },
+  "raw-step/seed-2/config.json": {
+    "bytes": 1399,
+    "sha256": "890e3426a85bd3a9e6f2c7daf85c04caa81201a2e4947784e01e3fe3f8e6d107"
+  },
+  "raw-step/seed-2/conversion_verification.json": {
+    "bytes": 407,
+    "sha256": "ca5d50a980943cf30431fda32ef49507606577dd2c04f252a8512533e996a044"
+  },
+  "raw-step/seed-2/model.safetensors": {
+    "bytes": 392360,
+    "sha256": "a96e32f0a540aaf176a0ec8479c3f9b18c9a43fbea392f1049ea028f3f229f87"
+  },
+  "raw-step/seed-2/results.json": {
+    "bytes": 4542,
+    "sha256": "b1c01769f6431e6663c7f2876dd4f48278d19cff2df6d18c5186971d452b8f5e"
+  },
+  "raw-step/seed-2/training.json": {
+    "bytes": 2479,
+    "sha256": "df4ecadf41931b5c4ea2b40f4a36e64bae0208c1f513b820de4b2e51d2621b07"
+  },
+  "step/seed-0/config.json": {
+    "bytes": 547,
+    "sha256": "7eff724be08aeac497e22428c16b4b9819863b566405703e18a734460d69a7c9"
+  },
+  "step/seed-0/conversion_verification.json": {
+    "bytes": 419,
+    "sha256": "c42d87c2fd15b8e28b5cd3ec405858c1656b4b77709d6638e3f0fe0ef93cefd0"
+  },
+  "step/seed-0/model.safetensors": {
+    "bytes": 341256,
+    "sha256": "baee7f0d12b2a70b1e9604c2d7d9a80554f156a2af111d3966f37cea9f21357f"
+  },
+  "step/seed-0/results.json": {
+    "bytes": 5108,
+    "sha256": "d2c1bf14a8b50ae7b2432472e0534d87239f4c67bf2ccadc4ea527e079f98cc3"
+  },
+  "step/seed-0/training.json": {
+    "bytes": 1767,
+    "sha256": "cfdbe38e6ab8be1dc283d71e55ee7871bdd90f6aad37db17ff01ac05800b3bb2"
+  },
+  "step/seed-1/config.json": {
+    "bytes": 547,
+    "sha256": "cc33223113373fba31603162fd9435927cf54b967ebb3a7a1f1c0cf3c5f2773e"
+  },
+  "step/seed-1/conversion_verification.json": {
+    "bytes": 419,
+    "sha256": "f023fd87d2e3e9bf4e848b348a0ff7a931555ea37c32131b76fd7f966103c1f2"
+  },
+  "step/seed-1/model.safetensors": {
+    "bytes": 341256,
+    "sha256": "2d25c9513d2378fa151f79d4cb4619a2e0d401f7825bd7b6c5a6c13769b37f5b"
+  },
+  "step/seed-1/results.json": {
+    "bytes": 5107,
+    "sha256": "11e1f39bbbae97bb43a3aa607f03de17873ffb36ec461b2d83dc30e71bfe19f1"
+  },
+  "step/seed-1/training.json": {
+    "bytes": 1766,
+    "sha256": "e506968b02ba32835f5d89e274b02f68f58a7af38144c475c4e8d5dc7a2777cf"
+  },
+  "step/seed-2/config.json": {
+    "bytes": 547,
+    "sha256": "cb49d14caf8bb7abd182df0c8a447322918fcec9126025be81a40a564767bd62"
+  },
+  "step/seed-2/conversion_verification.json": {
+    "bytes": 419,
+    "sha256": "267889e561e794217c7be6476aea192c39b9481f4dfccccbee2b10bc72017963"
+  },
+  "step/seed-2/model.safetensors": {
+    "bytes": 341256,
+    "sha256": "7a2d655c83a05f98193abb506ab5e7d181d4463fd21a6dbc1934b43cc7abf213"
+  },
+  "step/seed-2/results.json": {
+    "bytes": 5108,
+    "sha256": "f69f7d4971418960aa05301e873dca3d05318f8bc4acb86de97f548ffee7f4ef"
+  },
+  "step/seed-2/training.json": {
+    "bytes": 1767,
+    "sha256": "d9144554c88c324759c5e9783da13ce97adf40bcc6d39b1e6f95eee4455a9856"
+  },
+  "trace/seed-0/config.json": {
+    "bytes": 579,
+    "sha256": "ed06ce0b09a9e153f047818e927ae6f81ec16b0f8339454ed37988e877690a89"
+  },
+  "trace/seed-0/conversion_verification.json": {
+    "bytes": 440,
+    "sha256": "00cdf1dc3ca0d73effc171d6645ee875fac173eaf66f26c37e913b829b420f5d"
+  },
+  "trace/seed-0/model.safetensors": {
+    "bytes": 344364,
+    "sha256": "e506bfd329fcf518b60b9d904e29045cbb4521a8ee93fce9ab19884654afe35d"
+  },
+  "trace/seed-0/results.json": {
+    "bytes": 1034,
+    "sha256": "92b7aa49f9981daf61871a4a5c2274d0fb89999fb43051bb518f4914bcec7c4e"
+  },
+  "trace/seed-0/training.json": {
+    "bytes": 1088,
+    "sha256": "76189eed4d919df72f532f1fb70ed012a07b45e515e6d318c0b1be4bc884b4ea"
+  },
+  "trace/seed-1/config.json": {
+    "bytes": 579,
+    "sha256": "fd49cb8f7a732f668be343d75a6994d3a19570798f95079d3e20028d24067a23"
+  },
+  "trace/seed-1/conversion_verification.json": {
+    "bytes": 440,
+    "sha256": "ce9a4cd3e64a7e4404da34e9dff84126b828d68889c0f8515f99ba70f24bdfbd"
+  },
+  "trace/seed-1/model.safetensors": {
+    "bytes": 344364,
+    "sha256": "82b4057ba67f34df95a23f117ded59c80be9c10fbd7d3bf85a5ac36d48dfc299"
+  },
+  "trace/seed-1/results.json": {
+    "bytes": 1034,
+    "sha256": "fcd7d53af8393e8b362bf5ad1abc0ace880f04ed67f131463bb51b8a902f7d06"
+  },
+  "trace/seed-1/training.json": {
+    "bytes": 1089,
+    "sha256": "3c1554a1e0d23cd63b165468f18fbbdf74ac8eb01738abeca3ebdab2483c10be"
+  },
+  "trace/seed-2/config.json": {
+    "bytes": 579,
+    "sha256": "6de2e6f36fb39fad15016276144f0b6682f9dfcd6f1c8067d3a0134aa387ebdf"
+  },
+  "trace/seed-2/conversion_verification.json": {
+    "bytes": 440,
+    "sha256": "7bbbd83b9bd1d4f68cf5d40abed77fcfb83944d49a3ea722052134fcc05222ff"
+  },
+  "trace/seed-2/model.safetensors": {
+    "bytes": 344364,
+    "sha256": "f51b7be3c5995a693bad0e806a28481f7530533fcff44a52590ba99f2652d041"
+  },
+  "trace/seed-2/results.json": {
+    "bytes": 1034,
+    "sha256": "b0e0e1bda3c8a6e08fdee845b23b3d10278f6dfbf5e45551f0e456ac76659094"
+  },
+  "trace/seed-2/training.json": {
+    "bytes": 1089,
+    "sha256": "06e6e70d022c540a2f3556cf1e0e75667903fb681c9ac74b9fa5025779def80c"
+  }
+}

raw-step/seed-0/config.json

@@ -0,0 +1,132 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "raw-step",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "run_a6_x6.RawTransformer",
+  "outputs": 108,
+  "position_limit": 256,
+  "seed": 0,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "END",
+    "EMPTY",
+    "SEP",
+    "EOS",
+    "PUSH",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT",
+    " ",
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    "0",
+    "1",
+    "2",
+    "3",
+    "4",
+    "5",
+    "6",
+    "7",
+    "8",
+    "9",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "A",
+    "B",
+    "C",
+    "D",
+    "E",
+    "F",
+    "G",
+    "H",
+    "I",
+    "J",
+    "K",
+    "L",
+    "M",
+    "N",
+    "O",
+    "P",
+    "Q",
+    "R",
+    "S",
+    "T",
+    "U",
+    "V",
+    "W",
+    "X",
+    "Y",
+    "Z",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "a",
+    "b",
+    "c",
+    "d",
+    "e",
+    "f",
+    "g",
+    "h",
+    "i",
+    "j",
+    "k",
+    "l",
+    "m",
+    "n",
+    "o",
+    "p",
+    "q",
+    "r",
+    "s",
+    "t",
+    "u",
+    "v",
+    "w",
+    "x",
+    "y",
+    "z",
+    "{",
+    "|",
+    "}",
+    "~"
+  ],
+  "width": 64
+}

raw-step/seed-0/conversion_verification.json

@@ -0,0 +1,13 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "raw-step",
+  "published_sha256": "88a2b23e78cbd8ef1c0f09bd9972029827dc9529d49284327b5f83318827f21d",
+  "saved_predictions_reproduced": {
+    "exhaustive": 108,
+    "held_out": 92
+  },
+  "seed": 0,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "95dc9cee6071309f370910fc3ae84c958d4f9656e642f5831b388c76524eab01",
+  "torch_version": "2.14.1"
+}

raw-step/seed-0/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88a2b23e78cbd8ef1c0f09bd9972029827dc9529d49284327b5f83318827f21d
+size 392360

raw-step/seed-0/results.json

@@ -0,0 +1,283 @@
+{
+  "action_accuracy": [
+    0,
+    92
+  ],
+  "condition": "tiny_raw_autoregressive",
+  "exhaustive": {
+    "([)]#": [
+      12,
+      36
+    ],
+    "abcde": [
+      15,
+      36
+    ],
+    "uvwxy": [
+      36,
+      36
+    ]
+  },
+  "first_error_median": 1.0,
+  "held_out": {
+    "brackets": {
+      "by_depth": {
+        "0": [
+          0,
+          2
+        ],
+        "1": [
+          0,
+          4
+        ],
+        "16": [
+          0,
+          8
+        ],
+        "2": [
+          0,
+          8
+        ],
+        "3": [
+          0,
+          8
+        ],
+        "4": [
+          0,
+          8
+        ],
+        "8": [
+          0,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "below label-blind",
+        "1": "below label-blind",
+        "16": "below label-blind",
+        "2": "below label-blind",
+        "3": "below label-blind",
+        "4": "below label-blind",
+        "8": "below label-blind"
+      }
+    },
+    "letters": {
+      "by_depth": {
+        "0": [
+          0,
+          2
+        ],
+        "1": [
+          0,
+          4
+        ],
+        "16": [
+          0,
+          8
+        ],
+        "2": [
+          0,
+          8
+        ],
+        "3": [
+          0,
+          8
+        ],
+        "4": [
+          0,
+          8
+        ],
+        "8": [
+          0,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "below label-blind",
+        "1": "below label-blind",
+        "16": "below label-blind",
+        "2": "below label-blind",
+        "3": "below label-blind",
+        "4": "below label-blind",
+        "8": "below label-blind"
+      }
+    },
+    "total": [
+      0,
+      92
+    ]
+  },
+  "invalid_action_calls": 0,
+  "label_blind": {
+    "always_accept": [
+      46,
+      92
+    ],
+    "always_reject": [
+      46,
+      92
+    ],
+    "by_cell": {
+      "brackets": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      },
+      "letters": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      }
+    }
+  },
+  "push_value_errors": 66,
+  "seed": 0,
+  "training": {
+    "loss": 0.0016285243909806013,
+    "parameters": 81004,
+    "seconds": 2.2738583749996906,
+    "seed": 0,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  },
+  "usage": {
+    "completion_tokens": 0,
+    "prompt_tokens": 0
+  }
+}

raw-step/seed-0/training.json

@@ -0,0 +1,148 @@
+{
+  "plan": {
+    "architecture": "2-layer width-64 4-head Transformer; causal attention; autoregressive operation/value/EOS",
+    "batch": 36,
+    "clip": 1,
+    "domain": "three fixed supplied maps x 2 phases x 6 current symbols/end x 3 closing-symbol/empty tops",
+    "held_out_hash": "d2c7202ea08da83ad5398723a2cc4ce9d3175b7e5f7b84ce40aa529103e0d6e2",
+    "lr": 0.001,
+    "optimizer": "AdamW",
+    "output_limit": 3,
+    "prompt_loss_masked": true,
+    "schedule": "cosine to 1e-5",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_x6.py": "b8ce548d5a55069483b46ed89da10f34bb9a3edf976b361069cd4d11692fd685",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/a6_raw.py": "df9c2be312f2290939871364baca738ac9a849dca855d049e304a6d0bb74e630"
+    },
+    "steps": 800,
+    "torch": "2.14.1",
+    "training_hash": "11d5025e1246025f38e3ea840fd2a065ca73c2751cdf9de5f50005f4be7fa3a7",
+    "vocab": [
+      "PAD",
+      "CLS",
+      "opening",
+      "closing",
+      "END",
+      "EMPTY",
+      "SEP",
+      "EOS",
+      "PUSH",
+      "CENTER",
+      "POP",
+      "ACCEPT",
+      "REJECT",
+      " ",
+      "!",
+      "\"",
+      "#",
+      "$",
+      "%",
+      "&",
+      "'",
+      "(",
+      ")",
+      "*",
+      "+",
+      ",",
+      "-",
+      ".",
+      "<local-path>",
+      "0",
+      "1",
+      "2",
+      "3",
+      "4",
+      "5",
+      "6",
+      "7",
+      "8",
+      "9",
+      ":",
+      ";",
+      "<",
+      "=",
+      ">",
+      "?",
+      "@",
+      "A",
+      "B",
+      "C",
+      "D",
+      "E",
+      "F",
+      "G",
+      "H",
+      "I",
+      "J",
+      "K",
+      "L",
+      "M",
+      "N",
+      "O",
+      "P",
+      "Q",
+      "R",
+      "S",
+      "T",
+      "U",
+      "V",
+      "W",
+      "X",
+      "Y",
+      "Z",
+      "[",
+      "\\",
+      "]",
+      "^",
+      "_",
+      "`",
+      "a",
+      "b",
+      "c",
+      "d",
+      "e",
+      "f",
+      "g",
+      "h",
+      "i",
+      "j",
+      "k",
+      "l",
+      "m",
+      "n",
+      "o",
+      "p",
+      "q",
+      "r",
+      "s",
+      "t",
+      "u",
+      "v",
+      "w",
+      "x",
+      "y",
+      "z",
+      "{",
+      "|",
+      "}",
+      "~"
+    ],
+    "weight_decay": 0.01
+  },
+  "training": {
+    "loss": 0.0016285243909806013,
+    "parameters": 81004,
+    "seconds": 2.2738583749996906,
+    "seed": 0,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  }
+}

raw-step/seed-1/config.json

@@ -0,0 +1,132 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "raw-step",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "run_a6_x6.RawTransformer",
+  "outputs": 108,
+  "position_limit": 256,
+  "seed": 1,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "END",
+    "EMPTY",
+    "SEP",
+    "EOS",
+    "PUSH",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT",
+    " ",
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    "0",
+    "1",
+    "2",
+    "3",
+    "4",
+    "5",
+    "6",
+    "7",
+    "8",
+    "9",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "A",
+    "B",
+    "C",
+    "D",
+    "E",
+    "F",
+    "G",
+    "H",
+    "I",
+    "J",
+    "K",
+    "L",
+    "M",
+    "N",
+    "O",
+    "P",
+    "Q",
+    "R",
+    "S",
+    "T",
+    "U",
+    "V",
+    "W",
+    "X",
+    "Y",
+    "Z",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "a",
+    "b",
+    "c",
+    "d",
+    "e",
+    "f",
+    "g",
+    "h",
+    "i",
+    "j",
+    "k",
+    "l",
+    "m",
+    "n",
+    "o",
+    "p",
+    "q",
+    "r",
+    "s",
+    "t",
+    "u",
+    "v",
+    "w",
+    "x",
+    "y",
+    "z",
+    "{",
+    "|",
+    "}",
+    "~"
+  ],
+  "width": 64
+}

raw-step/seed-1/conversion_verification.json

@@ -0,0 +1,13 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "raw-step",
+  "published_sha256": "c0d4940618fce954f253a157d7167b27fabd561982dc73d174be29ab02c1278a",
+  "saved_predictions_reproduced": {
+    "exhaustive": 108,
+    "held_out": 92
+  },
+  "seed": 1,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "7c83c18936b476968f6c373d1bc9a034eb361dd4f96c333222d0f5d9c0aea0f0",
+  "torch_version": "2.14.1"
+}

raw-step/seed-1/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0d4940618fce954f253a157d7167b27fabd561982dc73d174be29ab02c1278a
+size 392360

raw-step/seed-1/results.json

@@ -0,0 +1,283 @@
+{
+  "action_accuracy": [
+    0,
+    92
+  ],
+  "condition": "tiny_raw_autoregressive",
+  "exhaustive": {
+    "([)]#": [
+      23,
+      36
+    ],
+    "abcde": [
+      23,
+      36
+    ],
+    "uvwxy": [
+      36,
+      36
+    ]
+  },
+  "first_error_median": 1.0,
+  "held_out": {
+    "brackets": {
+      "by_depth": {
+        "0": [
+          0,
+          2
+        ],
+        "1": [
+          0,
+          4
+        ],
+        "16": [
+          0,
+          8
+        ],
+        "2": [
+          0,
+          8
+        ],
+        "3": [
+          0,
+          8
+        ],
+        "4": [
+          0,
+          8
+        ],
+        "8": [
+          0,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "below label-blind",
+        "1": "below label-blind",
+        "16": "below label-blind",
+        "2": "below label-blind",
+        "3": "below label-blind",
+        "4": "below label-blind",
+        "8": "below label-blind"
+      }
+    },
+    "letters": {
+      "by_depth": {
+        "0": [
+          0,
+          2
+        ],
+        "1": [
+          0,
+          4
+        ],
+        "16": [
+          0,
+          8
+        ],
+        "2": [
+          0,
+          8
+        ],
+        "3": [
+          0,
+          8
+        ],
+        "4": [
+          0,
+          8
+        ],
+        "8": [
+          0,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "below label-blind",
+        "1": "below label-blind",
+        "16": "below label-blind",
+        "2": "below label-blind",
+        "3": "below label-blind",
+        "4": "below label-blind",
+        "8": "below label-blind"
+      }
+    },
+    "total": [
+      0,
+      92
+    ]
+  },
+  "invalid_action_calls": 0,
+  "label_blind": {
+    "always_accept": [
+      46,
+      92
+    ],
+    "always_reject": [
+      46,
+      92
+    ],
+    "by_cell": {
+      "brackets": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      },
+      "letters": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      }
+    }
+  },
+  "push_value_errors": 22,
+  "seed": 1,
+  "training": {
+    "loss": 0.0016210851026698947,
+    "parameters": 81004,
+    "seconds": 1.9787468750000698,
+    "seed": 1,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  },
+  "usage": {
+    "completion_tokens": 0,
+    "prompt_tokens": 0
+  }
+}

raw-step/seed-1/training.json

@@ -0,0 +1,148 @@
+{
+  "plan": {
+    "architecture": "2-layer width-64 4-head Transformer; causal attention; autoregressive operation/value/EOS",
+    "batch": 36,
+    "clip": 1,
+    "domain": "three fixed supplied maps x 2 phases x 6 current symbols/end x 3 closing-symbol/empty tops",
+    "held_out_hash": "d2c7202ea08da83ad5398723a2cc4ce9d3175b7e5f7b84ce40aa529103e0d6e2",
+    "lr": 0.001,
+    "optimizer": "AdamW",
+    "output_limit": 3,
+    "prompt_loss_masked": true,
+    "schedule": "cosine to 1e-5",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_x6.py": "b8ce548d5a55069483b46ed89da10f34bb9a3edf976b361069cd4d11692fd685",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/a6_raw.py": "df9c2be312f2290939871364baca738ac9a849dca855d049e304a6d0bb74e630"
+    },
+    "steps": 800,
+    "torch": "2.14.1",
+    "training_hash": "11d5025e1246025f38e3ea840fd2a065ca73c2751cdf9de5f50005f4be7fa3a7",
+    "vocab": [
+      "PAD",
+      "CLS",
+      "opening",
+      "closing",
+      "END",
+      "EMPTY",
+      "SEP",
+      "EOS",
+      "PUSH",
+      "CENTER",
+      "POP",
+      "ACCEPT",
+      "REJECT",
+      " ",
+      "!",
+      "\"",
+      "#",
+      "$",
+      "%",
+      "&",
+      "'",
+      "(",
+      ")",
+      "*",
+      "+",
+      ",",
+      "-",
+      ".",
+      "<local-path>",
+      "0",
+      "1",
+      "2",
+      "3",
+      "4",
+      "5",
+      "6",
+      "7",
+      "8",
+      "9",
+      ":",
+      ";",
+      "<",
+      "=",
+      ">",
+      "?",
+      "@",
+      "A",
+      "B",
+      "C",
+      "D",
+      "E",
+      "F",
+      "G",
+      "H",
+      "I",
+      "J",
+      "K",
+      "L",
+      "M",
+      "N",
+      "O",
+      "P",
+      "Q",
+      "R",
+      "S",
+      "T",
+      "U",
+      "V",
+      "W",
+      "X",
+      "Y",
+      "Z",
+      "[",
+      "\\",
+      "]",
+      "^",
+      "_",
+      "`",
+      "a",
+      "b",
+      "c",
+      "d",
+      "e",
+      "f",
+      "g",
+      "h",
+      "i",
+      "j",
+      "k",
+      "l",
+      "m",
+      "n",
+      "o",
+      "p",
+      "q",
+      "r",
+      "s",
+      "t",
+      "u",
+      "v",
+      "w",
+      "x",
+      "y",
+      "z",
+      "{",
+      "|",
+      "}",
+      "~"
+    ],
+    "weight_decay": 0.01
+  },
+  "training": {
+    "loss": 0.0016210851026698947,
+    "parameters": 81004,
+    "seconds": 1.9787468750000698,
+    "seed": 1,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  }
+}

raw-step/seed-2/config.json

@@ -0,0 +1,132 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "raw-step",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "run_a6_x6.RawTransformer",
+  "outputs": 108,
+  "position_limit": 256,
+  "seed": 2,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "END",
+    "EMPTY",
+    "SEP",
+    "EOS",
+    "PUSH",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT",
+    " ",
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    "0",
+    "1",
+    "2",
+    "3",
+    "4",
+    "5",
+    "6",
+    "7",
+    "8",
+    "9",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "A",
+    "B",
+    "C",
+    "D",
+    "E",
+    "F",
+    "G",
+    "H",
+    "I",
+    "J",
+    "K",
+    "L",
+    "M",
+    "N",
+    "O",
+    "P",
+    "Q",
+    "R",
+    "S",
+    "T",
+    "U",
+    "V",
+    "W",
+    "X",
+    "Y",
+    "Z",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "a",
+    "b",
+    "c",
+    "d",
+    "e",
+    "f",
+    "g",
+    "h",
+    "i",
+    "j",
+    "k",
+    "l",
+    "m",
+    "n",
+    "o",
+    "p",
+    "q",
+    "r",
+    "s",
+    "t",
+    "u",
+    "v",
+    "w",
+    "x",
+    "y",
+    "z",
+    "{",
+    "|",
+    "}",
+    "~"
+  ],
+  "width": 64
+}

raw-step/seed-2/conversion_verification.json

@@ -0,0 +1,13 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "raw-step",
+  "published_sha256": "a96e32f0a540aaf176a0ec8479c3f9b18c9a43fbea392f1049ea028f3f229f87",
+  "saved_predictions_reproduced": {
+    "exhaustive": 108,
+    "held_out": 92
+  },
+  "seed": 2,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "d1d3ef582c2f85a043ba507cd502aae062a8e5daac3cacaa0702ddce9404abf4",
+  "torch_version": "2.14.1"
+}

raw-step/seed-2/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a96e32f0a540aaf176a0ec8479c3f9b18c9a43fbea392f1049ea028f3f229f87
+size 392360

raw-step/seed-2/results.json

@@ -0,0 +1,283 @@
+{
+  "action_accuracy": [
+    0,
+    92
+  ],
+  "condition": "tiny_raw_autoregressive",
+  "exhaustive": {
+    "([)]#": [
+      23,
+      36
+    ],
+    "abcde": [
+      20,
+      36
+    ],
+    "uvwxy": [
+      36,
+      36
+    ]
+  },
+  "first_error_median": 1.0,
+  "held_out": {
+    "brackets": {
+      "by_depth": {
+        "0": [
+          0,
+          2
+        ],
+        "1": [
+          0,
+          4
+        ],
+        "16": [
+          0,
+          8
+        ],
+        "2": [
+          0,
+          8
+        ],
+        "3": [
+          0,
+          8
+        ],
+        "4": [
+          0,
+          8
+        ],
+        "8": [
+          0,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "below label-blind",
+        "1": "below label-blind",
+        "16": "below label-blind",
+        "2": "below label-blind",
+        "3": "below label-blind",
+        "4": "below label-blind",
+        "8": "below label-blind"
+      }
+    },
+    "letters": {
+      "by_depth": {
+        "0": [
+          0,
+          2
+        ],
+        "1": [
+          0,
+          4
+        ],
+        "16": [
+          0,
+          8
+        ],
+        "2": [
+          0,
+          8
+        ],
+        "3": [
+          0,
+          8
+        ],
+        "4": [
+          0,
+          8
+        ],
+        "8": [
+          0,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "below label-blind",
+        "1": "below label-blind",
+        "16": "below label-blind",
+        "2": "below label-blind",
+        "3": "below label-blind",
+        "4": "below label-blind",
+        "8": "below label-blind"
+      }
+    },
+    "total": [
+      0,
+      92
+    ]
+  },
+  "invalid_action_calls": 0,
+  "label_blind": {
+    "always_accept": [
+      46,
+      92
+    ],
+    "always_reject": [
+      46,
+      92
+    ],
+    "by_cell": {
+      "brackets": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      },
+      "letters": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      }
+    }
+  },
+  "push_value_errors": 22,
+  "seed": 2,
+  "training": {
+    "loss": 0.001583919976837933,
+    "parameters": 81004,
+    "seconds": 1.9714072089991532,
+    "seed": 2,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  },
+  "usage": {
+    "completion_tokens": 0,
+    "prompt_tokens": 0
+  }
+}

raw-step/seed-2/training.json

@@ -0,0 +1,148 @@
+{
+  "plan": {
+    "architecture": "2-layer width-64 4-head Transformer; causal attention; autoregressive operation/value/EOS",
+    "batch": 36,
+    "clip": 1,
+    "domain": "three fixed supplied maps x 2 phases x 6 current symbols/end x 3 closing-symbol/empty tops",
+    "held_out_hash": "d2c7202ea08da83ad5398723a2cc4ce9d3175b7e5f7b84ce40aa529103e0d6e2",
+    "lr": 0.001,
+    "optimizer": "AdamW",
+    "output_limit": 3,
+    "prompt_loss_masked": true,
+    "schedule": "cosine to 1e-5",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_x6.py": "b8ce548d5a55069483b46ed89da10f34bb9a3edf976b361069cd4d11692fd685",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/a6_raw.py": "df9c2be312f2290939871364baca738ac9a849dca855d049e304a6d0bb74e630"
+    },
+    "steps": 800,
+    "torch": "2.14.1",
+    "training_hash": "11d5025e1246025f38e3ea840fd2a065ca73c2751cdf9de5f50005f4be7fa3a7",
+    "vocab": [
+      "PAD",
+      "CLS",
+      "opening",
+      "closing",
+      "END",
+      "EMPTY",
+      "SEP",
+      "EOS",
+      "PUSH",
+      "CENTER",
+      "POP",
+      "ACCEPT",
+      "REJECT",
+      " ",
+      "!",
+      "\"",
+      "#",
+      "$",
+      "%",
+      "&",
+      "'",
+      "(",
+      ")",
+      "*",
+      "+",
+      ",",
+      "-",
+      ".",
+      "<local-path>",
+      "0",
+      "1",
+      "2",
+      "3",
+      "4",
+      "5",
+      "6",
+      "7",
+      "8",
+      "9",
+      ":",
+      ";",
+      "<",
+      "=",
+      ">",
+      "?",
+      "@",
+      "A",
+      "B",
+      "C",
+      "D",
+      "E",
+      "F",
+      "G",
+      "H",
+      "I",
+      "J",
+      "K",
+      "L",
+      "M",
+      "N",
+      "O",
+      "P",
+      "Q",
+      "R",
+      "S",
+      "T",
+      "U",
+      "V",
+      "W",
+      "X",
+      "Y",
+      "Z",
+      "[",
+      "\\",
+      "]",
+      "^",
+      "_",
+      "`",
+      "a",
+      "b",
+      "c",
+      "d",
+      "e",
+      "f",
+      "g",
+      "h",
+      "i",
+      "j",
+      "k",
+      "l",
+      "m",
+      "n",
+      "o",
+      "p",
+      "q",
+      "r",
+      "s",
+      "t",
+      "u",
+      "v",
+      "w",
+      "x",
+      "y",
+      "z",
+      "{",
+      "|",
+      "}",
+      "~"
+    ],
+    "weight_decay": 0.01
+  },
+  "training": {
+    "loss": 0.001583919976837933,
+    "parameters": 81004,
+    "seconds": 1.9714072089991532,
+    "seed": 2,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  }
+}

step/seed-0/config.json

@@ -0,0 +1,36 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "step",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "abstractgym.a6_model.TinyTransformer",
+  "outputs": 6,
+  "position_limit": 256,
+  "seed": 0,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "CENTER",
+    "END",
+    "EMPTY",
+    "DIRECT"
+  ],
+  "width": 64
+}

step/seed-0/conversion_verification.json

@@ -0,0 +1,14 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "step",
+  "published_sha256": "baee7f0d12b2a70b1e9604c2d7d9a80554f156a2af111d3966f37cea9f21357f",
+  "saved_predictions_reproduced": {
+    "exhaustive": 36,
+    "extra": 32,
+    "held_out": 92
+  },
+  "seed": 0,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "a6949c9c2a8679152cd71b1a64c68a5147cf8cc3c6dca94bcfd52c5820ae478e",
+  "torch_version": "2.14.1"
+}

step/seed-0/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:baee7f0d12b2a70b1e9604c2d7d9a80554f156a2af111d3966f37cea9f21357f
+size 341256

step/seed-0/results.json

@@ -0,0 +1,291 @@
+{
+  "action_accuracy": [
+    988,
+    988
+  ],
+  "condition": "tiny_trained",
+  "evaluation_seconds": 0.07790250000107335,
+  "exhaustive_check": [
+    36,
+    36
+  ],
+  "extra_depths": {
+    "32": [
+      16,
+      16
+    ],
+    "64": [
+      16,
+      16
+    ]
+  },
+  "first_error_median": null,
+  "held_out": {
+    "brackets": {
+      "by_depth": {
+        "0": [
+          2,
+          2
+        ],
+        "1": [
+          4,
+          4
+        ],
+        "16": [
+          8,
+          8
+        ],
+        "2": [
+          8,
+          8
+        ],
+        "3": [
+          8,
+          8
+        ],
+        "4": [
+          8,
+          8
+        ],
+        "8": [
+          8,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "beats both label-blind baselines",
+        "1": "beats both label-blind baselines",
+        "16": "beats both label-blind baselines",
+        "2": "beats both label-blind baselines",
+        "3": "beats both label-blind baselines",
+        "4": "beats both label-blind baselines",
+        "8": "beats both label-blind baselines"
+      }
+    },
+    "letters": {
+      "by_depth": {
+        "0": [
+          2,
+          2
+        ],
+        "1": [
+          4,
+          4
+        ],
+        "16": [
+          8,
+          8
+        ],
+        "2": [
+          8,
+          8
+        ],
+        "3": [
+          8,
+          8
+        ],
+        "4": [
+          8,
+          8
+        ],
+        "8": [
+          8,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "beats both label-blind baselines",
+        "1": "beats both label-blind baselines",
+        "16": "beats both label-blind baselines",
+        "2": "beats both label-blind baselines",
+        "3": "beats both label-blind baselines",
+        "4": "beats both label-blind baselines",
+        "8": "beats both label-blind baselines"
+      }
+    },
+    "total": [
+      92,
+      92
+    ]
+  },
+  "label_blind": {
+    "always_accept": [
+      46,
+      92
+    ],
+    "always_reject": [
+      46,
+      92
+    ],
+    "by_cell": {
+      "brackets": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      },
+      "letters": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      }
+    }
+  },
+  "model_revisions": {
+    "direct-0.pt": "39bf28b49785115db3ca0b58a584230aff6db09361ee79619ba6e7fca8bcb8cd",
+    "step-0.pt": "a6949c9c2a8679152cd71b1a64c68a5147cf8cc3c6dca94bcfd52c5820ae478e"
+  },
+  "push_value_errors": 0,
+  "seed": 0,
+  "token_usage": {
+    "direct_input_tokens": 1442,
+    "execution_input_tokens": 3952,
+    "execution_output_classes": 988
+  },
+  "train_examples": 2305,
+  "training": {
+    "batch_size": 36,
+    "final_loss": 0.0006350057665258646,
+    "parameters": 68230,
+    "seconds": 1.4034502079994127,
+    "steps": 800,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  },
+  "unique_train_examples": 36
+}

step/seed-0/training.json

@@ -0,0 +1,63 @@
+{
+  "plan": {
+    "architecture": {
+      "attention": "bidirectional encoder, CLS readout",
+      "dropout": 0,
+      "feedforward": 128,
+      "heads": 4,
+      "layers": 2,
+      "position": "fixed sinusoidal",
+      "width": 64
+    },
+    "device": "cpu",
+    "dtype": "float32",
+    "git_revision": "39bca8d1f7dd0324010f76d6eb44a68ea719d572",
+    "hardware": "arm",
+    "optimizer": {
+      "clip_norm": 1.0,
+      "lr": 0.001,
+      "name": "AdamW",
+      "schedule": "cosine to 1e-5",
+      "weight_decay": 0.01
+    },
+    "packages": {
+      "torch": "2.14.1"
+    },
+    "platform": "macOS-26.7-arm64-arm-64bit",
+    "python": "3.12.13",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_tiny.py": "62a632c65003a10c5f35b1a586f7a62cfae3605db0c39d9ae48914367ea7aa1e",
+      "src/abstractgym/a6.py": "975b3d76fc9503e11e8f0ad8c57aa27162250298d4316d949145e6df80dbbfe2",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/brackets.py": "9fa8b835c0aac8fa26177b032f1fc07e1bb0964f7ec01fd0f83ad11b74e0550f"
+    },
+    "threads": 2,
+    "training": {
+      "deduplicated_step_examples": 36,
+      "direct_examples": 367,
+      "direct_positive": 31,
+      "direct_sampling": "balanced labels, replacement; separate matched model",
+      "max_depth": 4,
+      "raw_step_examples": 2305,
+      "symbols": "uvwxy",
+      "trajectories": 367
+    },
+    "training_hash": "8fa8728efdda11e741e8ec16c2e0267cf0fd321e849119caea177055d15e5df0"
+  },
+  "training": {
+    "batch_size": 36,
+    "final_loss": 0.0006350057665258646,
+    "parameters": 68230,
+    "seconds": 1.4034502079994127,
+    "steps": 800,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  }
+}

step/seed-1/config.json

@@ -0,0 +1,36 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "step",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "abstractgym.a6_model.TinyTransformer",
+  "outputs": 6,
+  "position_limit": 256,
+  "seed": 1,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "CENTER",
+    "END",
+    "EMPTY",
+    "DIRECT"
+  ],
+  "width": 64
+}

step/seed-1/conversion_verification.json

@@ -0,0 +1,14 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "step",
+  "published_sha256": "2d25c9513d2378fa151f79d4cb4619a2e0d401f7825bd7b6c5a6c13769b37f5b",
+  "saved_predictions_reproduced": {
+    "exhaustive": 36,
+    "extra": 32,
+    "held_out": 92
+  },
+  "seed": 1,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "341d85d422c11d1d34345d65e05cef12e2ee0a4d1608e065857eb64cda564a77",
+  "torch_version": "2.14.1"
+}

step/seed-1/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d25c9513d2378fa151f79d4cb4619a2e0d401f7825bd7b6c5a6c13769b37f5b
+size 341256

step/seed-1/results.json

@@ -0,0 +1,291 @@
+{
+  "action_accuracy": [
+    988,
+    988
+  ],
+  "condition": "tiny_trained",
+  "evaluation_seconds": 0.05974966700159712,
+  "exhaustive_check": [
+    36,
+    36
+  ],
+  "extra_depths": {
+    "32": [
+      16,
+      16
+    ],
+    "64": [
+      16,
+      16
+    ]
+  },
+  "first_error_median": null,
+  "held_out": {
+    "brackets": {
+      "by_depth": {
+        "0": [
+          2,
+          2
+        ],
+        "1": [
+          4,
+          4
+        ],
+        "16": [
+          8,
+          8
+        ],
+        "2": [
+          8,
+          8
+        ],
+        "3": [
+          8,
+          8
+        ],
+        "4": [
+          8,
+          8
+        ],
+        "8": [
+          8,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "beats both label-blind baselines",
+        "1": "beats both label-blind baselines",
+        "16": "beats both label-blind baselines",
+        "2": "beats both label-blind baselines",
+        "3": "beats both label-blind baselines",
+        "4": "beats both label-blind baselines",
+        "8": "beats both label-blind baselines"
+      }
+    },
+    "letters": {
+      "by_depth": {
+        "0": [
+          2,
+          2
+        ],
+        "1": [
+          4,
+          4
+        ],
+        "16": [
+          8,
+          8
+        ],
+        "2": [
+          8,
+          8
+        ],
+        "3": [
+          8,
+          8
+        ],
+        "4": [
+          8,
+          8
+        ],
+        "8": [
+          8,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "beats both label-blind baselines",
+        "1": "beats both label-blind baselines",
+        "16": "beats both label-blind baselines",
+        "2": "beats both label-blind baselines",
+        "3": "beats both label-blind baselines",
+        "4": "beats both label-blind baselines",
+        "8": "beats both label-blind baselines"
+      }
+    },
+    "total": [
+      92,
+      92
+    ]
+  },
+  "label_blind": {
+    "always_accept": [
+      46,
+      92
+    ],
+    "always_reject": [
+      46,
+      92
+    ],
+    "by_cell": {
+      "brackets": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      },
+      "letters": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      }
+    }
+  },
+  "model_revisions": {
+    "direct-1.pt": "07445f6621896c71053a5e64ba61b78c430954a840575bba113e7aeaef462a43",
+    "step-1.pt": "341d85d422c11d1d34345d65e05cef12e2ee0a4d1608e065857eb64cda564a77"
+  },
+  "push_value_errors": 0,
+  "seed": 1,
+  "token_usage": {
+    "direct_input_tokens": 1442,
+    "execution_input_tokens": 3952,
+    "execution_output_classes": 988
+  },
+  "train_examples": 2305,
+  "training": {
+    "batch_size": 36,
+    "final_loss": 0.0007644929573871195,
+    "parameters": 68230,
+    "seconds": 1.179319791001035,
+    "steps": 800,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  },
+  "unique_train_examples": 36
+}

step/seed-1/training.json

@@ -0,0 +1,63 @@
+{
+  "plan": {
+    "architecture": {
+      "attention": "bidirectional encoder, CLS readout",
+      "dropout": 0,
+      "feedforward": 128,
+      "heads": 4,
+      "layers": 2,
+      "position": "fixed sinusoidal",
+      "width": 64
+    },
+    "device": "cpu",
+    "dtype": "float32",
+    "git_revision": "39bca8d1f7dd0324010f76d6eb44a68ea719d572",
+    "hardware": "arm",
+    "optimizer": {
+      "clip_norm": 1.0,
+      "lr": 0.001,
+      "name": "AdamW",
+      "schedule": "cosine to 1e-5",
+      "weight_decay": 0.01
+    },
+    "packages": {
+      "torch": "2.14.1"
+    },
+    "platform": "macOS-26.7-arm64-arm-64bit",
+    "python": "3.12.13",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_tiny.py": "62a632c65003a10c5f35b1a586f7a62cfae3605db0c39d9ae48914367ea7aa1e",
+      "src/abstractgym/a6.py": "975b3d76fc9503e11e8f0ad8c57aa27162250298d4316d949145e6df80dbbfe2",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/brackets.py": "9fa8b835c0aac8fa26177b032f1fc07e1bb0964f7ec01fd0f83ad11b74e0550f"
+    },
+    "threads": 2,
+    "training": {
+      "deduplicated_step_examples": 36,
+      "direct_examples": 367,
+      "direct_positive": 31,
+      "direct_sampling": "balanced labels, replacement; separate matched model",
+      "max_depth": 4,
+      "raw_step_examples": 2305,
+      "symbols": "uvwxy",
+      "trajectories": 367
+    },
+    "training_hash": "8fa8728efdda11e741e8ec16c2e0267cf0fd321e849119caea177055d15e5df0"
+  },
+  "training": {
+    "batch_size": 36,
+    "final_loss": 0.0007644929573871195,
+    "parameters": 68230,
+    "seconds": 1.179319791001035,
+    "steps": 800,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  }
+}

step/seed-2/config.json

@@ -0,0 +1,36 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "step",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "abstractgym.a6_model.TinyTransformer",
+  "outputs": 6,
+  "position_limit": 256,
+  "seed": 2,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "opening",
+    "closing",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "CENTER",
+    "END",
+    "EMPTY",
+    "DIRECT"
+  ],
+  "width": 64
+}

step/seed-2/conversion_verification.json

@@ -0,0 +1,14 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "step",
+  "published_sha256": "7a2d655c83a05f98193abb506ab5e7d181d4463fd21a6dbc1934b43cc7abf213",
+  "saved_predictions_reproduced": {
+    "exhaustive": 36,
+    "extra": 32,
+    "held_out": 92
+  },
+  "seed": 2,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "e70093585cce216028e40d2ff4648b6bf356383d96e1706ffb5a2e86f656ca30",
+  "torch_version": "2.14.1"
+}

step/seed-2/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a2d655c83a05f98193abb506ab5e7d181d4463fd21a6dbc1934b43cc7abf213
+size 341256

step/seed-2/results.json

@@ -0,0 +1,291 @@
+{
+  "action_accuracy": [
+    988,
+    988
+  ],
+  "condition": "tiny_trained",
+  "evaluation_seconds": 0.12443979099771241,
+  "exhaustive_check": [
+    36,
+    36
+  ],
+  "extra_depths": {
+    "32": [
+      16,
+      16
+    ],
+    "64": [
+      16,
+      16
+    ]
+  },
+  "first_error_median": null,
+  "held_out": {
+    "brackets": {
+      "by_depth": {
+        "0": [
+          2,
+          2
+        ],
+        "1": [
+          4,
+          4
+        ],
+        "16": [
+          8,
+          8
+        ],
+        "2": [
+          8,
+          8
+        ],
+        "3": [
+          8,
+          8
+        ],
+        "4": [
+          8,
+          8
+        ],
+        "8": [
+          8,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "beats both label-blind baselines",
+        "1": "beats both label-blind baselines",
+        "16": "beats both label-blind baselines",
+        "2": "beats both label-blind baselines",
+        "3": "beats both label-blind baselines",
+        "4": "beats both label-blind baselines",
+        "8": "beats both label-blind baselines"
+      }
+    },
+    "letters": {
+      "by_depth": {
+        "0": [
+          2,
+          2
+        ],
+        "1": [
+          4,
+          4
+        ],
+        "16": [
+          8,
+          8
+        ],
+        "2": [
+          8,
+          8
+        ],
+        "3": [
+          8,
+          8
+        ],
+        "4": [
+          8,
+          8
+        ],
+        "8": [
+          8,
+          8
+        ]
+      },
+      "interpretation": {
+        "0": "beats both label-blind baselines",
+        "1": "beats both label-blind baselines",
+        "16": "beats both label-blind baselines",
+        "2": "beats both label-blind baselines",
+        "3": "beats both label-blind baselines",
+        "4": "beats both label-blind baselines",
+        "8": "beats both label-blind baselines"
+      }
+    },
+    "total": [
+      92,
+      92
+    ]
+  },
+  "label_blind": {
+    "always_accept": [
+      46,
+      92
+    ],
+    "always_reject": [
+      46,
+      92
+    ],
+    "by_cell": {
+      "brackets": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      },
+      "letters": {
+        "0": {
+          "always_accept": [
+            1,
+            2
+          ],
+          "always_reject": [
+            1,
+            2
+          ]
+        },
+        "1": {
+          "always_accept": [
+            2,
+            4
+          ],
+          "always_reject": [
+            2,
+            4
+          ]
+        },
+        "16": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "2": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "3": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "4": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        },
+        "8": {
+          "always_accept": [
+            4,
+            8
+          ],
+          "always_reject": [
+            4,
+            8
+          ]
+        }
+      }
+    }
+  },
+  "model_revisions": {
+    "direct-2.pt": "5e14beeb2d5e742967d56c86677e37a4b0b8d2a1097d3c1d50f78d71fe29a782",
+    "step-2.pt": "e70093585cce216028e40d2ff4648b6bf356383d96e1706ffb5a2e86f656ca30"
+  },
+  "push_value_errors": 0,
+  "seed": 2,
+  "token_usage": {
+    "direct_input_tokens": 1442,
+    "execution_input_tokens": 3952,
+    "execution_output_classes": 988
+  },
+  "train_examples": 2305,
+  "training": {
+    "batch_size": 36,
+    "final_loss": 0.0006954485434107482,
+    "parameters": 68230,
+    "seconds": 1.3472057079998194,
+    "steps": 800,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  },
+  "unique_train_examples": 36
+}

step/seed-2/training.json

@@ -0,0 +1,63 @@
+{
+  "plan": {
+    "architecture": {
+      "attention": "bidirectional encoder, CLS readout",
+      "dropout": 0,
+      "feedforward": 128,
+      "heads": 4,
+      "layers": 2,
+      "position": "fixed sinusoidal",
+      "width": 64
+    },
+    "device": "cpu",
+    "dtype": "float32",
+    "git_revision": "39bca8d1f7dd0324010f76d6eb44a68ea719d572",
+    "hardware": "arm",
+    "optimizer": {
+      "clip_norm": 1.0,
+      "lr": 0.001,
+      "name": "AdamW",
+      "schedule": "cosine to 1e-5",
+      "weight_decay": 0.01
+    },
+    "packages": {
+      "torch": "2.14.1"
+    },
+    "platform": "macOS-26.7-arm64-arm-64bit",
+    "python": "3.12.13",
+    "seeds": [
+      0,
+      1,
+      2
+    ],
+    "source_sha256": {
+      "scripts/run_a6_tiny.py": "62a632c65003a10c5f35b1a586f7a62cfae3605db0c39d9ae48914367ea7aa1e",
+      "src/abstractgym/a6.py": "975b3d76fc9503e11e8f0ad8c57aa27162250298d4316d949145e6df80dbbfe2",
+      "src/abstractgym/a6_model.py": "d13c0b913ba39cd3c8371147d2495b185c24d8d15225cd68bc11e1aa2ebd72f7",
+      "src/abstractgym/brackets.py": "9fa8b835c0aac8fa26177b032f1fc07e1bb0964f7ec01fd0f83ad11b74e0550f"
+    },
+    "threads": 2,
+    "training": {
+      "deduplicated_step_examples": 36,
+      "direct_examples": 367,
+      "direct_positive": 31,
+      "direct_sampling": "balanced labels, replacement; separate matched model",
+      "max_depth": 4,
+      "raw_step_examples": 2305,
+      "symbols": "uvwxy",
+      "trajectories": 367
+    },
+    "training_hash": "8fa8728efdda11e741e8ec16c2e0267cf0fd321e849119caea177055d15e5df0"
+  },
+  "training": {
+    "batch_size": 36,
+    "final_loss": 0.0006954485434107482,
+    "parameters": 68230,
+    "seconds": 1.3472057079998194,
+    "steps": 800,
+    "train_accuracy": [
+      36,
+      36
+    ]
+  }
+}

trace/seed-0/config.json

@@ -0,0 +1,39 @@
+{
+  "actions": [
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "code_branch": "main",
+  "dropout": 0,
+  "dtype": "float32",
+  "family": "trace",
+  "feedforward": 128,
+  "heads": 4,
+  "layers": 2,
+  "model_class": "run_a6_tiny_trace.TraceTransformer",
+  "outputs": 15,
+  "position_limit": 256,
+  "seed": 0,
+  "vocabulary": [
+    "PAD",
+    "CLS",
+    "END",
+    "SEP",
+    "EOS",
+    "OPEN_0",
+    "OPEN_1",
+    "CLOSE_0",
+    "CLOSE_1",
+    "PUSH_0",
+    "PUSH_1",
+    "CENTER",
+    "POP",
+    "ACCEPT",
+    "REJECT"
+  ],
+  "width": 64
+}

trace/seed-0/conversion_verification.json

@@ -0,0 +1,14 @@
+{
+  "all_tensors_bitwise_equal": true,
+  "family": "trace",
+  "published_sha256": "e506bfd329fcf518b60b9d904e29045cbb4521a8ee93fce9ab19884654afe35d",
+  "saved_predictions_reproduced": {
+    "free_running": 92,
+    "injected_history": 192,
+    "teacher_forced": 92
+  },
+  "seed": 0,
+  "source_hash_manifest": "frozen.json",
+  "source_sha256": "bca522a8d4ac96fc3f10464784c96ecb182f505efeb5a73b41ac9cf71f0337eb",
+  "torch_version": "2.14.1"
+}