Report complete WT2 test for the frozen FP16 Recall adapter
Browse files- README.md +31 -2
- docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PINS.json +567 -0
- docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PROTOCOL.md +83 -0
- docs/WT2_TEST_V1_RESULTS.md +98 -0
- reports/wt2_test_v1/comparison.json +0 -0
- reports/wt2_test_v1/cpu_audit_v1.json +22 -0
- reports/wt2_test_v1/overlay_checksums.json +57 -0
- reports/wt2_test_v1/resurface.json +1464 -0
- reports/wt2_test_v1/weight_ledger.json +515 -0
- reports/wt2_test_v1/without_resurface.json +1226 -0
- scripts/audit_recall_native_prefill_wt2_test_v1.py +151 -0
- scripts/prepare_recall_wt2_test_inputs_v1.py +96 -0
- scripts/run_recall_native_prefill_wt2_test_v1.py +275 -0
README.md
CHANGED
|
@@ -19,6 +19,15 @@ datasets:
|
|
| 19 |
|
| 20 |
# Mamb2_8B_Recall
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
A **post-D Resurface-style readout adapter** for the uncompressed, pure
|
| 23 |
[`nvidia/mamba2-8b-3t-4k`](https://huggingface.co/nvidia/mamba2-8b-3t-4k)
|
| 24 |
language model. On the verified evaluation protocol, normal numeric multi-key
|
|
@@ -34,9 +43,9 @@ Source and reproduction code:
|
|
| 34 |
[`EndlessChasing/mamb2_8B_Recall`](https://github.com/EndlessChasing/mamb2_8B_Recall/tree/4478c034aff9e5dab7af47efa84214287824138d),
|
| 35 |
pinned at commit `4478c034aff9e5dab7af47efa84214287824138d`.
|
| 36 |
|
| 37 |
-
##
|
| 38 |
|
| 39 |
-
| Model | WikiText-2 validation PPL |
|
| 40 |
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 41 |
| Base model in FP16 runtime | 7.334175947318572 | 147/384 (38.28%) | 113/192 | 34/192 | 0/384 |
|
| 42 |
| Base model + this adapter | **7.052063515606311** | **365/384 (95.05%)** | **191/192** | **174/192** | 0/384 |
|
|
@@ -199,3 +208,23 @@ relicense the NVIDIA model or create an additional adapter-specific license.
|
|
| 199 |
Method credit: [Resurface — Multi-Binding Recall Is Latent in Mamba's State](https://github.com/Oso1106/Resurface-Multi-Binding-Recall-Is-Latent-in-Mamba-s-State).
|
| 200 |
Implementation and experiment details are available in the pinned GitHub
|
| 201 |
repository linked above.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
# Mamb2_8B_Recall
|
| 21 |
|
| 22 |
+
**Official WikiText-2 test PPL: 6.96282** with the fixed published Resurface
|
| 23 |
+
adapter (7.24453 without it), across **147 reset windows / 300,963 targets**.
|
| 24 |
+
The calculation retains the original native parallel SSD prefill path.
|
| 25 |
+
[Test results and reproduction](docs/WT2_TEST_V1_RESULTS.md) ·
|
| 26 |
+
[Paired raw evaluation](reports/wt2_test_v1/comparison.json) ·
|
| 27 |
+
[CPU audit](reports/wt2_test_v1/cpu_audit_v1.json).
|
| 28 |
+
Original validation and synthetic CONFIRM MK results are retained below.
|
| 29 |
+
|
| 30 |
+
|
| 31 |
A **post-D Resurface-style readout adapter** for the uncompressed, pure
|
| 32 |
[`nvidia/mamba2-8b-3t-4k`](https://huggingface.co/nvidia/mamba2-8b-3t-4k)
|
| 33 |
language model. On the verified evaluation protocol, normal numeric multi-key
|
|
|
|
| 43 |
[`EndlessChasing/mamb2_8B_Recall`](https://github.com/EndlessChasing/mamb2_8B_Recall/tree/4478c034aff9e5dab7af47efa84214287824138d),
|
| 44 |
pinned at commit `4478c034aff9e5dab7af47efa84214287824138d`.
|
| 45 |
|
| 46 |
+
## Historical validation and synthetic CONFIRM MK
|
| 47 |
|
| 48 |
+
| Model | WikiText-2 validation PPL | Synthetic CONFIRM MK | N=16 | N=64 | Target removed |
|
| 49 |
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 50 |
| Base model in FP16 runtime | 7.334175947318572 | 147/384 (38.28%) | 113/192 | 34/192 | 0/384 |
|
| 51 |
| Base model + this adapter | **7.052063515606311** | **365/384 (95.05%)** | **191/192** | **174/192** | 0/384 |
|
|
|
|
| 208 |
Method credit: [Resurface — Multi-Binding Recall Is Latent in Mamba's State](https://github.com/Oso1106/Resurface-Multi-Binding-Recall-Is-Latent-in-Mamba-s-State).
|
| 209 |
Implementation and experiment details are available in the pinned GitHub
|
| 210 |
repository linked above.
|
| 211 |
+
|
| 212 |
+
## Additional official test files
|
| 213 |
+
|
| 214 |
+
This main-branch docs overlay adds the new test evidence and helpers. If you
|
| 215 |
+
downloaded an original release tag, fetch the additional small files from main:
|
| 216 |
+
|
| 217 |
+
```bash
|
| 218 |
+
hf download EndlessChasing/Mamb2_8B_Recall \
|
| 219 |
+
--revision main --local-dir . \
|
| 220 |
+
--include 'scripts/*recall*wt2*test*v1.py' \
|
| 221 |
+
--include 'docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1*' \
|
| 222 |
+
--include 'docs/WT2_TEST_V1_RESULTS.md' \
|
| 223 |
+
--include 'reports/wt2_test_v1/*'
|
| 224 |
+
```
|
| 225 |
+
|
| 226 |
+
See [test results and reproduction](docs/WT2_TEST_V1_RESULTS.md). Original release manifests
|
| 227 |
+
and checksum lists refer to the original tagged bundle. The new
|
| 228 |
+
[overlay checksums](reports/wt2_test_v1/overlay_checksums.json) bind the small update.
|
| 229 |
+
The YAML metadata contains no model-index benchmark values; PPL tables
|
| 230 |
+
explicitly state their split.
|
docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PINS.json
ADDED
|
@@ -0,0 +1,567 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "RECALL_NATIVE_PREFILL_WT2_TEST_PINS_V1",
|
| 3 |
+
"test_dataset": {
|
| 4 |
+
"dataset": "Salesforce/wikitext",
|
| 5 |
+
"configuration": "wikitext-2-raw-v1",
|
| 6 |
+
"split": "test",
|
| 7 |
+
"revision_argument": "b08601e04326c79dfdd32d625aee71d232d685c3",
|
| 8 |
+
"dataset_fingerprint": "a46124b21ac53738",
|
| 9 |
+
"document_join": "two newline characters",
|
| 10 |
+
"text_sha256": "696cca6b65a171b0a358a4be6732cdfdf2dd6164a32e20fd70e3c13fc4dfae83",
|
| 11 |
+
"token_stream_sha256_int64le": "5b82bd46e833e77fcfc0af62bafeaac62e70e68cfdf214d375f0b7b132d4b608",
|
| 12 |
+
"total_tokens": 300964,
|
| 13 |
+
"tokenizer_sha256": "5862e2f71caf762bc9845662be5fec2867deb58d874568235a02a36c5111cd09",
|
| 14 |
+
"automatic_special_tokens": false
|
| 15 |
+
},
|
| 16 |
+
"window_count": 147,
|
| 17 |
+
"target_tokens": 300963,
|
| 18 |
+
"models": {
|
| 19 |
+
"fp16": {
|
| 20 |
+
"repo_name": "mamb2_8B_Recall",
|
| 21 |
+
"github_source_commit": "4478c034aff9e5dab7af47efa84214287824138d",
|
| 22 |
+
"hf_repo": "EndlessChasing/Mamb2_8B_Recall",
|
| 23 |
+
"hf_release_commit": "5b7b27af9cd49d2a55bd080383f1d873c181ce83",
|
| 24 |
+
"adapter_path": "artifacts/source_resurface_v1/adapter_fp16.pt",
|
| 25 |
+
"adapter_binding": {
|
| 26 |
+
"source_checkpoint_sha256": "47c2766f6aad89d73beafbeaecb334aab902d7370906d081764a90bb7a8bbbcb",
|
| 27 |
+
"tokenizer_sha256": "5862e2f71caf762bc9845662be5fec2867deb58d874568235a02a36c5111cd09",
|
| 28 |
+
"protocol_sha256": "dee9457bfaca067b63837dcd3bc94bf9ef9a3d805b1ab04227db05a4b92c54c2",
|
| 29 |
+
"train_manifest_sha256": "e711c87378dcee84c9ff5023ae15aa11c5abe6c64fbe68c52c543eaea70103b8",
|
| 30 |
+
"prose_manifest_sha256": "facb2ca461615a4199781bd21784d642d6674f5b862641b3b9edac3fb499b89d",
|
| 31 |
+
"prose_tokens_sha256": "e54b02e5162e042a9cdd504f4eb1b1652724fb240bbc2c97608967aa26297233",
|
| 32 |
+
"prose_tokens_int64le_sha256": "049dfb6847d778897fd19b67653c3189f29fbc19a36022e8a4801306d08228bc",
|
| 33 |
+
"prose_file_exact_historical": true,
|
| 34 |
+
"adapter": "MAMBA2_POST_D_RESURFACE_FP16_V1",
|
| 35 |
+
"successful_updates": 1536
|
| 36 |
+
},
|
| 37 |
+
"adapter": {
|
| 38 |
+
"file": "adapter_fp16.pt",
|
| 39 |
+
"bytes": 2374143,
|
| 40 |
+
"sha256": "e8b2b4dfe69f8e85dc9e147c9aeaa3297cff14f4043e558bb1795ad476c1fca0",
|
| 41 |
+
"parameters": 1154104,
|
| 42 |
+
"payload_bytes": 2308208,
|
| 43 |
+
"tensor_sha256": {
|
| 44 |
+
"layer0.V_read": "acbb93163b75e6eff584f4e4b3aa832e1419b7ff7119033a825efbd84b491eb4",
|
| 45 |
+
"layer0.g_read": "b4477d6efd51b32a25f33c0bfe59a8888d0921dde342170c76f61ec8daddfa2b",
|
| 46 |
+
"layer0.router_w": "458cefcfc8eefa68f3c969a9922e0a77e0e85bf69948ab58c0630089b7da00a1",
|
| 47 |
+
"layer0.router_b": "48029583821428761aa3d097456ec52ff94b9bce7a33bd788ae74d8918388443",
|
| 48 |
+
"layer1.V_read": "e7f67def7f12ad4b8b490892546b97ac61965d2e6e5bb82607ad16b847da2e4e",
|
| 49 |
+
"layer1.g_read": "c3d3ec0c99f3d3c2bc4bca5424bbc507db84b9d8ce9537f5bf4e13f250af182d",
|
| 50 |
+
"layer1.router_w": "f8feb0aca9c684ac43095c3d48382d691242ec2026eb7e664f37590ebad1809b",
|
| 51 |
+
"layer1.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 52 |
+
"layer2.V_read": "33db90f8fd77d43b58db2836f267c436975737efa77a31dc6bb6310a57889269",
|
| 53 |
+
"layer2.g_read": "3f51857d1e6d7f5cfe77d5b3ad17713cf04e90c08e600d491f5e7b10937f01ea",
|
| 54 |
+
"layer2.router_w": "ccb5864fb848de8df2784269f4a7888c9a46d843aa6da7d5ae1d002b97dc26fe",
|
| 55 |
+
"layer2.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 56 |
+
"layer3.V_read": "2fa3be14cf23207fe7f32eccdefcf36fe631b3aff7fce33cdb39de3b796cb344",
|
| 57 |
+
"layer3.g_read": "6f3f0331ec9a3bf2c9e36f773706f9d0ad924b8140a8ab8fd8f88105ae0e3a9c",
|
| 58 |
+
"layer3.router_w": "2a76cde8c029f103523681187f9ebfc9514055639cf93b09121f50113b199a62",
|
| 59 |
+
"layer3.router_b": "de6343c9dac0fa5e944b0260a84812bf8e0cc2efd38014a76d85709a524ad535",
|
| 60 |
+
"layer4.V_read": "0991b76aab9cb0464078d304f3ef460b0239b3c815e405eb476850c4bd89e523",
|
| 61 |
+
"layer4.g_read": "b6e06f82309dd582792cc05e5279d24f4e07d2036ace5e52314987c6a0530292",
|
| 62 |
+
"layer4.router_w": "209403602e80804d25e2e34ada09713033efcd947458237c8a076c3d923fade0",
|
| 63 |
+
"layer4.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 64 |
+
"layer5.V_read": "f60edcdb83b74a7d8b7511aed1abdf8e90faf6083ea84cd532694393e3a7b635",
|
| 65 |
+
"layer5.g_read": "3fea77dab90cfbbdc2c08e8407755ecf3d47b165cc8eaca3a5a8ec401f3af52d",
|
| 66 |
+
"layer5.router_w": "bfa4e2def4cea5f6e40f1f664c0200c9bb812995721998f027109b6d6a19ead2",
|
| 67 |
+
"layer5.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 68 |
+
"layer6.V_read": "5c0864235673c03f19285c0019320389aca05547ba122af8d15ed3e8afef768c",
|
| 69 |
+
"layer6.g_read": "0416d68791a8552f8722d36d7d0caf0cd57595693a90b4d2fa42b70cf1b5af9e",
|
| 70 |
+
"layer6.router_w": "563b89e03b5e6d7913580b5d486656341b1e9e84724b0d231f5de5ac28cb92c0",
|
| 71 |
+
"layer6.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 72 |
+
"layer7.V_read": "d0dbb98b4b4163c85770854cb4f97390874d7b8a8eedda5af7fce9af64020bf8",
|
| 73 |
+
"layer7.g_read": "9109e1e7ba2c012a903b7848f7529f18dcf9d5d8d552a2027f58652202dc95be",
|
| 74 |
+
"layer7.router_w": "2c6ebcf67f7de3a3ae12c938fa075bd2c169e9dd64ad04d072a58a8e11d031cc",
|
| 75 |
+
"layer7.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 76 |
+
"layer8.V_read": "6695fa69c3b6702868cb1fb3d2992b62b93485fbb49fe3335cffe121579a609a",
|
| 77 |
+
"layer8.g_read": "ebcd5327a3cb439ce1bd8660004aa6aff79570ba73173621938bade7b0dabfd5",
|
| 78 |
+
"layer8.router_w": "5f09a5e356890db8a388fe26c6f62cdf83d45a87fc9260343b805a24057bf969",
|
| 79 |
+
"layer8.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 80 |
+
"layer9.V_read": "e52b198e7d50aea801a5a7933354d4337bc803e8e1b8eb189a1531e808e5dae2",
|
| 81 |
+
"layer9.g_read": "bb0cfe4441ad5f0949e595ccd96aa88c057c85f61cd20bb9e2c1d39bf87e0449",
|
| 82 |
+
"layer9.router_w": "305877528ccea774050a4371a19881d4ff4132df431dad61a92e743f758bc432",
|
| 83 |
+
"layer9.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 84 |
+
"layer10.V_read": "b5a8b920d24aa7826191ab90da769226bc8d259dda30f9706799edbd7f718546",
|
| 85 |
+
"layer10.g_read": "ae15ddbad1412dfa1a3ce90c7add4ac857b551daba6e9020121465a735bd56ab",
|
| 86 |
+
"layer10.router_w": "0bdc2944213d7df1ab6827ffff224c7cd110e68795aad1ea5dfcc7f30e1d9708",
|
| 87 |
+
"layer10.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 88 |
+
"layer11.V_read": "1b105e6f0ba0fd524614f741823fec6e822cd2a46dc74eaced5636858b65afb8",
|
| 89 |
+
"layer11.g_read": "ff712f2f966ca299ceafb63832935f569fa562bb4b48263c7d9a9462f3b9569b",
|
| 90 |
+
"layer11.router_w": "d132b46115bc43e4ca660d126b3110daaa98f6fb8ba26873d2f529734885b28f",
|
| 91 |
+
"layer11.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 92 |
+
"layer12.V_read": "0fdd5af3782a2a165a6d82d0bac1bcbe946bd57a255e4f5e74777e0d5d90838f",
|
| 93 |
+
"layer12.g_read": "5ba39a0f3d7a11dec51509cd9723b9770d0f29d54ff6bdd16ee8f0a853013769",
|
| 94 |
+
"layer12.router_w": "98232121a4a8cafbfa7b3956dc43055a7f5830350d48a4bc5fbb6b2a5ea1260f",
|
| 95 |
+
"layer12.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 96 |
+
"layer13.V_read": "74004cb52e34a5dfe0ee12092f7ddb2e21e5c8ff470aab1c4bc11bda2c99bd76",
|
| 97 |
+
"layer13.g_read": "d774e150badd05bd200cee2089cf2ca2c1ae1581e8d991b6655a3522377b4ba7",
|
| 98 |
+
"layer13.router_w": "fd7312d873043cd64718bc31895a07147e2f2e88821bb2f2e14a171a681b126e",
|
| 99 |
+
"layer13.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 100 |
+
"layer14.V_read": "1de745e5fd11e59ce1c54f1f2a03d1feca64cfb607a52cec5a8cabdc83be8d11",
|
| 101 |
+
"layer14.g_read": "b57ea67a980150db7c91687a61f76f68b6deb238fb1b570d3385cd6feb658db8",
|
| 102 |
+
"layer14.router_w": "56430783c73b2250a9d9dcd3de173608f782e3b5c1308c74ad1fa03cee978f75",
|
| 103 |
+
"layer14.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 104 |
+
"layer15.V_read": "ad90c47d8f475c9cdbde883e9a59ee2b2c0e5c25d6348e8f965c3dbda258c326",
|
| 105 |
+
"layer15.g_read": "ceae141021d9d934490a9dca0e655656b0e80d8df870da426beb829a365e56d3",
|
| 106 |
+
"layer15.router_w": "4815b045f15450e69e2cce3332fe3994e075f3426f09e2fa334e057069004f67",
|
| 107 |
+
"layer15.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 108 |
+
"layer16.V_read": "7adc0b9a51b6bd02d3e36aab417225423f8d66c0fb03a5b3e62b04841e7c4896",
|
| 109 |
+
"layer16.g_read": "5e44144d2892ce7e7f48210350064383a0d9485101e76fc81b14805704ef7dff",
|
| 110 |
+
"layer16.router_w": "f94c27b36b256ff81b8d5bf35d4289829c34723016f24802ccffba3fe12dc29a",
|
| 111 |
+
"layer16.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 112 |
+
"layer17.V_read": "11009457018988fe510e65db34363a4d52972b05a0a655d5bd4be3f22aff80df",
|
| 113 |
+
"layer17.g_read": "376799acab3b7b55eed58b79034eb8bfa861c073a446e88037145cd8206661b9",
|
| 114 |
+
"layer17.router_w": "9638ef42133a5eedb14d8a569106fdee455c07da830fed4658881da15c883604",
|
| 115 |
+
"layer17.router_b": "f585636255da971019a4ff67ee970b644ef4d4f493a597af54cd4390f07fd033",
|
| 116 |
+
"layer18.V_read": "938246fa3a4a2c344dea5380b6ebf4fb2e0af50c9da754c4b0aa9fc3d9561c55",
|
| 117 |
+
"layer18.g_read": "c981b84b1aae28ef5e8130d251edaa877352f4a97f6d7b02b855f993cf1b5f84",
|
| 118 |
+
"layer18.router_w": "b25ee0021de616ccfd5ae3b94ea341f038494efc6a517b195a22f283b58f9bcf",
|
| 119 |
+
"layer18.router_b": "f585636255da971019a4ff67ee970b644ef4d4f493a597af54cd4390f07fd033",
|
| 120 |
+
"layer19.V_read": "b16fd7decc91f0009994fc313e841bbadb5bc7e037b28076cf54700aecd8fd6d",
|
| 121 |
+
"layer19.g_read": "0ad360f92cf2ed90256ef5f8c1c38fba21829fa4de4a8478a1bc23ee8c832479",
|
| 122 |
+
"layer19.router_w": "fcdbede22d6ede0bb3c26b6aa90bb0069bf7dd99b80b151173030010874d5939",
|
| 123 |
+
"layer19.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 124 |
+
"layer20.V_read": "c2e8924803aecfec49945c0c482ed6d6edc6005bf993787ea40019fdd4705d57",
|
| 125 |
+
"layer20.g_read": "de8a1d4c27fed3ed6e0a6343ba13a9c92f26f3b4ae4914355ece4c07a9ec3c99",
|
| 126 |
+
"layer20.router_w": "69655bbd274b9d5d911ffdf01a05e1c0dc5d8fa8d652e9e7e36b3498ee37843c",
|
| 127 |
+
"layer20.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 128 |
+
"layer21.V_read": "84dd135a6f6832679102d93ba83efbbbe8686221c86a73e06e9ce478a1ccc8f8",
|
| 129 |
+
"layer21.g_read": "59b7d74b02b75f627b40fff6d65e69c877dfb2dc2e11f1bff86970d33749f294",
|
| 130 |
+
"layer21.router_w": "e3b0d6acd08da906697a1e014f8e1ca8456db581b290a2b01104115340e474ec",
|
| 131 |
+
"layer21.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 132 |
+
"layer22.V_read": "f78f4703e13c2e677a9b3dd606b983c0807be7987a8ad03baff3decc07972cdc",
|
| 133 |
+
"layer22.g_read": "87bd57c65ababc859992824a45108805b7d2e909b9e1904ca309bb3b3528464f",
|
| 134 |
+
"layer22.router_w": "b17d62b05781030730ec480cac4bf17c94d7f1b57460e66e9b115c15f1a5abf9",
|
| 135 |
+
"layer22.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 136 |
+
"layer23.V_read": "a3be6134cf4efd99f829250ca60b24523b686e30006a9f39fece1603dbd9d378",
|
| 137 |
+
"layer23.g_read": "21e5f2238a3718502e0708e862d14b647f7d6065fa878e0d8f23bb9a29793acd",
|
| 138 |
+
"layer23.router_w": "24f9f1bc48d369bb86b49a682fbfb36f58eefe9088213206701bb5e0a5d4bb25",
|
| 139 |
+
"layer23.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 140 |
+
"layer24.V_read": "2d2e712a2383e870754d1d748690ecf5e0d8a84e8669999da29b60543fd259a6",
|
| 141 |
+
"layer24.g_read": "6927b95f16effae647ba7cd3f2d94498779004b1fc4d952e4105a49ace5ca7cd",
|
| 142 |
+
"layer24.router_w": "aca8e53213586f7e59122b4d0523d11c23f95d063aad8cef8f780938ba90e310",
|
| 143 |
+
"layer24.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 144 |
+
"layer25.V_read": "78ee73d73c34109e3c9935a5bc5b12e933a623218031845a84c3b1f444d6d415",
|
| 145 |
+
"layer25.g_read": "0b6b84b951b187a2a611e41a98256f660a0e39841385bfc8334af413ceae88f6",
|
| 146 |
+
"layer25.router_w": "17b2f32c6af11c8d5a115346ad1aef363b2cd5c8211ec7f5770bb440ab102860",
|
| 147 |
+
"layer25.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 148 |
+
"layer26.V_read": "6267dd0f6a9077f2d0ced9151b6745a7372b18bc46ad22f4c0004c7738515ffe",
|
| 149 |
+
"layer26.g_read": "dc3aa502134b682aa39d69d47c26d6b9f6dbe81bed70f86111eb885a7b559423",
|
| 150 |
+
"layer26.router_w": "5fa67113ad4054a3f456a2ce3915d272d35345781a418d7d4c27b78127d63d86",
|
| 151 |
+
"layer26.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 152 |
+
"layer27.V_read": "30649a9be6e9c316a602aa5d9af1f0b117c8b542a2d29e808be10e71761e4cfb",
|
| 153 |
+
"layer27.g_read": "cf94e71c9666cd56a4e7f64e14e08a858098a5f5e4dae4b893ed46fc9bdf8383",
|
| 154 |
+
"layer27.router_w": "fa5b088959330d66d107cca1cde24b210ae96a60669ba969500e13869ef082de",
|
| 155 |
+
"layer27.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 156 |
+
"layer28.V_read": "01a98b0b2519586522e92b3c43b2c135c89c76711f004fc83501dcf3880ff2d9",
|
| 157 |
+
"layer28.g_read": "ca83a526129c4d5caeddb1253ba750644a9ac3219dbfd0fec21c8d98647d4969",
|
| 158 |
+
"layer28.router_w": "825cb808a2185545829272ab8d11f637a4d87ef59b93d37ffb718e01955e19e2",
|
| 159 |
+
"layer28.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 160 |
+
"layer29.V_read": "79356640911705e471c44e9eff9dd676e9c04309bed974e6589f7bab88336c52",
|
| 161 |
+
"layer29.g_read": "aeb0d1c4809e35e12982ba62667d351a320256a17e807a9fc598c941b271775a",
|
| 162 |
+
"layer29.router_w": "46783f9d0b233ba23d703deaa1857c19f32a9e3ddf2ed215a6e6ddfb29998f12",
|
| 163 |
+
"layer29.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 164 |
+
"layer30.V_read": "fff68081265f94aeaa96719f8bb5de2d8c534ab7b8755c394724aa299666fb59",
|
| 165 |
+
"layer30.g_read": "e78c18028c02faeb85cb0f3061343a0f31770794d2d4eeef2f335522c3e2578e",
|
| 166 |
+
"layer30.router_w": "6ea21eb8e0baf13e127901c7391cc1ee1985da1e27fc23b1094a4292be6e320d",
|
| 167 |
+
"layer30.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 168 |
+
"layer31.V_read": "960b52288cae25aaa4526f589e522fcb43389aa6b84cbd41416ef2eeb584df31",
|
| 169 |
+
"layer31.g_read": "80e213c9633dfcc4754066dd28d594b6697ee6e7f1d14ddc6f71a94a41c40628",
|
| 170 |
+
"layer31.router_w": "736b61bc45aff35b77ef4dabc1471c7c25240319d4b71d3b11b0ca7b9d25c79b",
|
| 171 |
+
"layer31.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 172 |
+
"layer32.V_read": "65d9afa6a2010e14ff7d659bcd4570576297026b2a13f12b98a8f57a2a6f2a64",
|
| 173 |
+
"layer32.g_read": "9c02d68d5937b033abf4e9fe8a38a377d93066d7637e3f57bdbd581dcc735ba5",
|
| 174 |
+
"layer32.router_w": "d24bcef9c4a2636330e698df6b644516c462b34ceb233923236a9b3794770a90",
|
| 175 |
+
"layer32.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 176 |
+
"layer33.V_read": "f5b365067a71ae38e5aea7b1c0c08af1831a98cc24579ba89b5b5218bb0cd066",
|
| 177 |
+
"layer33.g_read": "1b9cd0f48b40a423f1debc063e2ccff72b22e114cbcf557652e6ecad1c9192c4",
|
| 178 |
+
"layer33.router_w": "2cb3289236db97b651ff1a0d917bb3267d567bbf65cbf67b3cdba67efb2f01a7",
|
| 179 |
+
"layer33.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 180 |
+
"layer34.V_read": "fab302f6d5c2dcf2d11dd8d97f047dc5fc39377ccae00aae6074dc0d87bc718a",
|
| 181 |
+
"layer34.g_read": "8fa3487358ddbd8d7f01ffcb96a83a643fde88701f78ee68196bbc27774fc132",
|
| 182 |
+
"layer34.router_w": "4f9edad72ae32340f69aa22b371ad46b8866460e7e633910cdbeda92d7b09db0",
|
| 183 |
+
"layer34.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 184 |
+
"layer35.V_read": "37dd7050a6c03b2d6c8576fe0976077a43ce258d159afbb15f9d3b6eae092187",
|
| 185 |
+
"layer35.g_read": "8f79acf046acbca6f2e5108b7b53d283ff164b79d07ddd12091ad1e4ce619578",
|
| 186 |
+
"layer35.router_w": "8f30c37d0784ea6d3347ec5e38d8763697ca8ebf61c8509e8e0f1c7bf0fc99e4",
|
| 187 |
+
"layer35.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 188 |
+
"layer36.V_read": "5b9ab1226710787592b3fd515fa0c79197e12a2f1963000d8487b360728e2a44",
|
| 189 |
+
"layer36.g_read": "f56be0851ba3de3be933beb07e49881add6c2d0501e36efc7f391cbc81331e60",
|
| 190 |
+
"layer36.router_w": "0a64c450ae5e238b9e0cbb804fe44d13566f06934bc7ad07e5a7b33a4ebd9c55",
|
| 191 |
+
"layer36.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 192 |
+
"layer37.V_read": "e4e630278f9cf7b4cc06d3a4572802ab09fd939931095ae8fd36bf84d1ef69cf",
|
| 193 |
+
"layer37.g_read": "e9f179ce940df2e6a86ba4d6089d110abd27528e92a55eb88e0e678038eeebe2",
|
| 194 |
+
"layer37.router_w": "be73e471e4217911977da45fb18cb1843bf7a07bc59a5b81d9c7ed32bffe303c",
|
| 195 |
+
"layer37.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 196 |
+
"layer38.V_read": "0220e99471aac1a35ef7f17e0969481f9093be0ee5d68c8e1ba160b8475539fd",
|
| 197 |
+
"layer38.g_read": "d0623a01efe8af6efd8afcc808f23efc24b071e69f9d85f28cf3f5a5f2a7c248",
|
| 198 |
+
"layer38.router_w": "e35efac6a118f723f5628051aa40df067164d9a7a0422ccd9e25d6671eb61902",
|
| 199 |
+
"layer38.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 200 |
+
"layer39.V_read": "20db41719bffcf75bedfc9e2e8341d6857b003e7b0eb4267dfd31c86cc56300e",
|
| 201 |
+
"layer39.g_read": "cb7801a184b1bc83380021fa67e160e00ecc33c695aed80bc974822493cb683c",
|
| 202 |
+
"layer39.router_w": "2caa083f0ce4682a5eb31bfc27b54616beefea540d2d182f7bf09944f205acbc",
|
| 203 |
+
"layer39.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 204 |
+
"layer40.V_read": "f26e7cbb00c98256fa43291a11ed335ef009b4ee96941c300a0079e976d1161e",
|
| 205 |
+
"layer40.g_read": "9a4602baa58acc0b3f64c642ffdb395de49576b73860cc5cd0d6929ec59a789e",
|
| 206 |
+
"layer40.router_w": "ced1c46686c5d2275481b692f8a23002c67e75408e255c148cb91ff693f4da7f",
|
| 207 |
+
"layer40.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 208 |
+
"layer41.V_read": "9270e10c0381b11aee2cd22b9e8b89f6610eff8c6f14a5e09b2e4b60a1161641",
|
| 209 |
+
"layer41.g_read": "dea048bfb55201b82e06d7ee1be785e6b8683a4dec14cf6bbcf57b07eb9402f4",
|
| 210 |
+
"layer41.router_w": "877981b64edb45fda12d1114887f093726fc44b4eb4a7cf806290e048c05d51d",
|
| 211 |
+
"layer41.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 212 |
+
"layer42.V_read": "2336f9a50806cc029dea41a23846369b69323b28793f605ca6512b8a809191b4",
|
| 213 |
+
"layer42.g_read": "731ca130ed585cc26cee18f8944e438d85f9e3455adccb3f1e326769ecc25fae",
|
| 214 |
+
"layer42.router_w": "2c2f4edb7b74f5ab9f53515a6eee49629e8ff372e2681dbdc22451b946fe065f",
|
| 215 |
+
"layer42.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 216 |
+
"layer43.V_read": "be633b5c6a65efbcd08275bdde90edc5105a5923a7db06545227c5687b2ab084",
|
| 217 |
+
"layer43.g_read": "6a0c1f2210e4cb409beb386ea1665929363785f4a2675fa79419b841b2f9a98e",
|
| 218 |
+
"layer43.router_w": "897bef6f5fdcfb72f743f8937197460307cfe7fd0689e93a565ed9cb5a18c11c",
|
| 219 |
+
"layer43.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 220 |
+
"layer44.V_read": "12240bc8f29cad323a0971a942e3236df552c9baa5d6b65225e603b32a105cfb",
|
| 221 |
+
"layer44.g_read": "9ddb3a4153cb876deda7f13a3694a9dbcd75593f9bf9b2f624f943225009a42b",
|
| 222 |
+
"layer44.router_w": "15852502c73f82718d3608f61506497038d55fefac716ea26ea30754fc4b6a44",
|
| 223 |
+
"layer44.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 224 |
+
"layer45.V_read": "c6e259d615a04224c603ad1560673cead08ef91d700a7c68b7a4bf51c242bbd6",
|
| 225 |
+
"layer45.g_read": "471b70bdf722bc78475ddb946bfb4fe5480640a9eea8ceaf763688a7061d3ce1",
|
| 226 |
+
"layer45.router_w": "cea33b64eb22142e9a441fc52c04e79351fa32004c43869545ddbaa78991aee1",
|
| 227 |
+
"layer45.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 228 |
+
"layer46.V_read": "febdf1a3c15e2a46038a9ddcaf8cdbfbab871bcb4133e0fe2eb3c3c8993eb4f0",
|
| 229 |
+
"layer46.g_read": "3007c79fd9fd7f63340c19994495e9303decf9c56ebbe0d71d2514a6c3902e15",
|
| 230 |
+
"layer46.router_w": "55e05ee5bd460dd3d42d88e67e63188faa078c93611c37cc5ca00520f2b2d657",
|
| 231 |
+
"layer46.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 232 |
+
"layer47.V_read": "c9774380497d9338e6bb325a86fb7e9406080b96ffe66627c5730d2e746141fd",
|
| 233 |
+
"layer47.g_read": "e3e2e19243ff02c0a536a6f3ee2e47e8a91abed16a0cf8bf0dd766f54eb59a99",
|
| 234 |
+
"layer47.router_w": "83c101bcd7761d0f8b890eb22bc57b42b1b0cf74a163d3e520ad3a15fa460a87",
|
| 235 |
+
"layer47.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 236 |
+
"layer48.V_read": "01413acae1c48270083817749978d3afe4f5190895d7c8ecaba3b99d6f838c31",
|
| 237 |
+
"layer48.g_read": "69b08496e23f518458163e80a872a913373992a56b0edff3b5864392f3df002e",
|
| 238 |
+
"layer48.router_w": "b71c2a0e5f4d476df429d72c971c3502dad832f4ef24400830aad1e90b9d1c21",
|
| 239 |
+
"layer48.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 240 |
+
"layer49.V_read": "6e6a1fc67654e3902c9c28d31e6877c802ba95968913a265897ef4a40c06607a",
|
| 241 |
+
"layer49.g_read": "5a3e9324e1243877ac164f9d0c0bd1ae9ece80e2f15c9acda6f295b1500308b1",
|
| 242 |
+
"layer49.router_w": "36652898bd574b54803b72acce4c6818770e1ef70b9605512ca5601d4abea612",
|
| 243 |
+
"layer49.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 244 |
+
"layer50.V_read": "10d5de7fb1e594cf7e44d009c0d08c7360aaa969b6cd6bb3abb20d0fe53784d0",
|
| 245 |
+
"layer50.g_read": "8e2b9fdc0f9296084ceac9fe96e79f937293b653bf276d16be876e960faccd97",
|
| 246 |
+
"layer50.router_w": "c37f8cd0776a306f5c0cc726384f23b19c94f499ed0cb95eaf156f8593584234",
|
| 247 |
+
"layer50.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 248 |
+
"layer51.V_read": "d3787ce0b8489582692bbd2f4a67ef1380098ff784f18898e6077fb812f54e83",
|
| 249 |
+
"layer51.g_read": "55359a2bb1cad616dfa055bd20da9c52142a0e37f9382eff9ee8dbedebf79977",
|
| 250 |
+
"layer51.router_w": "6b0225c43ef7db1d0415f3b67fb627b2fb098626c427d3ff57b90d9a7fbde3fa",
|
| 251 |
+
"layer51.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 252 |
+
"layer52.V_read": "4b5ec34ca0da5d988a58e59d4b6aea701e47c2d69314d3f77cf19c889740298e",
|
| 253 |
+
"layer52.g_read": "951a8e8554f1ef867b5d9ba072cfc139782cd9e8a657871058d7590fc334a6a6",
|
| 254 |
+
"layer52.router_w": "8fed96881d126a411d6a35d9bfd13282101dda66e90cdd063bd56136d3216472",
|
| 255 |
+
"layer52.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 256 |
+
"layer53.V_read": "0cddc47ef8a9d869eb3f1302889e11203394ed4e56923addc654d38419238fd4",
|
| 257 |
+
"layer53.g_read": "dffca0ba32359bd619327426d6d616e85db319b7316fae5628e05cf37960dff3",
|
| 258 |
+
"layer53.router_w": "6d3a53968652f6f05b51b4e9e2de007e3b56cb691db022595e59608e37ee07e9",
|
| 259 |
+
"layer53.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 260 |
+
"layer54.V_read": "a9dd4f7a3539ae26d868748682b65cca71dd135644a4c918571fc9ff930676c5",
|
| 261 |
+
"layer54.g_read": "8e131fbe86a8a9e3215b4f3b381a255921f10b06566d020f330a276f8af8c4de",
|
| 262 |
+
"layer54.router_w": "5f06feb297a625ab67df5ecf3990956681c60bee17f1e19e611629f41c9e7299",
|
| 263 |
+
"layer54.router_b": "de6343c9dac0fa5e944b0260a84812bf8e0cc2efd38014a76d85709a524ad535",
|
| 264 |
+
"layer55.V_read": "7d8e5fcbcf68199fd68b348b3f393c0a8728b765b600b68961ebef0c3090199e",
|
| 265 |
+
"layer55.g_read": "a3c954b3221b64c08fdd9b27f918faebd17f5b9dc9b6c6b978943bba180f4309",
|
| 266 |
+
"layer55.router_w": "48aa1e26ec214f74413e6763217efd089fbee7ecb6654c44ede62b948a1dcbbc",
|
| 267 |
+
"layer55.router_b": "1eb9418e6a22f0af4c2d5edd254d59768916f24ea54c54ac9d4cf45f74862628"
|
| 268 |
+
},
|
| 269 |
+
"gate_mode": "soft",
|
| 270 |
+
"roundtrip_bitwise_equal": true
|
| 271 |
+
},
|
| 272 |
+
"frozen_files": {
|
| 273 |
+
"mamba2_recall/__init__.py": "79b32606ac476c3248911c55571bf97f96689770067856ce8347837478ef33fd",
|
| 274 |
+
"mamba2_recall/calibration.py": "54e178521abfe2513fb7bac2aed2ef178fc01b63135222ecc212001ee3ce0a04",
|
| 275 |
+
"mamba2_recall/runtime.py": "73a405b4286dda403f252cf0b73d367adcfead96298a1e6f440b65f5b309530c",
|
| 276 |
+
"mamba2_recall/evaluation.py": "95cba0b201e37f92e6cbfadb9b3e9d79ba7b0cdc9efda6a58a6216b0d84d35e0",
|
| 277 |
+
"mamba2_recall/resurface_native.py": "2dd08c7ee8958c832f0ae9da7cf5f261dceeff3e528e493c905c7c52df7166d7",
|
| 278 |
+
"docs/PROTOCOL.md": "dee9457bfaca067b63837dcd3bc94bf9ef9a3d805b1ab04227db05a4b92c54c2",
|
| 279 |
+
"reports/source_resurface_v1_confirm_full.json": "4c99694d66af1db71924bd9d3113f1ea2f0a9b93191ed229f3d26f2320c8c940",
|
| 280 |
+
"reports/source_resurface_v1_train.json": "72c5c46ffff8e9981b8747a2fe10cb9ab67695d1555f2deb43d497ec7caed5e1"
|
| 281 |
+
},
|
| 282 |
+
"historical_ppl_split": "validation",
|
| 283 |
+
"historical_ppl_execution": "prefill",
|
| 284 |
+
"historical_ppl": {
|
| 285 |
+
"baseline_ppl": 7.334175947318572,
|
| 286 |
+
"adapter_ppl": 7.052063515606311
|
| 287 |
+
},
|
| 288 |
+
"source_checkpoint_sha256": "47c2766f6aad89d73beafbeaecb334aab902d7370906d081764a90bb7a8bbbcb",
|
| 289 |
+
"w4_manifest_sha256": null
|
| 290 |
+
},
|
| 291 |
+
"w4": {
|
| 292 |
+
"repo_name": "mamb2_8B_W4A16_Recall",
|
| 293 |
+
"github_source_commit": "82a1945e39a673da1d527bf42e9af7e1da81e648",
|
| 294 |
+
"hf_repo": "EndlessChasing/Mamb2_8B_W4A16_Recall",
|
| 295 |
+
"hf_release_commit": "e1baffa7aa680f3dc26d7d7df2104756248d1a24",
|
| 296 |
+
"adapter_path": "pretrained/w4_resurface_v1/adapter_fp16.pt",
|
| 297 |
+
"adapter_binding": {
|
| 298 |
+
"w4_manifest_sha256": "3add3f79f19d2da181c700680500390f773a47b2785d8f6e0ccaaf2ddd7bbc05",
|
| 299 |
+
"source_checkpoint_sha256": "47c2766f6aad89d73beafbeaecb334aab902d7370906d081764a90bb7a8bbbcb",
|
| 300 |
+
"tokenizer_sha256": "5862e2f71caf762bc9845662be5fec2867deb58d874568235a02a36c5111cd09",
|
| 301 |
+
"protocol_sha256": "9aaba5861cf818275acc1bad00c87ccab22d2d7fc3538f5f6ef5456e67c1d472",
|
| 302 |
+
"train_manifest_sha256": "3044ec372be012eeb1b51924fc4c93871e7623dfe94e7c6bd7ff3f7e87fbbe96",
|
| 303 |
+
"prose_manifest_sha256": "facb2ca461615a4199781bd21784d642d6674f5b862641b3b9edac3fb499b89d",
|
| 304 |
+
"prose_tokens_sha256": "e54b02e5162e042a9cdd504f4eb1b1652724fb240bbc2c97608967aa26297233",
|
| 305 |
+
"prose_tokens_int64le_sha256": "049dfb6847d778897fd19b67653c3189f29fbc19a36022e8a4801306d08228bc",
|
| 306 |
+
"prose_file_exact_historical": true,
|
| 307 |
+
"adapter": "MAMBA2_POST_D_RESURFACE_FP16_V1",
|
| 308 |
+
"successful_updates": 1536
|
| 309 |
+
},
|
| 310 |
+
"adapter": {
|
| 311 |
+
"file": "adapter_fp16.pt",
|
| 312 |
+
"bytes": 2374271,
|
| 313 |
+
"sha256": "6d38c24a102cbf83c5f1dfe4362a0b5c5e779e793a27bccd4e592ff6c9de7ca1",
|
| 314 |
+
"parameters": 1154104,
|
| 315 |
+
"payload_bytes": 2308208,
|
| 316 |
+
"tensor_sha256": {
|
| 317 |
+
"layer0.V_read": "d90097f8ad97738c1921e63ae335115c5ffa97cf70a1de9d58982e21a8277c81",
|
| 318 |
+
"layer0.g_read": "0a7d40285f7bba8132ee63849a5e5d8b09b0ce3c1e1320af4542c6c521d54480",
|
| 319 |
+
"layer0.router_w": "4605b00ea0ef1bd381bcebb491edc785e8bfce469b49f8103e1290906d214e74",
|
| 320 |
+
"layer0.router_b": "48029583821428761aa3d097456ec52ff94b9bce7a33bd788ae74d8918388443",
|
| 321 |
+
"layer1.V_read": "5dbc110276d170ac943c6f682448eb6096edd4d010b9c960ef03d4a70f1a6efe",
|
| 322 |
+
"layer1.g_read": "3941bae1c773b8a007048cf90c963aef9876f21082a5f194361aa4d3f91d831e",
|
| 323 |
+
"layer1.router_w": "0cc25c875b78afb98bd45dafa441b2c5e6319a1c9a352c8f46d74e6b008b36d5",
|
| 324 |
+
"layer1.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 325 |
+
"layer2.V_read": "db13d991a39ab9e43a8a9008f6db984ee93161a062ccae763d120753a20c0440",
|
| 326 |
+
"layer2.g_read": "c3a33116e131d9182e67150ea687c118492fcb33f93a17b4bce72c95227e8ebd",
|
| 327 |
+
"layer2.router_w": "bf7ac0046da5a2745f3e7a5ef4c222118f35f45b8082386bcd3b6767fce6618d",
|
| 328 |
+
"layer2.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 329 |
+
"layer3.V_read": "7a1eb9c6b402446c9053f5d5f91aa8de328a8c7ab1e4b20660b64543999c4135",
|
| 330 |
+
"layer3.g_read": "028eaf429f9611fc0f1cff1c350a522647c74a18f83ed4b73c3a4d6a0a3e2176",
|
| 331 |
+
"layer3.router_w": "1fbd197c280df5f8d4f22c46e0831c203d15e337bddd06697814cff46ecb967f",
|
| 332 |
+
"layer3.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 333 |
+
"layer4.V_read": "34ea8202e48954c7a941eaf920139efec0dabd4848d85a8e5e24b5a7d47394c3",
|
| 334 |
+
"layer4.g_read": "e588ff09407acd6e0cbcdb6427a98462e3dcf99b263832a184532431bf75ec8b",
|
| 335 |
+
"layer4.router_w": "4de397f1897096e1943d102385046670cd56471e4410aef2ad2ffea22c8ca0d8",
|
| 336 |
+
"layer4.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 337 |
+
"layer5.V_read": "074e798ce62aca37b9cda44e156672fb6af0ac3a82031305771c4622e479981a",
|
| 338 |
+
"layer5.g_read": "9a3dc2a0c7b1053ac74fb61f87fc04dc9918c5f6aa564442105bd4907fa27a81",
|
| 339 |
+
"layer5.router_w": "534c75cf8e08aed4ed637e5cf158dedbf27852e7e2ff67f5d94ffa15e5c1bf20",
|
| 340 |
+
"layer5.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 341 |
+
"layer6.V_read": "d8a5607f72548b5fb152021b8d01b9b0e1dab36a8da642db117d00b2b4bf2371",
|
| 342 |
+
"layer6.g_read": "2759dcedb03ed99793b4b5e8c404edb734589ad70414049b0ab2955cb23db56b",
|
| 343 |
+
"layer6.router_w": "6cfc5e066f0f996406472f43fba8c7da3747c7bcbf09d50ff8d16cd314a4d77f",
|
| 344 |
+
"layer6.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 345 |
+
"layer7.V_read": "ad93a2545aad2a1e63e996a4ac5d6317065863076391b6ebb08599c9b241116b",
|
| 346 |
+
"layer7.g_read": "6499db5b8bdd0e3e2e768c1b697ded2f5cb0d70f1aeadc94f483f234294e93eb",
|
| 347 |
+
"layer7.router_w": "86f670c6f76882b6be41f8bc75085f9279465dd4d07375890862ed7c6b219e48",
|
| 348 |
+
"layer7.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 349 |
+
"layer8.V_read": "f951b8fcb99f3846b8d1071b3503950b490e4402395971481ea7996f0cd0de03",
|
| 350 |
+
"layer8.g_read": "f276334650d217a86422fa47e20d43fd479a86f777aac224bf1dedff006ac5fe",
|
| 351 |
+
"layer8.router_w": "697684e68289e44359547ca48e35dd32571b9103238e47df57c3746be4cf7e4d",
|
| 352 |
+
"layer8.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 353 |
+
"layer9.V_read": "9abc069cf03ef89c811c9d7e8503e946ec3135f2a3acc3f3d3783345049fc4d2",
|
| 354 |
+
"layer9.g_read": "0a643d3355008f855390b199afb344704c66dc5b776d67c34346e576954c9872",
|
| 355 |
+
"layer9.router_w": "b09121ec6c32e851aba6708ef651ff99fa6a7ccfc4a740b8a6fc287a76ddecc9",
|
| 356 |
+
"layer9.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 357 |
+
"layer10.V_read": "9a41cb40e93342b9d81e0b173c63ab27737143626887039e0579eca7477f4275",
|
| 358 |
+
"layer10.g_read": "ab8376c4821a334b1ed803b87fb93acd541343222dbbb4842b0df3c80e926ba7",
|
| 359 |
+
"layer10.router_w": "60fba90afafa615ca394ff7ffa04736ab2372602d3da33f346093c8a931bbddb",
|
| 360 |
+
"layer10.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 361 |
+
"layer11.V_read": "a0e63dc1ab20e0b18d5a6fa955f2593c0f049d9702cb351e9270ace5647a8257",
|
| 362 |
+
"layer11.g_read": "4bd6f75dd3e239b806ac5b69a3467c69722df9c5bd9309d5f912b792e70540de",
|
| 363 |
+
"layer11.router_w": "72ead3751b06074fe147203c7d9a61186e73b13b9c417d177ff8061a3d0a75c5",
|
| 364 |
+
"layer11.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 365 |
+
"layer12.V_read": "a60c0970890eaf377a4ce9d25445266296e08595b24681c4eec7c9c82e60d7d3",
|
| 366 |
+
"layer12.g_read": "5b52628affb87c83766f5432bed3675656fca0f888554e126fc4eb13752122ae",
|
| 367 |
+
"layer12.router_w": "f7a0ee1456aa4a99cc6015a53858105b5215eddf6a8d95b3913aeb47001ddd11",
|
| 368 |
+
"layer12.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 369 |
+
"layer13.V_read": "0d9d4dbfaa18fcb34cf8eb1c4506b9e03513237e02b61df767540bff3ac68d2a",
|
| 370 |
+
"layer13.g_read": "bc744f1c22c224fd7660fe5077ac1a87d199fcf1ff62f4e7872bfb33d5c6c1b3",
|
| 371 |
+
"layer13.router_w": "82ff138912346cf0179d92b6447a623b4550ff80aa40ec7e149e39e009b277d8",
|
| 372 |
+
"layer13.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 373 |
+
"layer14.V_read": "3ddcd25f014ff8f10f4835cb1a0edfe60cd8cdf73e33e3202ec18f81fe3e4e39",
|
| 374 |
+
"layer14.g_read": "7f375a2efe46805b0cc2445585d1e21048e05e78a9d7f680840f5f37a9985d2e",
|
| 375 |
+
"layer14.router_w": "1884a6225aeb85b5d15bbd74806b0521f83957e53221cecbc65f8a456de8bff2",
|
| 376 |
+
"layer14.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 377 |
+
"layer15.V_read": "cf9a438ab4ae96ec1069a54ed6a6eb471385195de8954625b37b139c4cbfb638",
|
| 378 |
+
"layer15.g_read": "409a74a0de73e0d448309f1cb6649f39f9da47dfeb89404ae7cb0908589c9655",
|
| 379 |
+
"layer15.router_w": "aeba708edd39da3e6ceaa9f688c47e1b7644e47346e028534fbeab157b01c641",
|
| 380 |
+
"layer15.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 381 |
+
"layer16.V_read": "8c674b0f299cb619daacf72b9b9daf7684128af15bc5c181e95b70c2f94668e8",
|
| 382 |
+
"layer16.g_read": "d58efc4d05f17d14dbfd0fa9c68122abcefb93b9a3bbdcecb5ae3f0e3f46f4a0",
|
| 383 |
+
"layer16.router_w": "5310576aeddf689bd812a6bbffd52d216b151beb60e00001e84c13c94820e05a",
|
| 384 |
+
"layer16.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 385 |
+
"layer17.V_read": "8b610fcd1d797e7438620f4c5a3a916fc7d9ce2135cc2d1fdf3c1151e6a49dec",
|
| 386 |
+
"layer17.g_read": "1eead328ca9658f0f6b0f7ea49773df434f22cc148e288b149f47d427e9cee3e",
|
| 387 |
+
"layer17.router_w": "9db86f8fbc99dc560c124c6090d7d5cb7b9d55843d2b4778c54c76f0a83923a8",
|
| 388 |
+
"layer17.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 389 |
+
"layer18.V_read": "040674c78556471e5ef6ef9ec9ee188a20ac1c56eaf6dcd6344f7504cadbf0b3",
|
| 390 |
+
"layer18.g_read": "66ba031669a38a83477f53326ee15042c09f7fad128bc059fbc9d36cb55e7929",
|
| 391 |
+
"layer18.router_w": "9967a05683265afba9e9968a632f2ee1361c73054769e15b1b06acf2aa6a69a6",
|
| 392 |
+
"layer18.router_b": "f585636255da971019a4ff67ee970b644ef4d4f493a597af54cd4390f07fd033",
|
| 393 |
+
"layer19.V_read": "36c42ec12c2650f2b7af612f5c658dc926713ad651eb6c911cd9a96404bfc2ef",
|
| 394 |
+
"layer19.g_read": "34fc8906943417038db8cf7e75076567087dcc11a1c3047cf269778c765c838a",
|
| 395 |
+
"layer19.router_w": "4294b2b1fadbbd0fe95f8a00de3c043879d6c3d5f3d1a8e388a58c6bfc981a84",
|
| 396 |
+
"layer19.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 397 |
+
"layer20.V_read": "e19405de4445eae02addb60cc3bfe425f114fc0cecc408e9ad2c21e04fa5b40a",
|
| 398 |
+
"layer20.g_read": "28cf1699c856f9509f81479c9da7bb66b30383f05d0a2ed650a4cf31d77896dd",
|
| 399 |
+
"layer20.router_w": "a6cc1e679d28bc3da533201bbdf07a0e033453c9d2a870732d50e4231270f233",
|
| 400 |
+
"layer20.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 401 |
+
"layer21.V_read": "4b7d50d8b561d3dec6a71c5ccf45a98ae7a54988fb950f036860b97442ff9f58",
|
| 402 |
+
"layer21.g_read": "d1d6dbc4c1b1e1f9b1fd20d6373cffa5936964392861f832273c9dd73a2b27de",
|
| 403 |
+
"layer21.router_w": "e361070778f0eee049ff68d6e05da68bc6c3ffc0bdd7c28a5b143d98ebcb8a1e",
|
| 404 |
+
"layer21.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 405 |
+
"layer22.V_read": "9e4a8cb97ccc830df507a362a08d597d03b26629d28ea30ff42282efc64621d3",
|
| 406 |
+
"layer22.g_read": "865fb5f7c1c6cf74bab155a05d982d304efcf7f062f9d02b19d2abd7007b157f",
|
| 407 |
+
"layer22.router_w": "d676eef5641a74e784088c25a07844a81df2250febb941c3aba717cb0377906e",
|
| 408 |
+
"layer22.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 409 |
+
"layer23.V_read": "2388b9724e0122d677866ead53fd83f0200611496d5e8ff38003f58efe87aa5a",
|
| 410 |
+
"layer23.g_read": "b9c9cafedd9619ac93685873cd4171565aa0fdd9457ecc17462a2aabb8875989",
|
| 411 |
+
"layer23.router_w": "b530476f16ab481f27d50a1a686907f04c55ad1b9df7bd1d2265b2aa29f9bfe3",
|
| 412 |
+
"layer23.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 413 |
+
"layer24.V_read": "547dfe3eb0ff4f82b15e3e9c2ea83ddef3fb6a8960cc560a9e734fe2e86cd819",
|
| 414 |
+
"layer24.g_read": "2495618756c645aec3db31328adbe070084069ef677eef40f6fd36d3b4a915b6",
|
| 415 |
+
"layer24.router_w": "9c21d6ab18b2cc2950ffe8ede9f4787a52f0e418357560f841a42a3adcea3e2a",
|
| 416 |
+
"layer24.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 417 |
+
"layer25.V_read": "c4222a49258e7916cd44a48243c751ffc20541b6648f0e2fa1916c71db1509b7",
|
| 418 |
+
"layer25.g_read": "088504562ebcacd96c35da19fae6fefc9770263310ad9cd4a92abf9da726f0d8",
|
| 419 |
+
"layer25.router_w": "cceddacb67abd2f7751ba91b692727eac3cdb8a27ed3bed95a5fcde1de551a78",
|
| 420 |
+
"layer25.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 421 |
+
"layer26.V_read": "d8c304f2e931d1bad875459a42169c6252edda2502198ee49529c29dceb03866",
|
| 422 |
+
"layer26.g_read": "7d1eee53b9f52970539dbebaca6ded860b2a911b84fb46b255d8e69bccbe88bf",
|
| 423 |
+
"layer26.router_w": "ed867cffca703c57b6ff51bedbd4592c20fe06159ad93002e8cf555bee4ad43e",
|
| 424 |
+
"layer26.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 425 |
+
"layer27.V_read": "0b9b2c658f1faa44a67735e8cd22a7e11ce40e3b26cd4abb5e89e157b6acf7dc",
|
| 426 |
+
"layer27.g_read": "fcbcaa268dfb1c9acb5f8abecc5e4b63b79ebc36355ef1a57912ded9bca29787",
|
| 427 |
+
"layer27.router_w": "723d48de453cae66c326fe1d1f9cade101847242761e25681021fba102116240",
|
| 428 |
+
"layer27.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 429 |
+
"layer28.V_read": "037ffac22893e16b3100ed86084095b483e9bbd46e7c1115e039b1ed1900d104",
|
| 430 |
+
"layer28.g_read": "8c804bb0d6531e194c25f953ae4a945223af595c20794eb10b3b64da1b3d9c7f",
|
| 431 |
+
"layer28.router_w": "935a7948ec0a128b1165c7a16b14c5349dde9b7742d113d6d15c3393ca63bb29",
|
| 432 |
+
"layer28.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 433 |
+
"layer29.V_read": "3eba36ed82141fcebb9f5f68ff090095a3891d73111ffab572fb0b30f8fed70e",
|
| 434 |
+
"layer29.g_read": "760bfe42792b3fe82bb39e1f942c3c678c7d12bd4c7de921e24ff8f72a9c3826",
|
| 435 |
+
"layer29.router_w": "d0cb67afe18c948ff83cb04124116b0233148cfc779a652385ed72cb2b2654f7",
|
| 436 |
+
"layer29.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 437 |
+
"layer30.V_read": "514587e88347ed06fa1d07427bea92baf73784edbb6bb967c8d93f1e6718cb0a",
|
| 438 |
+
"layer30.g_read": "1a062ac14b8c864b357051ed0416409dac6427104fa96ed0d19e90748d7d796e",
|
| 439 |
+
"layer30.router_w": "5eb0ab72768db8dedac0f5ce7d0161c557493e08aceb34682e2f86b7f4470cf4",
|
| 440 |
+
"layer30.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 441 |
+
"layer31.V_read": "260d54c113a9a122cd226e77a2a39d4ce5b25415e3b39f0c8f7e36e9c5f501d0",
|
| 442 |
+
"layer31.g_read": "2864f4577194e09abb15a21a39aeda496bda3de00e310fc571bad3b0baa7a8c6",
|
| 443 |
+
"layer31.router_w": "ba881859790f512ce99c0a5b8971f9ab8e9cbfb56abb572fe32381e686ddbb89",
|
| 444 |
+
"layer31.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 445 |
+
"layer32.V_read": "4de9c9f4b2cd424669193f4429af5aa4c6e6655c782be8a533a96fd3cbba8dfb",
|
| 446 |
+
"layer32.g_read": "00c61d8728d79286f457d90a512cc3698a2816783e4d6e8460df9f91ad1e7531",
|
| 447 |
+
"layer32.router_w": "070abbd02b2b64f2479649f8e49c319174170c530902b0a7373a375ee28b08b5",
|
| 448 |
+
"layer32.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 449 |
+
"layer33.V_read": "c8585d9d2b6a636a9e796fbc90c04490a938c3eecd8fb3d0b250844d3a146e75",
|
| 450 |
+
"layer33.g_read": "e260f51c2ab51e37e38f3d6bfdc7afa59acb98b938dccccf6124ddd18718f59c",
|
| 451 |
+
"layer33.router_w": "018d531fe2cbb9416fa1a10cfd5001696a121a6b913d6ff26d163740681bf83f",
|
| 452 |
+
"layer33.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 453 |
+
"layer34.V_read": "cbce745860da7c74664b5b8279610bfe43432848b2baf5f168cf778a220c4d38",
|
| 454 |
+
"layer34.g_read": "56079fbb8b0d8aaf488cc7d6f883e753db6d1238d3ec2e783d56d86be33f2eea",
|
| 455 |
+
"layer34.router_w": "a7ec3923ae978d8e42de7ebd56a897b82cf323ac8f82801c925dfed6a273925d",
|
| 456 |
+
"layer34.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 457 |
+
"layer35.V_read": "46387a8ced0e36efbe0eb421b6c7c551068c27847424e78f950ef9e26cdad7e1",
|
| 458 |
+
"layer35.g_read": "a82c60b857ce048632634dce8a91de152955b493b56c544129a38eff4c6eaa41",
|
| 459 |
+
"layer35.router_w": "db30c05b0336273206ac770f810e647420522dc8e905f64a4698ecfce8281b9c",
|
| 460 |
+
"layer35.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 461 |
+
"layer36.V_read": "14b07e97e8f9f34f2e1c981589cc6a8c45184da16879a6909f98788bef5437aa",
|
| 462 |
+
"layer36.g_read": "13e81b6e123b00910af383f854299c929d8f0c2ba4659b3e6d4d5eec7464f6e1",
|
| 463 |
+
"layer36.router_w": "6f58fe3847db9e5db1e51b74826521cb67590d584fae64a0f64b4de52e45d7bc",
|
| 464 |
+
"layer36.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 465 |
+
"layer37.V_read": "8500c1b7f5e4b368bc9abc5b456fd08ec83076a2f999fe070dd02105ad535fc1",
|
| 466 |
+
"layer37.g_read": "0daec369c8fe061af451869d5f4ee15a422f417c3660479ef3505ba7eebb0697",
|
| 467 |
+
"layer37.router_w": "b982bc93091a25bdf18a5257dd753af09c8218bc494b12d14ecf627b1dde15ab",
|
| 468 |
+
"layer37.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 469 |
+
"layer38.V_read": "d3d9df818b7b915377e3ab8d2ece537580ad7c6c1b331b7c3ecc3c207f4415ca",
|
| 470 |
+
"layer38.g_read": "06913e0bf10ec5de2d22ce1444b81f1f71fe57ae6b82ef339b069a3c1795f194",
|
| 471 |
+
"layer38.router_w": "550386929621eb29501f360580ee362921c34373f00355466b95a0d1d8efe21c",
|
| 472 |
+
"layer38.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 473 |
+
"layer39.V_read": "c0ae06aaba2e48eb73809aee3576e5b3f203c09eda410e7f91cb255f79f092ac",
|
| 474 |
+
"layer39.g_read": "726f82363addcbc3a894d8417b6531d3c96eaa20ddc83f4e23394728d94a0626",
|
| 475 |
+
"layer39.router_w": "d20673e9c397f45fe49489bc5c51846695a2ff098cb5ead19474a4fe87fdb59e",
|
| 476 |
+
"layer39.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 477 |
+
"layer40.V_read": "dc567144fce5602cc4848eb7b10aa1ab2849128557dcf117ab8a85b9b49595aa",
|
| 478 |
+
"layer40.g_read": "2f528015be3d10c3d98b4f0068f2f586b3648906a9609c224c1be0457fd49653",
|
| 479 |
+
"layer40.router_w": "09cd89d3b06e62cfec7e35acdbc502a29e472606347df9d27f1473d49647e128",
|
| 480 |
+
"layer40.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 481 |
+
"layer41.V_read": "38da642a1ca47e6bb8cf99956af05dfb12fedc57b93c70446569c7f8451d00c3",
|
| 482 |
+
"layer41.g_read": "fb1a2a906b960ae8f2685cc27122f5a00930df89bda25c0557b45f670882254a",
|
| 483 |
+
"layer41.router_w": "734787cef3a4801dea58871eb19eae1667c2f33d414ed2507fd13ffa16879036",
|
| 484 |
+
"layer41.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 485 |
+
"layer42.V_read": "5faf519ffba85ec065f32cc6bd996d7cc73b8850391ac9fd02c7849903cd71fa",
|
| 486 |
+
"layer42.g_read": "9dcfe92d22ed393342eb6701fd0d0e0bee4ea43f0fbb8c1dd8cc4e5c8d7da092",
|
| 487 |
+
"layer42.router_w": "dbb9462003ad307bfcd012df445085fe5b9741787e5e826b3ddc0909408165a1",
|
| 488 |
+
"layer42.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 489 |
+
"layer43.V_read": "62798e186bd8187608f92b4ea72d38078e9b90e512449e808143f0a49ad95fd2",
|
| 490 |
+
"layer43.g_read": "785c378f92b2664af346dbefd98bf34d908a20e75bf223b164fc298bf38c4306",
|
| 491 |
+
"layer43.router_w": "220bcd27051bb9fa10a451ee4e2c97cb062f0a2221564326cbe3e0c5eec8d4ee",
|
| 492 |
+
"layer43.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 493 |
+
"layer44.V_read": "2468b185cc8c18fc21006989adca9df17d9212389dbe8c41bc5383ce2839c659",
|
| 494 |
+
"layer44.g_read": "eea77281568c50ef08c80ea97ad15a58f63434651959048531eb9d4e1e82216b",
|
| 495 |
+
"layer44.router_w": "37d580e99bb5fe1fa2341527527eca20df00d7fee654e3913b42d2d9ce006c77",
|
| 496 |
+
"layer44.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 497 |
+
"layer45.V_read": "aaf5b6b8892627c4bdd00160f9187a6aeb5aa63dc9afa95316d706a2cf0e6c3b",
|
| 498 |
+
"layer45.g_read": "d722e32431c8093a692db68794371487c523342768ea7c4a745cd1c9b67baed0",
|
| 499 |
+
"layer45.router_w": "72c4a00d025af4ed5c3d3145c0d3a71fe3e49870b94dc3ee1408d39f0fb7010d",
|
| 500 |
+
"layer45.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 501 |
+
"layer46.V_read": "676e8ea9916f44bb9a3abf12437b4f589a3a07412d15d179cb3da2fb5c1ab984",
|
| 502 |
+
"layer46.g_read": "6a85a8e157a2460468884968ca11353e55ae44b8129c2713f8efc74f0cf38b32",
|
| 503 |
+
"layer46.router_w": "0c87927aa3548c680c027616b849cd094835fb7350a200558366bf2dd1de0d34",
|
| 504 |
+
"layer46.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 505 |
+
"layer47.V_read": "65078a7a8c857e824db75060f957367d56daa53a8cf99fb71f3ea98239c7fba1",
|
| 506 |
+
"layer47.g_read": "342a58ce217f8032528ba647032ced3fcfb1efc5399e53a332608d2dd4b9bf0a",
|
| 507 |
+
"layer47.router_w": "cb56daa92d22e199ddd95ef06c0e22dfb8ed4ccc90ea77ca2ba3419d461fb0f7",
|
| 508 |
+
"layer47.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 509 |
+
"layer48.V_read": "5e29a6383b4813cd470675fcfe0cf67fe24744b447b3c2b9aac07f5f058ac93b",
|
| 510 |
+
"layer48.g_read": "af19f703973db6b5fd088332260bee557a6375bfc6a67e78ccd54166918772d5",
|
| 511 |
+
"layer48.router_w": "7c800168317d485e46740ef1264f2ff7852fcf5b61b1a6dd7bbf100d04c4f113",
|
| 512 |
+
"layer48.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 513 |
+
"layer49.V_read": "ee1171a1e5ee30e8d2b17da28843c9987699f36f1cf7b10d6c33ba81483995c3",
|
| 514 |
+
"layer49.g_read": "664de3eb3cd0b722e264bf80d223f87bf603f77b45907250931d01f1b5bc90e9",
|
| 515 |
+
"layer49.router_w": "44c19c14efe63530b12eedf7161d0a50f8c504a76e26849fb01bc07d963586d4",
|
| 516 |
+
"layer49.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 517 |
+
"layer50.V_read": "a4ac93cf941b7138c6f893659e79d59fdc5b26eb63eb6674722ae3fc7e35954b",
|
| 518 |
+
"layer50.g_read": "e8d4c25daf2a7361ed66ffd4b434da0f0a8ab7ea789f0515394feeda5bb85fe3",
|
| 519 |
+
"layer50.router_w": "c7424437f433206150bc31fb18d04722de97303c72e2724a62ac2502e0de13d7",
|
| 520 |
+
"layer50.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 521 |
+
"layer51.V_read": "f394cc7fd22ee45f7bf83a7203b511e193fddf4cd06ccef37059d78d1095ca2c",
|
| 522 |
+
"layer51.g_read": "6562b739a5c435cf9e6bf6b52a5abd17d3ce082c8ccd4a6dc2dec6780f524fd7",
|
| 523 |
+
"layer51.router_w": "5a2fbe73c9b7fb1c4a741ccdfe61f3608302c712fe5e1bd26af0ede2ca54727e",
|
| 524 |
+
"layer51.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 525 |
+
"layer52.V_read": "7273a268079bc1bc14e84ea51dd6436ac73c629958fbc20e091f4c105a3942f1",
|
| 526 |
+
"layer52.g_read": "7d99290830f85a76fc58ff78f18559734cace3840a33c46dac2e6d90b3e542aa",
|
| 527 |
+
"layer52.router_w": "80b451dd6d83d112e86147a7524872f3fc2171ee16f5322b4307333ae6deabe3",
|
| 528 |
+
"layer52.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 529 |
+
"layer53.V_read": "65811e94bd71878c7fd84cee6ac77da543173d012325ca9d8f605bacf3ab50da",
|
| 530 |
+
"layer53.g_read": "e4776fc3b29067603dfe7d3026f023eb14eb8a60b6ec9e906dd4b4321e34412f",
|
| 531 |
+
"layer53.router_w": "de452bec39aa72dc89b505bff9e2406467f99907391e7749580b75ed4a5b9cec",
|
| 532 |
+
"layer53.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 533 |
+
"layer54.V_read": "01ee7a38a0aafe60d29689813e907017048d8856be3020fef38696261b9faf8e",
|
| 534 |
+
"layer54.g_read": "da4080beba42955448a54e7000f825a167829fd5d8d66220eb60cc0a0a6f31a4",
|
| 535 |
+
"layer54.router_w": "0b9ad8721d97265b49e5a565a119e3232df089f5c107b6bcf104cca89526b0ed",
|
| 536 |
+
"layer54.router_b": "1eb9418e6a22f0af4c2d5edd254d59768916f24ea54c54ac9d4cf45f74862628",
|
| 537 |
+
"layer55.V_read": "5fa7aaf28048b03c2aa2ffd8188eede02d7a09a028a627e6ff6240a5f60fa1f3",
|
| 538 |
+
"layer55.g_read": "7d24a081710f1f135cc2f99908f4861ca8a10bd41f5844b3d8f1b281d5faf2d7",
|
| 539 |
+
"layer55.router_w": "88c3d5e76379fde6c15de17362d1d578c1f65e83d8b03c2234f5002008f3bbcf",
|
| 540 |
+
"layer55.router_b": "4acf16f8c69e905610638b3a55b23e51c5f08fdc98aa160c8624a2962334adce"
|
| 541 |
+
},
|
| 542 |
+
"gate_mode": "soft",
|
| 543 |
+
"roundtrip_bitwise_equal": true
|
| 544 |
+
},
|
| 545 |
+
"frozen_files": {
|
| 546 |
+
"mamba2_recall/__init__.py": "79b32606ac476c3248911c55571bf97f96689770067856ce8347837478ef33fd",
|
| 547 |
+
"mamba2_recall/calibration.py": "54e178521abfe2513fb7bac2aed2ef178fc01b63135222ecc212001ee3ce0a04",
|
| 548 |
+
"mamba2_recall/runtime.py": "0c84d51c10a08318f6ab7cacf275f739913a6744954de838c8f06c3782e75ad3",
|
| 549 |
+
"mamba2_recall/evaluation.py": "95cba0b201e37f92e6cbfadb9b3e9d79ba7b0cdc9efda6a58a6216b0d84d35e0",
|
| 550 |
+
"mamba2_recall/resurface_native.py": "2dd08c7ee8958c832f0ae9da7cf5f261dceeff3e528e493c905c7c52df7166d7",
|
| 551 |
+
"docs/PROTOCOL.md": "9aaba5861cf818275acc1bad00c87ccab22d2d7fc3538f5f6ef5456e67c1d472",
|
| 552 |
+
"reports/w4_resurface_v1_confirm_full_reference.json": "f49f205ab684f883a83d3122b1c84904c71c09ac4c5ed197b137b5c5c423da41",
|
| 553 |
+
"reports/w4_resurface_v1_train.json": "df57b37f2fe13f79cbf78f36bbb3f29a034357e28f2a2c729b7ff5f13f748f74",
|
| 554 |
+
"mamba2_recall/w4.py": "e1fb34175dabee56092fec7e627503f57d2005ff0abed52035ba661bc4ae1d79",
|
| 555 |
+
"reports/w4_base_v1_manifest.json": "3add3f79f19d2da181c700680500390f773a47b2785d8f6e0ccaaf2ddd7bbc05"
|
| 556 |
+
},
|
| 557 |
+
"historical_ppl_split": "validation",
|
| 558 |
+
"historical_ppl_execution": "prefill",
|
| 559 |
+
"historical_ppl": {
|
| 560 |
+
"baseline_ppl": 8.012095592914347,
|
| 561 |
+
"adapter_ppl": 7.614046815850065
|
| 562 |
+
},
|
| 563 |
+
"source_checkpoint_sha256": "47c2766f6aad89d73beafbeaecb334aab902d7370906d081764a90bb7a8bbbcb",
|
| 564 |
+
"w4_manifest_sha256": "3add3f79f19d2da181c700680500390f773a47b2785d8f6e0ccaaf2ddd7bbc05"
|
| 565 |
+
}
|
| 566 |
+
}
|
| 567 |
+
}
|
docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PROTOCOL.md
ADDED
|
@@ -0,0 +1,83 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Frozen FP16 and INT4 Recall: official WikiText-2 test PPL
|
| 2 |
+
|
| 3 |
+
This is an additional evaluation of the previously published checkpoints. It
|
| 4 |
+
does not train, select, rescale, requantize, or alter either checkpoint.
|
| 5 |
+
|
| 6 |
+
## Checkpoints and inference path
|
| 7 |
+
|
| 8 |
+
`RECALL_NATIVE_PREFILL_WT2_TEST_V1_PINS.json` fixes the original FP16 Recall and
|
| 9 |
+
independent affine group128 INT4 W4A16 Recall adapter files, their training
|
| 10 |
+
bindings, the published reference/training reports, original protocol, and all
|
| 11 |
+
runtime modules used here. Their previous 130-window PPL results used WikiText-2
|
| 12 |
+
**validation**. Both releases used native parallel SSD **prefill**, with internal
|
| 13 |
+
scan precision; they did not round recurrent states after every token. This
|
| 14 |
+
runner imports the original released `evaluation.evaluate_ppl` unchanged and
|
| 15 |
+
calls it explicitly with `execution="prefill", logits_chunk=64`.
|
| 16 |
+
|
| 17 |
+
The FP16 model is the pinned NVIDIA BF16 source cast to FP16 by its original
|
| 18 |
+
loader. The W4 model is decoded from the already published INT4 package by its
|
| 19 |
+
original loader; no new quantization occurs. Both remain frozen. These native
|
| 20 |
+
prefill baselines cannot be substituted with a per-token S16 baseline measured
|
| 21 |
+
by the later StateQuant runtime. This protocol measures each baseline directly.
|
| 22 |
+
|
| 23 |
+
## Population and arithmetic
|
| 24 |
+
|
| 25 |
+
- Dataset: `Salesforce/wikitext`, `wikitext-2-raw-v1`, revision
|
| 26 |
+
`b08601e04326c79dfdd32d625aee71d232d685c3`, official **test** split.
|
| 27 |
+
- Original join: two newlines between dataset rows; original NVIDIA
|
| 28 |
+
SentencePiece, no automatically inserted BOS or EOS. The pinned test token
|
| 29 |
+
stream SHA-256 is
|
| 30 |
+
`5b82bd46e833e77fcfc0af62bafeaac62e70e68cfdf214d375f0b7b132d4b608`.
|
| 31 |
+
- 300,964 token IDs, 300,963 next-token targets, 147 windows. Each full window
|
| 32 |
+
scores 2,048 targets; the final partial window is included. Windows reset
|
| 33 |
+
the native model state and cover each target exactly once.
|
| 34 |
+
- Sum next-token cross entropy in the original 64-position FP32 logits chunks;
|
| 35 |
+
compute `exp(total_nll / total_targets)`. Do not average window PPLs.
|
| 36 |
+
- Run adapter off, then the fixed exported FP16 adapter on. Each arm uses the
|
| 37 |
+
same complete population. Record per-window NLL and token SHA-256. No smoke,
|
| 38 |
+
truncation, or test-dependent selection option exists.
|
| 39 |
+
|
| 40 |
+
## Backend and integrity
|
| 41 |
+
|
| 42 |
+
Retain the original numerical settings: eight CPU threads, matmul TF32 disabled,
|
| 43 |
+
float32 matmul precision `highest`. Require the original recorded PyTorch,
|
| 44 |
+
Mamba, Triton, NumPy, datasets, SentencePiece versions and CUDA runtime. Record
|
| 45 |
+
the remaining backend flags and environment. Do not replace the old parallel
|
| 46 |
+
scan with a later singleton recurrent backend policy.
|
| 47 |
+
|
| 48 |
+
The runner binds its own source, this protocol, and the pins file in every
|
| 49 |
+
report. It verifies the fixed published adapter and its individual tensors.
|
| 50 |
+
It derives an FP16 weight ledger from the verified BF16 source in bounded CPU
|
| 51 |
+
chunks, or from the pinned W4 manifest's decoded hashes. It verifies actual
|
| 52 |
+
loaded FP16 contents before and after both arms, original parameter
|
| 53 |
+
identity/version/gradient status after each arm, and actual loaded adapter
|
| 54 |
+
contents. A fixed first-128-token native hidden-state probe must be bitwise
|
| 55 |
+
identical after adapter removal. All original hooks and forward methods must
|
| 56 |
+
also be restored. Raw little-endian int64 tokens, expected weight ledger, full
|
| 57 |
+
arm reports, and checksums are saved into a fresh output directory.
|
| 58 |
+
|
| 59 |
+
The independent CPU auditor reconstructs every window from the saved raw token
|
| 60 |
+
file and verifies its exact identity, full target coverage, aggregate NLL/PPL,
|
| 61 |
+
off/on pairing, all pinned code and checkpoint provenance, weight/adapter
|
| 62 |
+
ledgers, restoration evidence, and report hashes. It does not rerun inference
|
| 63 |
+
or claim an independent GPU numerical reproduction.
|
| 64 |
+
|
| 65 |
+
## Interpretation
|
| 66 |
+
|
| 67 |
+
Label the new result **official WikiText-2 test PPL, native parallel SSD
|
| 68 |
+
prefill**; keep the prior numbers explicitly labeled validation. The
|
| 69 |
+
checkpoints were frozen before this additional test and are not modified based
|
| 70 |
+
on it. Project test text was used in earlier 2.7B work. Base pretraining
|
| 71 |
+
contamination and cross-split duplicate text are not audited. These results do
|
| 72 |
+
not constitute an untouched generalization test or a new recall score. Existing
|
| 73 |
+
MK scores retain their original confirmation-population and inference labels.
|
| 74 |
+
|
| 75 |
+
## Commands
|
| 76 |
+
|
| 77 |
+
Run `scripts/run_recall_native_prefill_wt2_test_v1.py` from the research repo in
|
| 78 |
+
two separate Python processes, with `--model-kind fp16` or `w4`, the respective
|
| 79 |
+
`--release-root`, original `--source-dir` (also supplies the tokenizer), and a
|
| 80 |
+
fresh `--out-dir`. W4 additionally requires the frozen `--w4-dir`. A read-only
|
| 81 |
+
`--preflight-only` validates frozen files, adapter, and W4 manifest without
|
| 82 |
+
importing PyTorch or initializing CUDA. Run
|
| 83 |
+
`scripts/audit_recall_native_prefill_wt2_test_v1.py --output-dir ...` afterward.
|
docs/WT2_TEST_V1_RESULTS.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Official WikiText-2 test: Mamb2_8B_Recall
|
| 2 |
+
|
| 3 |
+
The previously published checkpoint is unchanged. This additional evaluation
|
| 4 |
+
uses the official **test** split and the release's original **native parallel
|
| 5 |
+
SSD prefill** PPL calculation; state is not rounded after each token.
|
| 6 |
+
|
| 7 |
+
| Fixed model | Official WT2 test PPL |
|
| 8 |
+
| --- | ---: |
|
| 9 |
+
| Source FP16, adapter off | 7.244532175944678 |
|
| 10 |
+
| Source FP16 + published Resurface adapter | **6.962817659651336** |
|
| 11 |
+
|
| 12 |
+
Adapter-relative PPL change: **-3.888650%**.
|
| 13 |
+
All **147 reset windows / 300,963 targets** are scored, including the final
|
| 14 |
+
partial window. Both arms use identical windows, original NVIDIA SentencePiece,
|
| 15 |
+
no automatically added BOS/EOS, zero initial state per window, and 64-position
|
| 16 |
+
FP32-logit scoring chunks. The original loader, weight decoding and PPL function
|
| 17 |
+
are unchanged.
|
| 18 |
+
|
| 19 |
+
## Historical validation and recall
|
| 20 |
+
|
| 21 |
+
| Original published arm | WT2 validation PPL | Synthetic CONFIRM normal MK | Target removed |
|
| 22 |
+
| --- | ---: | ---: | ---: |
|
| 23 |
+
| Source FP16, adapter off | 7.334175947318572 | 147/384 | 0/384 |
|
| 24 |
+
| Source FP16 + Resurface | 7.052063515606311 | 365/384 | 0/384 |
|
| 25 |
+
|
| 26 |
+
The old PPL population is **130 validation windows / 264,764 targets**.
|
| 27 |
+
MK remains the original **384 normal synthetic CONFIRM prompts plus 384
|
| 28 |
+
target-removed controls**, with a previously observed template family.
|
| 29 |
+
It uses native prefill followed by recurrent decoding. This update does
|
| 30 |
+
not rerun MK or relabel CONFIRM as a new MK test set.
|
| 31 |
+
|
| 32 |
+
## Integrity and scope
|
| 33 |
+
|
| 34 |
+
The fixed serialized adapter and original runtime match the published hashes.
|
| 35 |
+
No training or checkpoint selection uses this test. All 507 loaded FP16 base
|
| 36 |
+
tensors were checked against independent source/package hashes before and
|
| 37 |
+
after evaluation; actual loaded adapter tensors match its 224-tensor ledger.
|
| 38 |
+
Removing the adapter restores the native 128-token hidden probe bitwise and
|
| 39 |
+
restores native methods/hooks. An independent CPU evidence audit checks
|
| 40 |
+
saved-token identities, full coverage, NLL/PPL arithmetic, paired populations,
|
| 41 |
+
frozen bytes and restoration receipts; it does not rerun GPU inference.
|
| 42 |
+
|
| 43 |
+
Project test text was seen in earlier 2.7B experiments. Base pretraining
|
| 44 |
+
contamination and cross-split duplicate text are not audited. This additional
|
| 45 |
+
official test evaluation is not an untouched generalization claim.
|
| 46 |
+
|
| 47 |
+
Dataset revision: b08601e04326c79dfdd32d625aee71d232d685c3.
|
| 48 |
+
Test token stream SHA-256:
|
| 49 |
+
5b82bd46e833e77fcfc0af62bafeaac62e70e68cfdf214d375f0b7b132d4b608.
|
| 50 |
+
Adapter SHA-256: e8b2b4dfe69f8e85dc9e147c9aeaa3297cff14f4043e558bb1795ad476c1fca0.
|
| 51 |
+
Original HF checkpoint revision: 5b7b27af9cd49d2a55bd080383f1d873c181ce83.
|
| 52 |
+
|
| 53 |
+
Evidence: [paired comparison](../reports/wt2_test_v1/comparison.json),
|
| 54 |
+
[adapter-off windows](../reports/wt2_test_v1/without_resurface.json),
|
| 55 |
+
[adapter-on windows](../reports/wt2_test_v1/resurface.json),
|
| 56 |
+
and [CPU audit](../reports/wt2_test_v1/cpu_audit_v1.json).
|
| 57 |
+
See the [frozen test protocol](RECALL_NATIVE_PREFILL_WT2_TEST_V1_PROTOCOL.md)
|
| 58 |
+
and [checkpoint/code pins](RECALL_NATIVE_PREFILL_WT2_TEST_V1_PINS.json).
|
| 59 |
+
|
| 60 |
+
## Reproduce official test PPL
|
| 61 |
+
|
| 62 |
+
Use the recorded original CUDA environment. A GitHub checkout already has the
|
| 63 |
+
original runtime layout. For a downloaded Hugging Face bundle, assemble the
|
| 64 |
+
small frozen runtime first; no packed weights or source checkpoint are copied.
|
| 65 |
+
|
| 66 |
+
```bash
|
| 67 |
+
python scripts/prepare_recall_wt2_test_inputs_v1.py runtime \
|
| 68 |
+
--model-kind fp16 --download-root . --out-dir test_runtime
|
| 69 |
+
python scripts/run_recall_native_prefill_wt2_test_v1.py \
|
| 70 |
+
--model-kind fp16 --release-root test_runtime \
|
| 71 |
+
--source-dir models/source --out-dir wt2_test_replay
|
| 72 |
+
python scripts/audit_recall_native_prefill_wt2_test_v1.py \
|
| 73 |
+
--output-dir wt2_test_replay
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
FP16 requires the separately downloaded, pinned NVIDIA source checkpoint
|
| 77 |
+
and tokenizer at models/source, as documented in the original release.
|
| 78 |
+
|
| 79 |
+
For a GitHub checkout, use --release-root . and existing source/package
|
| 80 |
+
paths instead of test_runtime. Each GPU replay requires a fresh output directory.
|
| 81 |
+
|
| 82 |
+
Raw token IDs are not redistributed in this docs overlay. To independently
|
| 83 |
+
audit the recorded reports, copy them to a fresh directory and omit only the
|
| 84 |
+
previous CPU audit, then regenerate exact test tokens on CPU:
|
| 85 |
+
|
| 86 |
+
```bash
|
| 87 |
+
cp -R reports/wt2_test_v1 wt2_test_saved_audit
|
| 88 |
+
rm wt2_test_saved_audit/cpu_audit_v1.json
|
| 89 |
+
CUDA_VISIBLE_DEVICES= python scripts/prepare_recall_wt2_test_inputs_v1.py tokens \
|
| 90 |
+
--model-kind fp16 --release-root test_runtime \
|
| 91 |
+
--tokenizer-source-dir models/source --out-dir wt2_test_saved_audit
|
| 92 |
+
python scripts/audit_recall_native_prefill_wt2_test_v1.py \
|
| 93 |
+
--output-dir wt2_test_saved_audit
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
Original checkpoint/release-tag files retain their identities. Existing
|
| 97 |
+
release checksum lists describe the original tagged bundle; the additional
|
| 98 |
+
small-file hashes are in [overlay_checksums.json](../reports/wt2_test_v1/overlay_checksums.json).
|
reports/wt2_test_v1/comparison.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
reports/wt2_test_v1/cpu_audit_v1.json
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "RECALL_NATIVE_PREFILL_WT2_TEST_V1_CPU_AUDIT",
|
| 3 |
+
"complete": true,
|
| 4 |
+
"passed": true,
|
| 5 |
+
"comparison_sha256": "a9395dfb4e1f6f359c3eefc68fae679f6dbce38e42f0c3c1c9f7d8327b6aedee",
|
| 6 |
+
"pins_sha256": "7e85a47cad0733ac386205dd91438f9df0b64c3c92706529c8109f5e428f40aa",
|
| 7 |
+
"protocol_sha256": "4333b79286ee3698d426a760cfbf0824fbe42230d5eb67e4ebdeb437d63ac0d8",
|
| 8 |
+
"runner_sha256": "45de81f37c691e585d4769235a79f7a22bdf0b5a0123ab88b769c5b04fd4672e",
|
| 9 |
+
"auditor_sha256": "7fcba3b3789cb1660e67a0f7111c8432b1da71c6ad4ba47a8b65b7415fac63cd",
|
| 10 |
+
"model_kind": "fp16",
|
| 11 |
+
"split": "test",
|
| 12 |
+
"ppl_execution": "prefill",
|
| 13 |
+
"windows": 147,
|
| 14 |
+
"targets": 300963,
|
| 15 |
+
"ppl": {
|
| 16 |
+
"without_resurface": 7.244532175944678,
|
| 17 |
+
"resurface": 6.962817659651336
|
| 18 |
+
},
|
| 19 |
+
"cuda_initialized": false,
|
| 20 |
+
"torch_imported": false,
|
| 21 |
+
"scope": "Independent saved-token/hash/NLL arithmetic and frozen-checkpoint evidence audit; no GPU inference replay."
|
| 22 |
+
}
|
reports/wt2_test_v1/overlay_checksums.json
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "RECALL_WT2_TEST_DOCS_OVERLAY_CHECKSUMS_V1",
|
| 3 |
+
"model_kind": "fp16",
|
| 4 |
+
"github_base_main_commit": "ac4584a2f0e771b2ab4373f12765308321fca9a3",
|
| 5 |
+
"hf_base_main_commit": "5b7b27af9cd49d2a55bd080383f1d873c181ce83",
|
| 6 |
+
"scope": "Additional small docs/evaluation files only; original checkpoints/tags unchanged",
|
| 7 |
+
"files_excluding_this_checksum_file": {
|
| 8 |
+
"README.md": {
|
| 9 |
+
"bytes": 11058,
|
| 10 |
+
"sha256": "a79887e4226111a670007566bd7318f1cdbf04886351e38ec5ba73d315bf074b"
|
| 11 |
+
},
|
| 12 |
+
"docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PINS.json": {
|
| 13 |
+
"bytes": 50110,
|
| 14 |
+
"sha256": "7e85a47cad0733ac386205dd91438f9df0b64c3c92706529c8109f5e428f40aa"
|
| 15 |
+
},
|
| 16 |
+
"docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PROTOCOL.md": {
|
| 17 |
+
"bytes": 4831,
|
| 18 |
+
"sha256": "4333b79286ee3698d426a760cfbf0824fbe42230d5eb67e4ebdeb437d63ac0d8"
|
| 19 |
+
},
|
| 20 |
+
"docs/WT2_TEST_V1_RESULTS.md": {
|
| 21 |
+
"bytes": 4851,
|
| 22 |
+
"sha256": "0aa3b9eec95550d37dc2637289297cd60f76c32dad5f867f222a4d479751bd71"
|
| 23 |
+
},
|
| 24 |
+
"reports/wt2_test_v1/comparison.json": {
|
| 25 |
+
"bytes": 162961,
|
| 26 |
+
"sha256": "a9395dfb4e1f6f359c3eefc68fae679f6dbce38e42f0c3c1c9f7d8327b6aedee"
|
| 27 |
+
},
|
| 28 |
+
"reports/wt2_test_v1/cpu_audit_v1.json": {
|
| 29 |
+
"bytes": 924,
|
| 30 |
+
"sha256": "e854d0bbe66ab7331e00cb574fc8dd478fe0abe8992b685c8e45caec9b8f89b1"
|
| 31 |
+
},
|
| 32 |
+
"reports/wt2_test_v1/resurface.json": {
|
| 33 |
+
"bytes": 62521,
|
| 34 |
+
"sha256": "c56cfa329a23b474accbe834b91b6f0b9ea056fd8937273a5b7268a6dfbd72b0"
|
| 35 |
+
},
|
| 36 |
+
"reports/wt2_test_v1/weight_ledger.json": {
|
| 37 |
+
"bytes": 55929,
|
| 38 |
+
"sha256": "b08e24e05a8a6af5db3fc910a613714a1c578228a213c691ab4685d6b01ca272"
|
| 39 |
+
},
|
| 40 |
+
"reports/wt2_test_v1/without_resurface.json": {
|
| 41 |
+
"bytes": 41353,
|
| 42 |
+
"sha256": "fc48c74624779f0bbd167cc7626088611095320d396e03813815f8f0a26dc8c1"
|
| 43 |
+
},
|
| 44 |
+
"scripts/audit_recall_native_prefill_wt2_test_v1.py": {
|
| 45 |
+
"bytes": 9784,
|
| 46 |
+
"sha256": "7fcba3b3789cb1660e67a0f7111c8432b1da71c6ad4ba47a8b65b7415fac63cd"
|
| 47 |
+
},
|
| 48 |
+
"scripts/prepare_recall_wt2_test_inputs_v1.py": {
|
| 49 |
+
"bytes": 4821,
|
| 50 |
+
"sha256": "e0599313917f54247a648a3ed1e825395e1abcc7f692380e69b528355fcd9e9f"
|
| 51 |
+
},
|
| 52 |
+
"scripts/run_recall_native_prefill_wt2_test_v1.py": {
|
| 53 |
+
"bytes": 15984,
|
| 54 |
+
"sha256": "45de81f37c691e585d4769235a79f7a22bdf0b5a0123ab88b769c5b04fd4672e"
|
| 55 |
+
}
|
| 56 |
+
}
|
| 57 |
+
}
|
reports/wt2_test_v1/resurface.json
ADDED
|
@@ -0,0 +1,1464 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "RECALL_NATIVE_PREFILL_WT2_TEST_V1",
|
| 3 |
+
"arm": "resurface",
|
| 4 |
+
"model_kind": "fp16",
|
| 5 |
+
"complete": true,
|
| 6 |
+
"adapter_loaded": true,
|
| 7 |
+
"adapter_sha256": "e8b2b4dfe69f8e85dc9e147c9aeaa3297cff14f4043e558bb1795ad476c1fca0",
|
| 8 |
+
"dataset": {
|
| 9 |
+
"dataset": "Salesforce/wikitext",
|
| 10 |
+
"configuration": "wikitext-2-raw-v1",
|
| 11 |
+
"split": "test",
|
| 12 |
+
"revision_argument": "b08601e04326c79dfdd32d625aee71d232d685c3",
|
| 13 |
+
"dataset_fingerprint": "a46124b21ac53738",
|
| 14 |
+
"document_join": "two newline characters",
|
| 15 |
+
"text_sha256": "696cca6b65a171b0a358a4be6732cdfdf2dd6164a32e20fd70e3c13fc4dfae83",
|
| 16 |
+
"token_stream_sha256_int64le": "5b82bd46e833e77fcfc0af62bafeaac62e70e68cfdf214d375f0b7b132d4b608",
|
| 17 |
+
"total_tokens": 300964,
|
| 18 |
+
"tokenizer_sha256": "5862e2f71caf762bc9845662be5fec2867deb58d874568235a02a36c5111cd09",
|
| 19 |
+
"automatic_special_tokens": false
|
| 20 |
+
},
|
| 21 |
+
"ppl": {
|
| 22 |
+
"ppl": 6.962817659651336,
|
| 23 |
+
"nll": 584044.0511858463,
|
| 24 |
+
"target_tokens": 300963,
|
| 25 |
+
"execution": "prefill",
|
| 26 |
+
"windows": [
|
| 27 |
+
{
|
| 28 |
+
"start": 0,
|
| 29 |
+
"input_tokens": 2048,
|
| 30 |
+
"target_tokens": 2048,
|
| 31 |
+
"token_sha256_int64le": "1a2d6b883799adf5ffc7f16d48b0460668675e47f0f434ff5a718f9dc4b4df46",
|
| 32 |
+
"nll": 3436.5770874023438,
|
| 33 |
+
"ppl": 5.354922096339743
|
| 34 |
+
},
|
| 35 |
+
{
|
| 36 |
+
"start": 2048,
|
| 37 |
+
"input_tokens": 2048,
|
| 38 |
+
"target_tokens": 2048,
|
| 39 |
+
"token_sha256_int64le": "7697abe4489bc66e2c85a387f701d5fed7488ad33bf58c10222d23c1183126a4",
|
| 40 |
+
"nll": 4518.347229003906,
|
| 41 |
+
"ppl": 9.08136246812367
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"start": 4096,
|
| 45 |
+
"input_tokens": 2048,
|
| 46 |
+
"target_tokens": 2048,
|
| 47 |
+
"token_sha256_int64le": "7e16a843f8aaf1e2785a41a80563825a702e36684f88bea80334ea1d24d103ca",
|
| 48 |
+
"nll": 4911.522567749023,
|
| 49 |
+
"ppl": 11.00340069139861
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"start": 6144,
|
| 53 |
+
"input_tokens": 2048,
|
| 54 |
+
"target_tokens": 2048,
|
| 55 |
+
"token_sha256_int64le": "1a1e313f7e4842994100c1c05d95a74161f38ce648ee8492cb965cc90d1e42e0",
|
| 56 |
+
"nll": 4093.687734603882,
|
| 57 |
+
"ppl": 7.380718297510351
|
| 58 |
+
},
|
| 59 |
+
{
|
| 60 |
+
"start": 8192,
|
| 61 |
+
"input_tokens": 2048,
|
| 62 |
+
"target_tokens": 2048,
|
| 63 |
+
"token_sha256_int64le": "36e892d798f0575cdf0d2c213de6e9741cbd1b6b2f9559300637bddcbd1054bb",
|
| 64 |
+
"nll": 3381.547866821289,
|
| 65 |
+
"ppl": 5.212952632605377
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"start": 10240,
|
| 69 |
+
"input_tokens": 2048,
|
| 70 |
+
"target_tokens": 2048,
|
| 71 |
+
"token_sha256_int64le": "861b7f6f1b60b293c5c29869f740bca014bac01991b7ea49194c4d849fbfc56c",
|
| 72 |
+
"nll": 3414.210075378418,
|
| 73 |
+
"ppl": 5.2967570894191525
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"start": 12288,
|
| 77 |
+
"input_tokens": 2048,
|
| 78 |
+
"target_tokens": 2048,
|
| 79 |
+
"token_sha256_int64le": "0fa99db0d520d1f840220596695f453b79857e1d58bcfa1621b961029ab5627b",
|
| 80 |
+
"nll": 2673.495931625366,
|
| 81 |
+
"ppl": 3.6892306314976633
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"start": 14336,
|
| 85 |
+
"input_tokens": 2048,
|
| 86 |
+
"target_tokens": 2048,
|
| 87 |
+
"token_sha256_int64le": "81e4e56a99d407898ed96c05d2761d1d8a259687d425a947a136a5aad43db69a",
|
| 88 |
+
"nll": 3593.6505584716797,
|
| 89 |
+
"ppl": 5.781783428604847
|
| 90 |
+
},
|
| 91 |
+
{
|
| 92 |
+
"start": 16384,
|
| 93 |
+
"input_tokens": 2048,
|
| 94 |
+
"target_tokens": 2048,
|
| 95 |
+
"token_sha256_int64le": "529d7b8692a2930dfdf62f3433cd902ce6efc941a682650be224f7681b04d55d",
|
| 96 |
+
"nll": 3924.4447708129883,
|
| 97 |
+
"ppl": 6.795310882747134
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"start": 18432,
|
| 101 |
+
"input_tokens": 2048,
|
| 102 |
+
"target_tokens": 2048,
|
| 103 |
+
"token_sha256_int64le": "e8d680b5ad89bbe9fc06e27c8e266813e3384e309d24c635bd8097b75957bd0c",
|
| 104 |
+
"nll": 4170.69059753418,
|
| 105 |
+
"ppl": 7.663509348482596
|
| 106 |
+
},
|
| 107 |
+
{
|
| 108 |
+
"start": 20480,
|
| 109 |
+
"input_tokens": 2048,
|
| 110 |
+
"target_tokens": 2048,
|
| 111 |
+
"token_sha256_int64le": "b55804a097552f93e7a3ada5d3e5ca88b193da7602e08f60043164f7d276f26a",
|
| 112 |
+
"nll": 3835.713447570801,
|
| 113 |
+
"ppl": 6.507185037692459
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"start": 22528,
|
| 117 |
+
"input_tokens": 2048,
|
| 118 |
+
"target_tokens": 2048,
|
| 119 |
+
"token_sha256_int64le": "79ac58154608ca6c4faa60a26006cb25a52ffee8ebed64c9b3a1aadab5966456",
|
| 120 |
+
"nll": 3951.262897491455,
|
| 121 |
+
"ppl": 6.88487920018988
|
| 122 |
+
},
|
| 123 |
+
{
|
| 124 |
+
"start": 24576,
|
| 125 |
+
"input_tokens": 2048,
|
| 126 |
+
"target_tokens": 2048,
|
| 127 |
+
"token_sha256_int64le": "5a4bd71e5725cff95a16854b008ca69d89f687a1c72829b37d9db72cf2ca092c",
|
| 128 |
+
"nll": 3988.5370330810547,
|
| 129 |
+
"ppl": 7.011333058169227
|
| 130 |
+
},
|
| 131 |
+
{
|
| 132 |
+
"start": 26624,
|
| 133 |
+
"input_tokens": 2048,
|
| 134 |
+
"target_tokens": 2048,
|
| 135 |
+
"token_sha256_int64le": "0dff6eb969a20082d5454585184896f13c5f82644d509724add1fb00df67d640",
|
| 136 |
+
"nll": 4371.0771484375,
|
| 137 |
+
"ppl": 8.451255531030105
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"start": 28672,
|
| 141 |
+
"input_tokens": 2048,
|
| 142 |
+
"target_tokens": 2048,
|
| 143 |
+
"token_sha256_int64le": "05cf104fcf3f4da284adc1a9c0f75188010b8d99da8a339d6ad6bf67ea63179a",
|
| 144 |
+
"nll": 4334.110809326172,
|
| 145 |
+
"ppl": 8.300079089530877
|
| 146 |
+
},
|
| 147 |
+
{
|
| 148 |
+
"start": 30720,
|
| 149 |
+
"input_tokens": 2048,
|
| 150 |
+
"target_tokens": 2048,
|
| 151 |
+
"token_sha256_int64le": "5bcb87c98edf5bd5e2cc1dc8b9cb6035e1039da343a4c1e0d5bd2315bd301d2f",
|
| 152 |
+
"nll": 4452.337730407715,
|
| 153 |
+
"ppl": 8.793326068324635
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"start": 32768,
|
| 157 |
+
"input_tokens": 2048,
|
| 158 |
+
"target_tokens": 2048,
|
| 159 |
+
"token_sha256_int64le": "e5572a47b0f3190b6d5fe26f1851a8cebe56c94568f8d0bff7507bc82b3ee874",
|
| 160 |
+
"nll": 4804.991928100586,
|
| 161 |
+
"ppl": 10.44566923138044
|
| 162 |
+
},
|
| 163 |
+
{
|
| 164 |
+
"start": 34816,
|
| 165 |
+
"input_tokens": 2048,
|
| 166 |
+
"target_tokens": 2048,
|
| 167 |
+
"token_sha256_int64le": "9594c2d6d8c598ef18b85af375fd588e8ed0d894378c4bfb38203b0e795c36f5",
|
| 168 |
+
"nll": 4270.321510314941,
|
| 169 |
+
"ppl": 8.04554019575149
|
| 170 |
+
},
|
| 171 |
+
{
|
| 172 |
+
"start": 36864,
|
| 173 |
+
"input_tokens": 2048,
|
| 174 |
+
"target_tokens": 2048,
|
| 175 |
+
"token_sha256_int64le": "523e0fc4c878920a442d2a0dd7056700c83983dc91a3d324e9487e2897267a22",
|
| 176 |
+
"nll": 4077.611198425293,
|
| 177 |
+
"ppl": 7.323007417486329
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"start": 38912,
|
| 181 |
+
"input_tokens": 2048,
|
| 182 |
+
"target_tokens": 2048,
|
| 183 |
+
"token_sha256_int64le": "79fb5ecc92a0428cf4226e62ff4cacfd4e9c10ac87d9459be06606cee84ecd56",
|
| 184 |
+
"nll": 3348.6508226394653,
|
| 185 |
+
"ppl": 5.129885860356234
|
| 186 |
+
},
|
| 187 |
+
{
|
| 188 |
+
"start": 40960,
|
| 189 |
+
"input_tokens": 2048,
|
| 190 |
+
"target_tokens": 2048,
|
| 191 |
+
"token_sha256_int64le": "9f1b10da5f0e0a8f3038e97313456ca3654c01ebcdb7c2b9af430f567522fe12",
|
| 192 |
+
"nll": 3869.634231567383,
|
| 193 |
+
"ppl": 6.615860284199247
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"start": 43008,
|
| 197 |
+
"input_tokens": 2048,
|
| 198 |
+
"target_tokens": 2048,
|
| 199 |
+
"token_sha256_int64le": "c298e4f30124d7dd5b2acd1af1c20fd10d5c760f914bccf6596222b9af6a4817",
|
| 200 |
+
"nll": 4630.743003845215,
|
| 201 |
+
"ppl": 9.593684163680884
|
| 202 |
+
},
|
| 203 |
+
{
|
| 204 |
+
"start": 45056,
|
| 205 |
+
"input_tokens": 2048,
|
| 206 |
+
"target_tokens": 2048,
|
| 207 |
+
"token_sha256_int64le": "1ee4f672f4aa05e0bca489aa34e611d5c3672fd45d35ecaa80650172f517e1f3",
|
| 208 |
+
"nll": 4188.924774169922,
|
| 209 |
+
"ppl": 7.732045337036358
|
| 210 |
+
},
|
| 211 |
+
{
|
| 212 |
+
"start": 47104,
|
| 213 |
+
"input_tokens": 2048,
|
| 214 |
+
"target_tokens": 2048,
|
| 215 |
+
"token_sha256_int64le": "a32fa420db87e3b71bbd34feabf530fa3f10459413f5d4f5dc362971ae6c713c",
|
| 216 |
+
"nll": 4309.43196105957,
|
| 217 |
+
"ppl": 8.200661524502856
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"start": 49152,
|
| 221 |
+
"input_tokens": 2048,
|
| 222 |
+
"target_tokens": 2048,
|
| 223 |
+
"token_sha256_int64le": "3a080801aee2913550da2ef76116d30b8544f72f3e415efeb32a28c853f885b0",
|
| 224 |
+
"nll": 4301.506385803223,
|
| 225 |
+
"ppl": 8.168987032402843
|
| 226 |
+
},
|
| 227 |
+
{
|
| 228 |
+
"start": 51200,
|
| 229 |
+
"input_tokens": 2048,
|
| 230 |
+
"target_tokens": 2048,
|
| 231 |
+
"token_sha256_int64le": "9a31a733e7fd3359f73439a67d44ebb8b226ade7db57b6d8dfdc365e38ab8578",
|
| 232 |
+
"nll": 4594.691581726074,
|
| 233 |
+
"ppl": 9.42628203379792
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"start": 53248,
|
| 237 |
+
"input_tokens": 2048,
|
| 238 |
+
"target_tokens": 2048,
|
| 239 |
+
"token_sha256_int64le": "00ab7d803922a2919b32b0a911ea2c785a0c10b1daf6a8a41822ee3fd4dbbbfe",
|
| 240 |
+
"nll": 4121.912635803223,
|
| 241 |
+
"ppl": 7.483141228759246
|
| 242 |
+
},
|
| 243 |
+
{
|
| 244 |
+
"start": 55296,
|
| 245 |
+
"input_tokens": 2048,
|
| 246 |
+
"target_tokens": 2048,
|
| 247 |
+
"token_sha256_int64le": "ea0361a9b10122b903c6c0c566223c34ba6777438ad9760a4fd2780df416bb4c",
|
| 248 |
+
"nll": 5181.478782653809,
|
| 249 |
+
"ppl": 12.553743862689906
|
| 250 |
+
},
|
| 251 |
+
{
|
| 252 |
+
"start": 57344,
|
| 253 |
+
"input_tokens": 2048,
|
| 254 |
+
"target_tokens": 2048,
|
| 255 |
+
"token_sha256_int64le": "273931d43814c4fc37e89a18cdf28ad55c4b4a7838eb730bf1c50524dd63b07b",
|
| 256 |
+
"nll": 4731.182247161865,
|
| 257 |
+
"ppl": 10.075911564662505
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"start": 59392,
|
| 261 |
+
"input_tokens": 2048,
|
| 262 |
+
"target_tokens": 2048,
|
| 263 |
+
"token_sha256_int64le": "504bf8092c24e386bb14d7fd516f3b9b87de76cbed8d5b04e77b5b648959e721",
|
| 264 |
+
"nll": 4103.32975769043,
|
| 265 |
+
"ppl": 7.415548787086481
|
| 266 |
+
},
|
| 267 |
+
{
|
| 268 |
+
"start": 61440,
|
| 269 |
+
"input_tokens": 2048,
|
| 270 |
+
"target_tokens": 2048,
|
| 271 |
+
"token_sha256_int64le": "5ae8981c50c7d44bdf981826af279d76b58672a60b39ab9f42208d5ca4e05f58",
|
| 272 |
+
"nll": 4029.481819152832,
|
| 273 |
+
"ppl": 7.152918244409464
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"start": 63488,
|
| 277 |
+
"input_tokens": 2048,
|
| 278 |
+
"target_tokens": 2048,
|
| 279 |
+
"token_sha256_int64le": "d83e6672b7bc41afde8ee4f57a55a33ded70664969f6b10f23fcccc465cb318a",
|
| 280 |
+
"nll": 3733.418888092041,
|
| 281 |
+
"ppl": 6.190144567619678
|
| 282 |
+
},
|
| 283 |
+
{
|
| 284 |
+
"start": 65536,
|
| 285 |
+
"input_tokens": 2048,
|
| 286 |
+
"target_tokens": 2048,
|
| 287 |
+
"token_sha256_int64le": "dce022be768611664ee67ddfdae6abc4af0dadfd9643d691f67ab6bcc1673795",
|
| 288 |
+
"nll": 3167.1987075805664,
|
| 289 |
+
"ppl": 4.694932539323592
|
| 290 |
+
},
|
| 291 |
+
{
|
| 292 |
+
"start": 67584,
|
| 293 |
+
"input_tokens": 2048,
|
| 294 |
+
"target_tokens": 2048,
|
| 295 |
+
"token_sha256_int64le": "3a1c84c593afe2049a4686370c7f15fc16c23acb44d0e6d80117025044d099cc",
|
| 296 |
+
"nll": 3876.8755416870117,
|
| 297 |
+
"ppl": 6.639294020284099
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"start": 69632,
|
| 301 |
+
"input_tokens": 2048,
|
| 302 |
+
"target_tokens": 2048,
|
| 303 |
+
"token_sha256_int64le": "d7c822f494cbef0ca81ba2c79bc8038895e9c80c4fd293a97f0a02ef6295c082",
|
| 304 |
+
"nll": 3769.160972595215,
|
| 305 |
+
"ppl": 6.299124346701176
|
| 306 |
+
},
|
| 307 |
+
{
|
| 308 |
+
"start": 71680,
|
| 309 |
+
"input_tokens": 2048,
|
| 310 |
+
"target_tokens": 2048,
|
| 311 |
+
"token_sha256_int64le": "39af145da34bb3e1d399335c388ca897f7b8015b915dffd9126d06a66ca9ab69",
|
| 312 |
+
"nll": 4117.780836105347,
|
| 313 |
+
"ppl": 7.4680593573941465
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"start": 73728,
|
| 317 |
+
"input_tokens": 2048,
|
| 318 |
+
"target_tokens": 2048,
|
| 319 |
+
"token_sha256_int64le": "f92c643e9edc615dbea357215ad6d0dd830b12807354551edae764294e8c1f1c",
|
| 320 |
+
"nll": 3566.740131378174,
|
| 321 |
+
"ppl": 5.706308571609322
|
| 322 |
+
},
|
| 323 |
+
{
|
| 324 |
+
"start": 75776,
|
| 325 |
+
"input_tokens": 2048,
|
| 326 |
+
"target_tokens": 2048,
|
| 327 |
+
"token_sha256_int64le": "c888968354fe6914e5290adaf9ea6fe52f04ca26a15fcb1eed68a0bd391e3830",
|
| 328 |
+
"nll": 2360.515697479248,
|
| 329 |
+
"ppl": 3.1664008218190403
|
| 330 |
+
},
|
| 331 |
+
{
|
| 332 |
+
"start": 77824,
|
| 333 |
+
"input_tokens": 2048,
|
| 334 |
+
"target_tokens": 2048,
|
| 335 |
+
"token_sha256_int64le": "bdea806e668159c6cb35589ad033df517b7f254734d9f51944f13c97c9d3ae68",
|
| 336 |
+
"nll": 2763.0769538879395,
|
| 337 |
+
"ppl": 3.854181523429257
|
| 338 |
+
},
|
| 339 |
+
{
|
| 340 |
+
"start": 79872,
|
| 341 |
+
"input_tokens": 2048,
|
| 342 |
+
"target_tokens": 2048,
|
| 343 |
+
"token_sha256_int64le": "4b07638d19b5c9ad746568806beb439065d4cb78383216c80f9a3e4d27d463e5",
|
| 344 |
+
"nll": 2764.469207763672,
|
| 345 |
+
"ppl": 3.856802531002998
|
| 346 |
+
},
|
| 347 |
+
{
|
| 348 |
+
"start": 81920,
|
| 349 |
+
"input_tokens": 2048,
|
| 350 |
+
"target_tokens": 2048,
|
| 351 |
+
"token_sha256_int64le": "63d59d1b1701e47518b37c6dbe96328f74b55cfa0c39fa777572550316c4e30b",
|
| 352 |
+
"nll": 2769.534324645996,
|
| 353 |
+
"ppl": 3.866352986436678
|
| 354 |
+
},
|
| 355 |
+
{
|
| 356 |
+
"start": 83968,
|
| 357 |
+
"input_tokens": 2048,
|
| 358 |
+
"target_tokens": 2048,
|
| 359 |
+
"token_sha256_int64le": "728708cfa01a3dace51ee3914cd1b7df8e8965b3be89c074ec1d526e26ffe9ca",
|
| 360 |
+
"nll": 3169.716278076172,
|
| 361 |
+
"ppl": 4.700707486378601
|
| 362 |
+
},
|
| 363 |
+
{
|
| 364 |
+
"start": 86016,
|
| 365 |
+
"input_tokens": 2048,
|
| 366 |
+
"target_tokens": 2048,
|
| 367 |
+
"token_sha256_int64le": "415a1604f02c5b8d7be0dcf28cacd543e9d7cae35f1a096ac2bd9bc558397e80",
|
| 368 |
+
"nll": 4252.620632171631,
|
| 369 |
+
"ppl": 7.976302179618764
|
| 370 |
+
},
|
| 371 |
+
{
|
| 372 |
+
"start": 88064,
|
| 373 |
+
"input_tokens": 2048,
|
| 374 |
+
"target_tokens": 2048,
|
| 375 |
+
"token_sha256_int64le": "9f1a7dfd05a3fe0f23f3240717e27ce520d3bb2ef23d9563ba3c1d0c1c886fc6",
|
| 376 |
+
"nll": 5121.513290405273,
|
| 377 |
+
"ppl": 12.19149907010618
|
| 378 |
+
},
|
| 379 |
+
{
|
| 380 |
+
"start": 90112,
|
| 381 |
+
"input_tokens": 2048,
|
| 382 |
+
"target_tokens": 2048,
|
| 383 |
+
"token_sha256_int64le": "c3e4dd63053d31e9664902b623588bc69bc8db1825922f50a5317d6883611927",
|
| 384 |
+
"nll": 5523.192771911621,
|
| 385 |
+
"ppl": 14.833252790864314
|
| 386 |
+
},
|
| 387 |
+
{
|
| 388 |
+
"start": 92160,
|
| 389 |
+
"input_tokens": 2048,
|
| 390 |
+
"target_tokens": 2048,
|
| 391 |
+
"token_sha256_int64le": "176f55c4882d5853d613dce487101474627fd410c11a8c65b0fab479657da2f1",
|
| 392 |
+
"nll": 4627.729904174805,
|
| 393 |
+
"ppl": 9.579579928989075
|
| 394 |
+
},
|
| 395 |
+
{
|
| 396 |
+
"start": 94208,
|
| 397 |
+
"input_tokens": 2048,
|
| 398 |
+
"target_tokens": 2048,
|
| 399 |
+
"token_sha256_int64le": "5f0e51c563b7f36ec336f5e58d9da80d5afbf4a3ddfad34e24f0c9a7fb34aa3f",
|
| 400 |
+
"nll": 4224.858726501465,
|
| 401 |
+
"ppl": 7.868908015836135
|
| 402 |
+
},
|
| 403 |
+
{
|
| 404 |
+
"start": 96256,
|
| 405 |
+
"input_tokens": 2048,
|
| 406 |
+
"target_tokens": 2048,
|
| 407 |
+
"token_sha256_int64le": "cc5d6b982df6a3e63856f9558eadd8ff4ed07cf87028d7dd5dd35c07088f404e",
|
| 408 |
+
"nll": 4517.466156005859,
|
| 409 |
+
"ppl": 9.077456402514153
|
| 410 |
+
},
|
| 411 |
+
{
|
| 412 |
+
"start": 98304,
|
| 413 |
+
"input_tokens": 2048,
|
| 414 |
+
"target_tokens": 2048,
|
| 415 |
+
"token_sha256_int64le": "d95820340c62af23bd4070605867072e382dd287fad90f51a4212ede697981a2",
|
| 416 |
+
"nll": 4705.376243591309,
|
| 417 |
+
"ppl": 9.949745708816646
|
| 418 |
+
},
|
| 419 |
+
{
|
| 420 |
+
"start": 100352,
|
| 421 |
+
"input_tokens": 2048,
|
| 422 |
+
"target_tokens": 2048,
|
| 423 |
+
"token_sha256_int64le": "7ca14adfb3be5680cd954b14482135c6578e679f2c4dfca98ea175a5ab31423b",
|
| 424 |
+
"nll": 4315.199562072754,
|
| 425 |
+
"ppl": 8.22378887170437
|
| 426 |
+
},
|
| 427 |
+
{
|
| 428 |
+
"start": 102400,
|
| 429 |
+
"input_tokens": 2048,
|
| 430 |
+
"target_tokens": 2048,
|
| 431 |
+
"token_sha256_int64le": "3e21ad94cfbf0d1252f4ac9f5379feb0bb59325c6de00fda03d4ed492f49d4a1",
|
| 432 |
+
"nll": 3541.7329177856445,
|
| 433 |
+
"ppl": 5.637055057767182
|
| 434 |
+
},
|
| 435 |
+
{
|
| 436 |
+
"start": 104448,
|
| 437 |
+
"input_tokens": 2048,
|
| 438 |
+
"target_tokens": 2048,
|
| 439 |
+
"token_sha256_int64le": "fb345a3fee185cb82cfd3740826a0e8b1e5e0f97b2625e9a2e863c4e9a1c6665",
|
| 440 |
+
"nll": 4112.834621429443,
|
| 441 |
+
"ppl": 7.450044682279153
|
| 442 |
+
},
|
| 443 |
+
{
|
| 444 |
+
"start": 106496,
|
| 445 |
+
"input_tokens": 2048,
|
| 446 |
+
"target_tokens": 2048,
|
| 447 |
+
"token_sha256_int64le": "93d9382d0ac84a8987e86a0aee19a0897ba27521df2cf6a24e84759d473db170",
|
| 448 |
+
"nll": 3352.083694458008,
|
| 449 |
+
"ppl": 5.13849182132112
|
| 450 |
+
},
|
| 451 |
+
{
|
| 452 |
+
"start": 108544,
|
| 453 |
+
"input_tokens": 2048,
|
| 454 |
+
"target_tokens": 2048,
|
| 455 |
+
"token_sha256_int64le": "1d47fee2ba8b7a2c694cd0963df65056d2bbeb7e71f1aeb6038178384b831e14",
|
| 456 |
+
"nll": 3507.2476234436035,
|
| 457 |
+
"ppl": 5.542930065501617
|
| 458 |
+
},
|
| 459 |
+
{
|
| 460 |
+
"start": 110592,
|
| 461 |
+
"input_tokens": 2048,
|
| 462 |
+
"target_tokens": 2048,
|
| 463 |
+
"token_sha256_int64le": "31c43559c09f97b1d8b8ee46d021d266d93c3bd48661da1d0a8edf8b95aeeeba",
|
| 464 |
+
"nll": 3412.931209564209,
|
| 465 |
+
"ppl": 5.293450582066528
|
| 466 |
+
},
|
| 467 |
+
{
|
| 468 |
+
"start": 112640,
|
| 469 |
+
"input_tokens": 2048,
|
| 470 |
+
"target_tokens": 2048,
|
| 471 |
+
"token_sha256_int64le": "80967f476ea0f604ed9e5e4876994ee60e6a9520f700aa4cd4d06c025ab8555e",
|
| 472 |
+
"nll": 3226.7764205932617,
|
| 473 |
+
"ppl": 4.833517309738383
|
| 474 |
+
},
|
| 475 |
+
{
|
| 476 |
+
"start": 114688,
|
| 477 |
+
"input_tokens": 2048,
|
| 478 |
+
"target_tokens": 2048,
|
| 479 |
+
"token_sha256_int64le": "526722d86264dd0327b8134a217c3dbc0cfeada6cf152f86990e45619cd211a4",
|
| 480 |
+
"nll": 3407.232204437256,
|
| 481 |
+
"ppl": 5.278740881267415
|
| 482 |
+
},
|
| 483 |
+
{
|
| 484 |
+
"start": 116736,
|
| 485 |
+
"input_tokens": 2048,
|
| 486 |
+
"target_tokens": 2048,
|
| 487 |
+
"token_sha256_int64le": "f408a1b708dab5847c6d5199f41519e0b1350cc9c110762285620a836ecc1489",
|
| 488 |
+
"nll": 2958.5515518188477,
|
| 489 |
+
"ppl": 4.240178001767255
|
| 490 |
+
},
|
| 491 |
+
{
|
| 492 |
+
"start": 118784,
|
| 493 |
+
"input_tokens": 2048,
|
| 494 |
+
"target_tokens": 2048,
|
| 495 |
+
"token_sha256_int64le": "6052f506f3ab2f7b602857548485e766b10b516b92aaa3c0ad2218eaeca681c2",
|
| 496 |
+
"nll": 3445.6585693359375,
|
| 497 |
+
"ppl": 5.37872024557898
|
| 498 |
+
},
|
| 499 |
+
{
|
| 500 |
+
"start": 120832,
|
| 501 |
+
"input_tokens": 2048,
|
| 502 |
+
"target_tokens": 2048,
|
| 503 |
+
"token_sha256_int64le": "d4d38109cafdc61e186b9115a258de568de94086ad37b46b4bc112630f79337d",
|
| 504 |
+
"nll": 3215.7379293441772,
|
| 505 |
+
"ppl": 4.807535274351984
|
| 506 |
+
},
|
| 507 |
+
{
|
| 508 |
+
"start": 122880,
|
| 509 |
+
"input_tokens": 2048,
|
| 510 |
+
"target_tokens": 2048,
|
| 511 |
+
"token_sha256_int64le": "b4ab40620ac2cfe91e2b35e849c7e440aa3aceea004a2908c56a2cc9c831e2e6",
|
| 512 |
+
"nll": 4262.220703125,
|
| 513 |
+
"ppl": 8.013779141080578
|
| 514 |
+
},
|
| 515 |
+
{
|
| 516 |
+
"start": 124928,
|
| 517 |
+
"input_tokens": 2048,
|
| 518 |
+
"target_tokens": 2048,
|
| 519 |
+
"token_sha256_int64le": "8fdaa7359f3f50d044792b3753b413fa458cf415b9d6a9ffe43c024a3ebeda8b",
|
| 520 |
+
"nll": 4444.885856628418,
|
| 521 |
+
"ppl": 8.761388721031052
|
| 522 |
+
},
|
| 523 |
+
{
|
| 524 |
+
"start": 126976,
|
| 525 |
+
"input_tokens": 2048,
|
| 526 |
+
"target_tokens": 2048,
|
| 527 |
+
"token_sha256_int64le": "c252332b02aa8e79b37bd54c086c701397bf47a88463840dfe39dbcfe96717dc",
|
| 528 |
+
"nll": 4615.690567016602,
|
| 529 |
+
"ppl": 9.523430777545176
|
| 530 |
+
},
|
| 531 |
+
{
|
| 532 |
+
"start": 129024,
|
| 533 |
+
"input_tokens": 2048,
|
| 534 |
+
"target_tokens": 2048,
|
| 535 |
+
"token_sha256_int64le": "a25db88b3e46febc7fe2b128cad4112e43c7c0a85fe4357cf3b76fe2c0bedb18",
|
| 536 |
+
"nll": 4954.501243591309,
|
| 537 |
+
"ppl": 11.236754535153043
|
| 538 |
+
},
|
| 539 |
+
{
|
| 540 |
+
"start": 131072,
|
| 541 |
+
"input_tokens": 2048,
|
| 542 |
+
"target_tokens": 2048,
|
| 543 |
+
"token_sha256_int64le": "53fbaab0079ba77dae982f6d960b73bb0a15c32599c1b5b9b1fc5ebae01e4508",
|
| 544 |
+
"nll": 4219.251708984375,
|
| 545 |
+
"ppl": 7.847393971523713
|
| 546 |
+
},
|
| 547 |
+
{
|
| 548 |
+
"start": 133120,
|
| 549 |
+
"input_tokens": 2048,
|
| 550 |
+
"target_tokens": 2048,
|
| 551 |
+
"token_sha256_int64le": "5bdbdf09ceef09d6aa431db8277b89196813a7a38af6c6273ecb7f8e14d6a706",
|
| 552 |
+
"nll": 2785.5019569396973,
|
| 553 |
+
"ppl": 3.8966155840242793
|
| 554 |
+
},
|
| 555 |
+
{
|
| 556 |
+
"start": 135168,
|
| 557 |
+
"input_tokens": 2048,
|
| 558 |
+
"target_tokens": 2048,
|
| 559 |
+
"token_sha256_int64le": "c234d0ef8f6ce3509e1113c4bf0597f4be1c2a93cc7a62dffd29db050897b3df",
|
| 560 |
+
"nll": 4184.33708190918,
|
| 561 |
+
"ppl": 7.714744290260678
|
| 562 |
+
},
|
| 563 |
+
{
|
| 564 |
+
"start": 137216,
|
| 565 |
+
"input_tokens": 2048,
|
| 566 |
+
"target_tokens": 2048,
|
| 567 |
+
"token_sha256_int64le": "b464cbc80b68bae974a72bd7611c7fcd3b51915fc071646cd3aee2c819c4ca89",
|
| 568 |
+
"nll": 3443.5902366638184,
|
| 569 |
+
"ppl": 5.373290867164608
|
| 570 |
+
},
|
| 571 |
+
{
|
| 572 |
+
"start": 139264,
|
| 573 |
+
"input_tokens": 2048,
|
| 574 |
+
"target_tokens": 2048,
|
| 575 |
+
"token_sha256_int64le": "57e524a37ce26addc3be29f5865763cd6c6e43362c9093b7031f92c9a5447444",
|
| 576 |
+
"nll": 3492.1497688293457,
|
| 577 |
+
"ppl": 5.502217838444414
|
| 578 |
+
},
|
| 579 |
+
{
|
| 580 |
+
"start": 141312,
|
| 581 |
+
"input_tokens": 2048,
|
| 582 |
+
"target_tokens": 2048,
|
| 583 |
+
"token_sha256_int64le": "456dd4d2463d87007d5c12aaff4b37037dff22e884aece61620063b2b8d47eb3",
|
| 584 |
+
"nll": 3438.922782897949,
|
| 585 |
+
"ppl": 5.361058919008387
|
| 586 |
+
},
|
| 587 |
+
{
|
| 588 |
+
"start": 143360,
|
| 589 |
+
"input_tokens": 2048,
|
| 590 |
+
"target_tokens": 2048,
|
| 591 |
+
"token_sha256_int64le": "e605eedd4934713a8d1cca99200965930b02475c2ea7526ea5dd7c83c73e0c2d",
|
| 592 |
+
"nll": 2914.1078300476074,
|
| 593 |
+
"ppl": 4.149152986987717
|
| 594 |
+
},
|
| 595 |
+
{
|
| 596 |
+
"start": 145408,
|
| 597 |
+
"input_tokens": 2048,
|
| 598 |
+
"target_tokens": 2048,
|
| 599 |
+
"token_sha256_int64le": "7d67022273bc70fb56deb220e15ba87a371679cfe5bfe6c69ec2c3053eb06285",
|
| 600 |
+
"nll": 3486.5251960754395,
|
| 601 |
+
"ppl": 5.487127425089475
|
| 602 |
+
},
|
| 603 |
+
{
|
| 604 |
+
"start": 147456,
|
| 605 |
+
"input_tokens": 2048,
|
| 606 |
+
"target_tokens": 2048,
|
| 607 |
+
"token_sha256_int64le": "326f43859059e3233ce0cfb7e01af3b9e5890edacad013a48b79926520aecfd6",
|
| 608 |
+
"nll": 3999.656265258789,
|
| 609 |
+
"ppl": 7.049503302322231
|
| 610 |
+
},
|
| 611 |
+
{
|
| 612 |
+
"start": 149504,
|
| 613 |
+
"input_tokens": 2048,
|
| 614 |
+
"target_tokens": 2048,
|
| 615 |
+
"token_sha256_int64le": "1f1dcb8d4ed2394e11ee7a01ab57f51f99795aff8d15fd18bb9be12229b715f2",
|
| 616 |
+
"nll": 4501.510971069336,
|
| 617 |
+
"ppl": 9.007012168833096
|
| 618 |
+
},
|
| 619 |
+
{
|
| 620 |
+
"start": 151552,
|
| 621 |
+
"input_tokens": 2048,
|
| 622 |
+
"target_tokens": 2048,
|
| 623 |
+
"token_sha256_int64le": "efcdcd786b861a83480e50dc169dbe42cbdd0ce5703be0200852c9a5a35d45dc",
|
| 624 |
+
"nll": 4040.3324432373047,
|
| 625 |
+
"ppl": 7.190916093593992
|
| 626 |
+
},
|
| 627 |
+
{
|
| 628 |
+
"start": 153600,
|
| 629 |
+
"input_tokens": 2048,
|
| 630 |
+
"target_tokens": 2048,
|
| 631 |
+
"token_sha256_int64le": "0435f9685afc422dd9b3361859e2a5b8c5aeff77430443ab114adf8eac40ec6a",
|
| 632 |
+
"nll": 3626.8726196289062,
|
| 633 |
+
"ppl": 5.876338690598954
|
| 634 |
+
},
|
| 635 |
+
{
|
| 636 |
+
"start": 155648,
|
| 637 |
+
"input_tokens": 2048,
|
| 638 |
+
"target_tokens": 2048,
|
| 639 |
+
"token_sha256_int64le": "7812055c909040c52b02e4d86ce7b0008243027c6db97e16b343c8677bfa869c",
|
| 640 |
+
"nll": 3994.145835876465,
|
| 641 |
+
"ppl": 7.030561126071425
|
| 642 |
+
},
|
| 643 |
+
{
|
| 644 |
+
"start": 157696,
|
| 645 |
+
"input_tokens": 2048,
|
| 646 |
+
"target_tokens": 2048,
|
| 647 |
+
"token_sha256_int64le": "81df4320e9b14dc3b1510df717f280150752a0f66d5be4cc3e7699044375f231",
|
| 648 |
+
"nll": 3960.796142578125,
|
| 649 |
+
"ppl": 6.91700236474824
|
| 650 |
+
},
|
| 651 |
+
{
|
| 652 |
+
"start": 159744,
|
| 653 |
+
"input_tokens": 2048,
|
| 654 |
+
"target_tokens": 2048,
|
| 655 |
+
"token_sha256_int64le": "61756936a0aab088ae5e8596ac465e24f73203487adf4c0c78fff1491d1a027b",
|
| 656 |
+
"nll": 3951.164619445801,
|
| 657 |
+
"ppl": 6.884548821167551
|
| 658 |
+
},
|
| 659 |
+
{
|
| 660 |
+
"start": 161792,
|
| 661 |
+
"input_tokens": 2048,
|
| 662 |
+
"target_tokens": 2048,
|
| 663 |
+
"token_sha256_int64le": "17ec9a2e4d188258766d53057c8db2aa19a95c56224b72c427ef9bbd9ecab588",
|
| 664 |
+
"nll": 4200.183437347412,
|
| 665 |
+
"ppl": 7.774668488469466
|
| 666 |
+
},
|
| 667 |
+
{
|
| 668 |
+
"start": 163840,
|
| 669 |
+
"input_tokens": 2048,
|
| 670 |
+
"target_tokens": 2048,
|
| 671 |
+
"token_sha256_int64le": "2ae8b819a850b1b8400abc174d036cd916bc71a4f85301a45c325a28af686590",
|
| 672 |
+
"nll": 3958.5277252197266,
|
| 673 |
+
"ppl": 6.9093451568589
|
| 674 |
+
},
|
| 675 |
+
{
|
| 676 |
+
"start": 165888,
|
| 677 |
+
"input_tokens": 2048,
|
| 678 |
+
"target_tokens": 2048,
|
| 679 |
+
"token_sha256_int64le": "075d9bf13dd08dca31814252bf3dfae2cd0105b5c292104fdeffe4e31aa3aa06",
|
| 680 |
+
"nll": 4910.51725769043,
|
| 681 |
+
"ppl": 10.998000732973086
|
| 682 |
+
},
|
| 683 |
+
{
|
| 684 |
+
"start": 167936,
|
| 685 |
+
"input_tokens": 2048,
|
| 686 |
+
"target_tokens": 2048,
|
| 687 |
+
"token_sha256_int64le": "033950cf80d4d6474d828d07d37130bf6195302e6edfd28dad24cb9d8c96350d",
|
| 688 |
+
"nll": 3539.569793701172,
|
| 689 |
+
"ppl": 5.631104270519938
|
| 690 |
+
},
|
| 691 |
+
{
|
| 692 |
+
"start": 169984,
|
| 693 |
+
"input_tokens": 2048,
|
| 694 |
+
"target_tokens": 2048,
|
| 695 |
+
"token_sha256_int64le": "994ac15313bb5f36f32f68aa4539d765a0e97c57cdf66085322b084221a5f627",
|
| 696 |
+
"nll": 3709.8994102478027,
|
| 697 |
+
"ppl": 6.119462839876241
|
| 698 |
+
},
|
| 699 |
+
{
|
| 700 |
+
"start": 172032,
|
| 701 |
+
"input_tokens": 2048,
|
| 702 |
+
"target_tokens": 2048,
|
| 703 |
+
"token_sha256_int64le": "b7cf37e2d3ae4533b02c832a041399d763ff8280f4baa7d21b74870aa97f0acf",
|
| 704 |
+
"nll": 3935.524314880371,
|
| 705 |
+
"ppl": 6.832172683639364
|
| 706 |
+
},
|
| 707 |
+
{
|
| 708 |
+
"start": 174080,
|
| 709 |
+
"input_tokens": 2048,
|
| 710 |
+
"target_tokens": 2048,
|
| 711 |
+
"token_sha256_int64le": "82ca3f6ee68359b0d695bc304b36b96eced80e978228697fbe648e91bea4c191",
|
| 712 |
+
"nll": 3368.2025604248047,
|
| 713 |
+
"ppl": 5.179094098186987
|
| 714 |
+
},
|
| 715 |
+
{
|
| 716 |
+
"start": 176128,
|
| 717 |
+
"input_tokens": 2048,
|
| 718 |
+
"target_tokens": 2048,
|
| 719 |
+
"token_sha256_int64le": "77095ab5a4ceffb0d157e5ca439d1e77fb930046e06f494f66ed1ec652dfe2f0",
|
| 720 |
+
"nll": 3568.766975402832,
|
| 721 |
+
"ppl": 5.711958729070598
|
| 722 |
+
},
|
| 723 |
+
{
|
| 724 |
+
"start": 178176,
|
| 725 |
+
"input_tokens": 2048,
|
| 726 |
+
"target_tokens": 2048,
|
| 727 |
+
"token_sha256_int64le": "860303bd273211d412d6ae9b6601861e3a02e88dca9d49e59424550e52b6da25",
|
| 728 |
+
"nll": 3057.6400833129883,
|
| 729 |
+
"ppl": 4.450374820381037
|
| 730 |
+
},
|
| 731 |
+
{
|
| 732 |
+
"start": 180224,
|
| 733 |
+
"input_tokens": 2048,
|
| 734 |
+
"target_tokens": 2048,
|
| 735 |
+
"token_sha256_int64le": "ff2154ce3f53bef7bfaf3bbc5e80c89db57da2ca9db5452df04e24ae798327e0",
|
| 736 |
+
"nll": 3416.5098876953125,
|
| 737 |
+
"ppl": 5.302708451441056
|
| 738 |
+
},
|
| 739 |
+
{
|
| 740 |
+
"start": 182272,
|
| 741 |
+
"input_tokens": 2048,
|
| 742 |
+
"target_tokens": 2048,
|
| 743 |
+
"token_sha256_int64le": "205014bcf6efc0ae1b54c5fde5bbbe6eef5cca032624f9209a0fbfb63f12d0b4",
|
| 744 |
+
"nll": 4170.503269195557,
|
| 745 |
+
"ppl": 7.662808407652404
|
| 746 |
+
},
|
| 747 |
+
{
|
| 748 |
+
"start": 184320,
|
| 749 |
+
"input_tokens": 2048,
|
| 750 |
+
"target_tokens": 2048,
|
| 751 |
+
"token_sha256_int64le": "5c93807a4025356a713b59385ae3b9026d317d5895ea297ecb81ece4bea5a24b",
|
| 752 |
+
"nll": 4136.596008300781,
|
| 753 |
+
"ppl": 7.536985263798499
|
| 754 |
+
},
|
| 755 |
+
{
|
| 756 |
+
"start": 186368,
|
| 757 |
+
"input_tokens": 2048,
|
| 758 |
+
"target_tokens": 2048,
|
| 759 |
+
"token_sha256_int64le": "a540fbca609950b4f89ee7aacf16647200ac4662aed4a435d34fa5a591506767",
|
| 760 |
+
"nll": 4804.407333374023,
|
| 761 |
+
"ppl": 10.442687975670752
|
| 762 |
+
},
|
| 763 |
+
{
|
| 764 |
+
"start": 188416,
|
| 765 |
+
"input_tokens": 2048,
|
| 766 |
+
"target_tokens": 2048,
|
| 767 |
+
"token_sha256_int64le": "def9d1107a3f9217e45385d23ad448d07071e08afc3f8fdc4d215719dda5e676",
|
| 768 |
+
"nll": 5642.802658081055,
|
| 769 |
+
"ppl": 15.725360622487925
|
| 770 |
+
},
|
| 771 |
+
{
|
| 772 |
+
"start": 190464,
|
| 773 |
+
"input_tokens": 2048,
|
| 774 |
+
"target_tokens": 2048,
|
| 775 |
+
"token_sha256_int64le": "f85f70be3d7c0aaf1f1672276af223b53c8babb4665b6669d9c50500e26ce06c",
|
| 776 |
+
"nll": 5265.274139404297,
|
| 777 |
+
"ppl": 13.078041970533178
|
| 778 |
+
},
|
| 779 |
+
{
|
| 780 |
+
"start": 192512,
|
| 781 |
+
"input_tokens": 2048,
|
| 782 |
+
"target_tokens": 2048,
|
| 783 |
+
"token_sha256_int64le": "9f149a2ac78574d25c8884f3eb8f885354612ca1a0a7c52d3c2e4a9c87681d6d",
|
| 784 |
+
"nll": 4850.5742111206055,
|
| 785 |
+
"ppl": 10.680764774031873
|
| 786 |
+
},
|
| 787 |
+
{
|
| 788 |
+
"start": 194560,
|
| 789 |
+
"input_tokens": 2048,
|
| 790 |
+
"target_tokens": 2048,
|
| 791 |
+
"token_sha256_int64le": "31f093666f3973d82cd9f91b24eeee8d8f85ea118a69c23466b8fee222e8b095",
|
| 792 |
+
"nll": 5061.233200073242,
|
| 793 |
+
"ppl": 11.837888453475129
|
| 794 |
+
},
|
| 795 |
+
{
|
| 796 |
+
"start": 196608,
|
| 797 |
+
"input_tokens": 2048,
|
| 798 |
+
"target_tokens": 2048,
|
| 799 |
+
"token_sha256_int64le": "30cf2b9c3370fee885bec22db9331bd4102cf9692790988fae9b6fae6546e633",
|
| 800 |
+
"nll": 5161.986022949219,
|
| 801 |
+
"ppl": 12.434824798460985
|
| 802 |
+
},
|
| 803 |
+
{
|
| 804 |
+
"start": 198656,
|
| 805 |
+
"input_tokens": 2048,
|
| 806 |
+
"target_tokens": 2048,
|
| 807 |
+
"token_sha256_int64le": "b2137f528db02ad66f5baba2608cd170f1ea8da93d055f5cd5bd663cd1cbf38f",
|
| 808 |
+
"nll": 4237.099308013916,
|
| 809 |
+
"ppl": 7.916080100436991
|
| 810 |
+
},
|
| 811 |
+
{
|
| 812 |
+
"start": 200704,
|
| 813 |
+
"input_tokens": 2048,
|
| 814 |
+
"target_tokens": 2048,
|
| 815 |
+
"token_sha256_int64le": "8617a5ef9748481a079ebace75ffecf5d73dbc71fc0f31277ed397a8b2117214",
|
| 816 |
+
"nll": 4282.845260620117,
|
| 817 |
+
"ppl": 8.094890315780486
|
| 818 |
+
},
|
| 819 |
+
{
|
| 820 |
+
"start": 202752,
|
| 821 |
+
"input_tokens": 2048,
|
| 822 |
+
"target_tokens": 2048,
|
| 823 |
+
"token_sha256_int64le": "119e062f2d3c08a3edffd75b84f2ae1bdfbf8df8921b4bbaaedb4f0d8d49e622",
|
| 824 |
+
"nll": 4257.966270446777,
|
| 825 |
+
"ppl": 7.997148918656326
|
| 826 |
+
},
|
| 827 |
+
{
|
| 828 |
+
"start": 204800,
|
| 829 |
+
"input_tokens": 2048,
|
| 830 |
+
"target_tokens": 2048,
|
| 831 |
+
"token_sha256_int64le": "219c6bee49173e63ab46fba96fb2eebf64b936c66625821dd0ab3887e0c3fe84",
|
| 832 |
+
"nll": 4408.205581665039,
|
| 833 |
+
"ppl": 8.605866584419072
|
| 834 |
+
},
|
| 835 |
+
{
|
| 836 |
+
"start": 206848,
|
| 837 |
+
"input_tokens": 2048,
|
| 838 |
+
"target_tokens": 2048,
|
| 839 |
+
"token_sha256_int64le": "8fd9219793088ab936c3a2a3591cdb99b0494706f761f7a0a8477f9a9517e948",
|
| 840 |
+
"nll": 3325.141345977783,
|
| 841 |
+
"ppl": 5.071335386887594
|
| 842 |
+
},
|
| 843 |
+
{
|
| 844 |
+
"start": 208896,
|
| 845 |
+
"input_tokens": 2048,
|
| 846 |
+
"target_tokens": 2048,
|
| 847 |
+
"token_sha256_int64le": "956e5ccaf62e8a667c920d83a905e3a5a5c5370f4c4a392c88db1883330f36a0",
|
| 848 |
+
"nll": 4421.665672302246,
|
| 849 |
+
"ppl": 8.662613280580711
|
| 850 |
+
},
|
| 851 |
+
{
|
| 852 |
+
"start": 210944,
|
| 853 |
+
"input_tokens": 2048,
|
| 854 |
+
"target_tokens": 2048,
|
| 855 |
+
"token_sha256_int64le": "e0c9b5ceaf6f8f8ca27d14e0cba95c7069cf85bb6e82a275fa825fa2af9874c9",
|
| 856 |
+
"nll": 4517.765586853027,
|
| 857 |
+
"ppl": 9.078783682382396
|
| 858 |
+
},
|
| 859 |
+
{
|
| 860 |
+
"start": 212992,
|
| 861 |
+
"input_tokens": 2048,
|
| 862 |
+
"target_tokens": 2048,
|
| 863 |
+
"token_sha256_int64le": "65db3f6e716e7d8b73ca309b59c4d3b6fdef7dae19c3d55afe6a2444b42c574f",
|
| 864 |
+
"nll": 4098.926452636719,
|
| 865 |
+
"ppl": 7.3996221045356645
|
| 866 |
+
},
|
| 867 |
+
{
|
| 868 |
+
"start": 215040,
|
| 869 |
+
"input_tokens": 2048,
|
| 870 |
+
"target_tokens": 2048,
|
| 871 |
+
"token_sha256_int64le": "a549a37af9e6d60435c3a8b1120e8626444ae5a2a19af4d07a065cb5b507dd5e",
|
| 872 |
+
"nll": 4152.651466369629,
|
| 873 |
+
"ppl": 7.596304270431158
|
| 874 |
+
},
|
| 875 |
+
{
|
| 876 |
+
"start": 217088,
|
| 877 |
+
"input_tokens": 2048,
|
| 878 |
+
"target_tokens": 2048,
|
| 879 |
+
"token_sha256_int64le": "24c630974ce5a2b74bc1e865952e1996cacc28c84402c1d82e1a434acf2a407d",
|
| 880 |
+
"nll": 3942.975700378418,
|
| 881 |
+
"ppl": 6.857075944425822
|
| 882 |
+
},
|
| 883 |
+
{
|
| 884 |
+
"start": 219136,
|
| 885 |
+
"input_tokens": 2048,
|
| 886 |
+
"target_tokens": 2048,
|
| 887 |
+
"token_sha256_int64le": "c6b4a33d822aa0b404b9c270ff6fa15feff495147a42ecf2e2c75eaab2c706a0",
|
| 888 |
+
"nll": 4927.236038208008,
|
| 889 |
+
"ppl": 11.088150014266931
|
| 890 |
+
},
|
| 891 |
+
{
|
| 892 |
+
"start": 221184,
|
| 893 |
+
"input_tokens": 2048,
|
| 894 |
+
"target_tokens": 2048,
|
| 895 |
+
"token_sha256_int64le": "16a348f3f3a2f91aa52792912ad664b3584734cb7d6a7059db39273ac357d81f",
|
| 896 |
+
"nll": 3763.4387016296387,
|
| 897 |
+
"ppl": 6.281548669676192
|
| 898 |
+
},
|
| 899 |
+
{
|
| 900 |
+
"start": 223232,
|
| 901 |
+
"input_tokens": 2048,
|
| 902 |
+
"target_tokens": 2048,
|
| 903 |
+
"token_sha256_int64le": "cce3c564bb6bb43e222444ca912ed084f09a60a2b9ff37b827ce2aa6bbde6621",
|
| 904 |
+
"nll": 3879.4870376586914,
|
| 905 |
+
"ppl": 6.647765479673234
|
| 906 |
+
},
|
| 907 |
+
{
|
| 908 |
+
"start": 225280,
|
| 909 |
+
"input_tokens": 2048,
|
| 910 |
+
"target_tokens": 2048,
|
| 911 |
+
"token_sha256_int64le": "6a65cef2c5163831329dd01c22f4b581fa1e3f65a6aa344150b7351bb88bbf2d",
|
| 912 |
+
"nll": 3575.55952835083,
|
| 913 |
+
"ppl": 5.730934898404471
|
| 914 |
+
},
|
| 915 |
+
{
|
| 916 |
+
"start": 227328,
|
| 917 |
+
"input_tokens": 2048,
|
| 918 |
+
"target_tokens": 2048,
|
| 919 |
+
"token_sha256_int64le": "16a0dee22ed67df3c33d86b8342cfd4d2c3355ca6115d2178fb8a27c73e2961c",
|
| 920 |
+
"nll": 3465.8662490844727,
|
| 921 |
+
"ppl": 5.432054940500246
|
| 922 |
+
},
|
| 923 |
+
{
|
| 924 |
+
"start": 229376,
|
| 925 |
+
"input_tokens": 2048,
|
| 926 |
+
"target_tokens": 2048,
|
| 927 |
+
"token_sha256_int64le": "91de99f25597d4261eed3a7a201ca425d91bcd92ae90e59d8f499f1b6f8513f6",
|
| 928 |
+
"nll": 3549.0789680480957,
|
| 929 |
+
"ppl": 5.657311135105566
|
| 930 |
+
},
|
| 931 |
+
{
|
| 932 |
+
"start": 231424,
|
| 933 |
+
"input_tokens": 2048,
|
| 934 |
+
"target_tokens": 2048,
|
| 935 |
+
"token_sha256_int64le": "475eaef6ca5dca8059ced50fb2c9f16729a4881dc494f6b8e58868f2cff583e1",
|
| 936 |
+
"nll": 3572.8295669555664,
|
| 937 |
+
"ppl": 5.72330071471162
|
| 938 |
+
},
|
| 939 |
+
{
|
| 940 |
+
"start": 233472,
|
| 941 |
+
"input_tokens": 2048,
|
| 942 |
+
"target_tokens": 2048,
|
| 943 |
+
"token_sha256_int64le": "23eca1cf7560b0090dcb8a656167c8a9841d6ff24637b21b6e86663854e819d9",
|
| 944 |
+
"nll": 4233.49747467041,
|
| 945 |
+
"ppl": 7.902170264764181
|
| 946 |
+
},
|
| 947 |
+
{
|
| 948 |
+
"start": 235520,
|
| 949 |
+
"input_tokens": 2048,
|
| 950 |
+
"target_tokens": 2048,
|
| 951 |
+
"token_sha256_int64le": "98e388fba1b866cdd04fee868ce0ce6ef963302fec97146a792636baabbd7ba8",
|
| 952 |
+
"nll": 4235.980239868164,
|
| 953 |
+
"ppl": 7.911755777559415
|
| 954 |
+
},
|
| 955 |
+
{
|
| 956 |
+
"start": 237568,
|
| 957 |
+
"input_tokens": 2048,
|
| 958 |
+
"target_tokens": 2048,
|
| 959 |
+
"token_sha256_int64le": "789d2a3c1b8f7dee0dba62f3fa7ebfabe30f9c472b505aafee1043632b617593",
|
| 960 |
+
"nll": 4317.790840148926,
|
| 961 |
+
"ppl": 8.234200791162795
|
| 962 |
+
},
|
| 963 |
+
{
|
| 964 |
+
"start": 239616,
|
| 965 |
+
"input_tokens": 2048,
|
| 966 |
+
"target_tokens": 2048,
|
| 967 |
+
"token_sha256_int64le": "e0952a12d1d8d0ad826249adba609c357d311c02052dfd73bcc4118a5a2321e6",
|
| 968 |
+
"nll": 4510.517280578613,
|
| 969 |
+
"ppl": 9.046708735292576
|
| 970 |
+
},
|
| 971 |
+
{
|
| 972 |
+
"start": 241664,
|
| 973 |
+
"input_tokens": 2048,
|
| 974 |
+
"target_tokens": 2048,
|
| 975 |
+
"token_sha256_int64le": "7e4db6466013f514b43fe7187a5506493340098c9d6e9cb19e2de44c1b25aaa5",
|
| 976 |
+
"nll": 3460.4056968688965,
|
| 977 |
+
"ppl": 5.417590823772169
|
| 978 |
+
},
|
| 979 |
+
{
|
| 980 |
+
"start": 243712,
|
| 981 |
+
"input_tokens": 2048,
|
| 982 |
+
"target_tokens": 2048,
|
| 983 |
+
"token_sha256_int64le": "cabbd779acfeab157a23dccccacdc8eab0e4e2462d78450747ad971242481f12",
|
| 984 |
+
"nll": 3790.805072784424,
|
| 985 |
+
"ppl": 6.366049085777286
|
| 986 |
+
},
|
| 987 |
+
{
|
| 988 |
+
"start": 245760,
|
| 989 |
+
"input_tokens": 2048,
|
| 990 |
+
"target_tokens": 2048,
|
| 991 |
+
"token_sha256_int64le": "94cc1da9c6faa265aae847ab2608dc502d9af48b4745cfe3507f980c5353e3a3",
|
| 992 |
+
"nll": 4137.716957092285,
|
| 993 |
+
"ppl": 7.541111673495298
|
| 994 |
+
},
|
| 995 |
+
{
|
| 996 |
+
"start": 247808,
|
| 997 |
+
"input_tokens": 2048,
|
| 998 |
+
"target_tokens": 2048,
|
| 999 |
+
"token_sha256_int64le": "d7e70afddb18ab1e802433e1e1f5fc210cb01f95d54efcfa7aef331e7b289ee2",
|
| 1000 |
+
"nll": 4032.8946075439453,
|
| 1001 |
+
"ppl": 7.164847808487741
|
| 1002 |
+
},
|
| 1003 |
+
{
|
| 1004 |
+
"start": 249856,
|
| 1005 |
+
"input_tokens": 2048,
|
| 1006 |
+
"target_tokens": 2048,
|
| 1007 |
+
"token_sha256_int64le": "aa828523335b939e8d0dbfc231fabd2e7e491c08ed1c7a0eef1562c6a4f1c2d4",
|
| 1008 |
+
"nll": 3544.955924987793,
|
| 1009 |
+
"ppl": 5.645933267026774
|
| 1010 |
+
},
|
| 1011 |
+
{
|
| 1012 |
+
"start": 251904,
|
| 1013 |
+
"input_tokens": 2048,
|
| 1014 |
+
"target_tokens": 2048,
|
| 1015 |
+
"token_sha256_int64le": "160c9f18c8aa5fa6e564a28a5d392bcc036fefdb935ed9f1b1df7bc8d3fb78d4",
|
| 1016 |
+
"nll": 3554.1356887817383,
|
| 1017 |
+
"ppl": 5.671296871894878
|
| 1018 |
+
},
|
| 1019 |
+
{
|
| 1020 |
+
"start": 253952,
|
| 1021 |
+
"input_tokens": 2048,
|
| 1022 |
+
"target_tokens": 2048,
|
| 1023 |
+
"token_sha256_int64le": "b73ede3b386dc168dde9de45a9b65cd026d9bdce7abaf99a17642d6d8f3a88e0",
|
| 1024 |
+
"nll": 3538.511547088623,
|
| 1025 |
+
"ppl": 5.628195306806005
|
| 1026 |
+
},
|
| 1027 |
+
{
|
| 1028 |
+
"start": 256000,
|
| 1029 |
+
"input_tokens": 2048,
|
| 1030 |
+
"target_tokens": 2048,
|
| 1031 |
+
"token_sha256_int64le": "24d9589ab416fd91e908b5226dbfa7c06409f330a1c7a4bd9a1e021646601822",
|
| 1032 |
+
"nll": 3758.7328872680664,
|
| 1033 |
+
"ppl": 6.267131742303105
|
| 1034 |
+
},
|
| 1035 |
+
{
|
| 1036 |
+
"start": 258048,
|
| 1037 |
+
"input_tokens": 2048,
|
| 1038 |
+
"target_tokens": 2048,
|
| 1039 |
+
"token_sha256_int64le": "9ab8aec5d86c9ee1a0a29b35cb7dde4e8a927f0760cf843a0a1a1cc592f11ddc",
|
| 1040 |
+
"nll": 4029.023956298828,
|
| 1041 |
+
"ppl": 7.151319274930051
|
| 1042 |
+
},
|
| 1043 |
+
{
|
| 1044 |
+
"start": 260096,
|
| 1045 |
+
"input_tokens": 2048,
|
| 1046 |
+
"target_tokens": 2048,
|
| 1047 |
+
"token_sha256_int64le": "5c921b66917fdb0f1fc83be2aedab3c41b8f1283368d67596617f415ca0f04f8",
|
| 1048 |
+
"nll": 3634.195713043213,
|
| 1049 |
+
"ppl": 5.89738849825821
|
| 1050 |
+
},
|
| 1051 |
+
{
|
| 1052 |
+
"start": 262144,
|
| 1053 |
+
"input_tokens": 2048,
|
| 1054 |
+
"target_tokens": 2048,
|
| 1055 |
+
"token_sha256_int64le": "205de0d3f69598cc0383e171a574a31198e7723fbad128f324710843fe6ff569",
|
| 1056 |
+
"nll": 3814.212146759033,
|
| 1057 |
+
"ppl": 6.4392255378360455
|
| 1058 |
+
},
|
| 1059 |
+
{
|
| 1060 |
+
"start": 264192,
|
| 1061 |
+
"input_tokens": 2048,
|
| 1062 |
+
"target_tokens": 2048,
|
| 1063 |
+
"token_sha256_int64le": "384ceca0ff3a4e3408e82053069067cc17dfe752f9b5daece51e04cafab4bb3b",
|
| 1064 |
+
"nll": 3524.2154083251953,
|
| 1065 |
+
"ppl": 5.5890442905342095
|
| 1066 |
+
},
|
| 1067 |
+
{
|
| 1068 |
+
"start": 266240,
|
| 1069 |
+
"input_tokens": 2048,
|
| 1070 |
+
"target_tokens": 2048,
|
| 1071 |
+
"token_sha256_int64le": "bce567c974d90531fd5cc7659ca7e07304a70dfae98a8ceecbb42d5663a0c3b7",
|
| 1072 |
+
"nll": 3608.3584213256836,
|
| 1073 |
+
"ppl": 5.82345518748015
|
| 1074 |
+
},
|
| 1075 |
+
{
|
| 1076 |
+
"start": 268288,
|
| 1077 |
+
"input_tokens": 2048,
|
| 1078 |
+
"target_tokens": 2048,
|
| 1079 |
+
"token_sha256_int64le": "887e1cf68ef3a7a19c44fc7df1fcb38786a2a904189ad40d3b21a5eb7d09046d",
|
| 1080 |
+
"nll": 3479.1733055114746,
|
| 1081 |
+
"ppl": 5.467465101129896
|
| 1082 |
+
},
|
| 1083 |
+
{
|
| 1084 |
+
"start": 270336,
|
| 1085 |
+
"input_tokens": 2048,
|
| 1086 |
+
"target_tokens": 2048,
|
| 1087 |
+
"token_sha256_int64le": "278594634f7f30199be67eac18d43ca49b972dc8278bae7a857ff8b5d7026140",
|
| 1088 |
+
"nll": 3888.0074462890625,
|
| 1089 |
+
"ppl": 6.675480160679251
|
| 1090 |
+
},
|
| 1091 |
+
{
|
| 1092 |
+
"start": 272384,
|
| 1093 |
+
"input_tokens": 2048,
|
| 1094 |
+
"target_tokens": 2048,
|
| 1095 |
+
"token_sha256_int64le": "86c7c332b1436eac3225fa316bdd62402521144b36fff138763f645677f79a93",
|
| 1096 |
+
"nll": 3958.936882019043,
|
| 1097 |
+
"ppl": 6.910725668559693
|
| 1098 |
+
},
|
| 1099 |
+
{
|
| 1100 |
+
"start": 274432,
|
| 1101 |
+
"input_tokens": 2048,
|
| 1102 |
+
"target_tokens": 2048,
|
| 1103 |
+
"token_sha256_int64le": "4873332acc6a1221452ccf6ab2bc94099dcccd456f5b6ba321958ea8f9c60cef",
|
| 1104 |
+
"nll": 4162.521408081055,
|
| 1105 |
+
"ppl": 7.633001553111431
|
| 1106 |
+
},
|
| 1107 |
+
{
|
| 1108 |
+
"start": 276480,
|
| 1109 |
+
"input_tokens": 2048,
|
| 1110 |
+
"target_tokens": 2048,
|
| 1111 |
+
"token_sha256_int64le": "017db0f35d2e651d10a7f095493247ef38a8aff669f5e729f854df00f2c3e715",
|
| 1112 |
+
"nll": 4691.923789978027,
|
| 1113 |
+
"ppl": 9.884604177459353
|
| 1114 |
+
},
|
| 1115 |
+
{
|
| 1116 |
+
"start": 278528,
|
| 1117 |
+
"input_tokens": 2048,
|
| 1118 |
+
"target_tokens": 2048,
|
| 1119 |
+
"token_sha256_int64le": "579f6cada791a9b5110c0e1908ed8ae1747c354821a9ea59a1115b467c861a0a",
|
| 1120 |
+
"nll": 3716.9302940368652,
|
| 1121 |
+
"ppl": 6.140507356734123
|
| 1122 |
+
},
|
| 1123 |
+
{
|
| 1124 |
+
"start": 280576,
|
| 1125 |
+
"input_tokens": 2048,
|
| 1126 |
+
"target_tokens": 2048,
|
| 1127 |
+
"token_sha256_int64le": "588bc423b576ea870c073d6e32cf95544b2c5e0ba4710892fe8731279da28a38",
|
| 1128 |
+
"nll": 3333.039680480957,
|
| 1129 |
+
"ppl": 5.090931305384063
|
| 1130 |
+
},
|
| 1131 |
+
{
|
| 1132 |
+
"start": 282624,
|
| 1133 |
+
"input_tokens": 2048,
|
| 1134 |
+
"target_tokens": 2048,
|
| 1135 |
+
"token_sha256_int64le": "b43354340e3257bf28d75c85570e060a9f70db695fd34326d61e4d9b50569044",
|
| 1136 |
+
"nll": 2822.4017107486725,
|
| 1137 |
+
"ppl": 3.967458983832295
|
| 1138 |
+
},
|
| 1139 |
+
{
|
| 1140 |
+
"start": 284672,
|
| 1141 |
+
"input_tokens": 2048,
|
| 1142 |
+
"target_tokens": 2048,
|
| 1143 |
+
"token_sha256_int64le": "3b33a66548afb24214097359a4c10380e0cefcde0292a0773bbfc78d745b11b5",
|
| 1144 |
+
"nll": 3821.451446533203,
|
| 1145 |
+
"ppl": 6.462027280846049
|
| 1146 |
+
},
|
| 1147 |
+
{
|
| 1148 |
+
"start": 286720,
|
| 1149 |
+
"input_tokens": 2048,
|
| 1150 |
+
"target_tokens": 2048,
|
| 1151 |
+
"token_sha256_int64le": "eb6ed0c741a4746f386fea624aae9629739463bb40e476d73df622dd264ea6d7",
|
| 1152 |
+
"nll": 4396.409111022949,
|
| 1153 |
+
"ppl": 8.556439319282417
|
| 1154 |
+
},
|
| 1155 |
+
{
|
| 1156 |
+
"start": 288768,
|
| 1157 |
+
"input_tokens": 2048,
|
| 1158 |
+
"target_tokens": 2048,
|
| 1159 |
+
"token_sha256_int64le": "cb791a719554b5899c11832912f980c09146c458b4aef527a3d5d0d406fd0fb2",
|
| 1160 |
+
"nll": 4946.471031188965,
|
| 1161 |
+
"ppl": 11.192781462061207
|
| 1162 |
+
},
|
| 1163 |
+
{
|
| 1164 |
+
"start": 290816,
|
| 1165 |
+
"input_tokens": 2048,
|
| 1166 |
+
"target_tokens": 2048,
|
| 1167 |
+
"token_sha256_int64le": "e7c568baaa1211ddf979a3cc4ebb53c6a45e42464dee6a4c5c7afe10fac4db8e",
|
| 1168 |
+
"nll": 3950.5697708129883,
|
| 1169 |
+
"ppl": 6.88254947069413
|
| 1170 |
+
},
|
| 1171 |
+
{
|
| 1172 |
+
"start": 292864,
|
| 1173 |
+
"input_tokens": 2048,
|
| 1174 |
+
"target_tokens": 2048,
|
| 1175 |
+
"token_sha256_int64le": "bbe3a070e9975e49aca0b68eaf924eea9d3c47fad52a4b0a1748cf5bcd6e3b56",
|
| 1176 |
+
"nll": 4339.320503234863,
|
| 1177 |
+
"ppl": 8.321219673570617
|
| 1178 |
+
},
|
| 1179 |
+
{
|
| 1180 |
+
"start": 294912,
|
| 1181 |
+
"input_tokens": 2048,
|
| 1182 |
+
"target_tokens": 2048,
|
| 1183 |
+
"token_sha256_int64le": "85f551396ba9f11283f03cead2c53f29ce2abedf4051c45bc16df0f34c048088",
|
| 1184 |
+
"nll": 4060.268455505371,
|
| 1185 |
+
"ppl": 7.261257017748293
|
| 1186 |
+
},
|
| 1187 |
+
{
|
| 1188 |
+
"start": 296960,
|
| 1189 |
+
"input_tokens": 2048,
|
| 1190 |
+
"target_tokens": 2048,
|
| 1191 |
+
"token_sha256_int64le": "2accf71a83660262bce4bfa2f3ce41f54f864b36b0aa6bb97447f8fb7522b4fe",
|
| 1192 |
+
"nll": 4390.114532470703,
|
| 1193 |
+
"ppl": 8.530181264889737
|
| 1194 |
+
},
|
| 1195 |
+
{
|
| 1196 |
+
"start": 299008,
|
| 1197 |
+
"input_tokens": 1955,
|
| 1198 |
+
"target_tokens": 1955,
|
| 1199 |
+
"token_sha256_int64le": "4da41d914b86530c1a9d5d118b5b0d9cb2ecf64f1884239fb41e30aef78f8e27",
|
| 1200 |
+
"nll": 3764.603406906128,
|
| 1201 |
+
"ppl": 6.859457391280554
|
| 1202 |
+
}
|
| 1203 |
+
],
|
| 1204 |
+
"elapsed_seconds": 25.85800075531006,
|
| 1205 |
+
"cache_dtype": "SSD scan internal precision",
|
| 1206 |
+
"cache_bytes": null,
|
| 1207 |
+
"logits_chunk_tokens": 64,
|
| 1208 |
+
"score": "next-token cross entropy; no BOS/EOS added; each window starts from zero state"
|
| 1209 |
+
},
|
| 1210 |
+
"loaded_adapter_content": {
|
| 1211 |
+
"complete": true,
|
| 1212 |
+
"tensor_sha256": {
|
| 1213 |
+
"layer0.V_read": "acbb93163b75e6eff584f4e4b3aa832e1419b7ff7119033a825efbd84b491eb4",
|
| 1214 |
+
"layer0.g_read": "b4477d6efd51b32a25f33c0bfe59a8888d0921dde342170c76f61ec8daddfa2b",
|
| 1215 |
+
"layer0.router_w": "458cefcfc8eefa68f3c969a9922e0a77e0e85bf69948ab58c0630089b7da00a1",
|
| 1216 |
+
"layer0.router_b": "48029583821428761aa3d097456ec52ff94b9bce7a33bd788ae74d8918388443",
|
| 1217 |
+
"layer1.V_read": "e7f67def7f12ad4b8b490892546b97ac61965d2e6e5bb82607ad16b847da2e4e",
|
| 1218 |
+
"layer1.g_read": "c3d3ec0c99f3d3c2bc4bca5424bbc507db84b9d8ce9537f5bf4e13f250af182d",
|
| 1219 |
+
"layer1.router_w": "f8feb0aca9c684ac43095c3d48382d691242ec2026eb7e664f37590ebad1809b",
|
| 1220 |
+
"layer1.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1221 |
+
"layer2.V_read": "33db90f8fd77d43b58db2836f267c436975737efa77a31dc6bb6310a57889269",
|
| 1222 |
+
"layer2.g_read": "3f51857d1e6d7f5cfe77d5b3ad17713cf04e90c08e600d491f5e7b10937f01ea",
|
| 1223 |
+
"layer2.router_w": "ccb5864fb848de8df2784269f4a7888c9a46d843aa6da7d5ae1d002b97dc26fe",
|
| 1224 |
+
"layer2.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1225 |
+
"layer3.V_read": "2fa3be14cf23207fe7f32eccdefcf36fe631b3aff7fce33cdb39de3b796cb344",
|
| 1226 |
+
"layer3.g_read": "6f3f0331ec9a3bf2c9e36f773706f9d0ad924b8140a8ab8fd8f88105ae0e3a9c",
|
| 1227 |
+
"layer3.router_w": "2a76cde8c029f103523681187f9ebfc9514055639cf93b09121f50113b199a62",
|
| 1228 |
+
"layer3.router_b": "de6343c9dac0fa5e944b0260a84812bf8e0cc2efd38014a76d85709a524ad535",
|
| 1229 |
+
"layer4.V_read": "0991b76aab9cb0464078d304f3ef460b0239b3c815e405eb476850c4bd89e523",
|
| 1230 |
+
"layer4.g_read": "b6e06f82309dd582792cc05e5279d24f4e07d2036ace5e52314987c6a0530292",
|
| 1231 |
+
"layer4.router_w": "209403602e80804d25e2e34ada09713033efcd947458237c8a076c3d923fade0",
|
| 1232 |
+
"layer4.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 1233 |
+
"layer5.V_read": "f60edcdb83b74a7d8b7511aed1abdf8e90faf6083ea84cd532694393e3a7b635",
|
| 1234 |
+
"layer5.g_read": "3fea77dab90cfbbdc2c08e8407755ecf3d47b165cc8eaca3a5a8ec401f3af52d",
|
| 1235 |
+
"layer5.router_w": "bfa4e2def4cea5f6e40f1f664c0200c9bb812995721998f027109b6d6a19ead2",
|
| 1236 |
+
"layer5.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 1237 |
+
"layer6.V_read": "5c0864235673c03f19285c0019320389aca05547ba122af8d15ed3e8afef768c",
|
| 1238 |
+
"layer6.g_read": "0416d68791a8552f8722d36d7d0caf0cd57595693a90b4d2fa42b70cf1b5af9e",
|
| 1239 |
+
"layer6.router_w": "563b89e03b5e6d7913580b5d486656341b1e9e84724b0d231f5de5ac28cb92c0",
|
| 1240 |
+
"layer6.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1241 |
+
"layer7.V_read": "d0dbb98b4b4163c85770854cb4f97390874d7b8a8eedda5af7fce9af64020bf8",
|
| 1242 |
+
"layer7.g_read": "9109e1e7ba2c012a903b7848f7529f18dcf9d5d8d552a2027f58652202dc95be",
|
| 1243 |
+
"layer7.router_w": "2c6ebcf67f7de3a3ae12c938fa075bd2c169e9dd64ad04d072a58a8e11d031cc",
|
| 1244 |
+
"layer7.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1245 |
+
"layer8.V_read": "6695fa69c3b6702868cb1fb3d2992b62b93485fbb49fe3335cffe121579a609a",
|
| 1246 |
+
"layer8.g_read": "ebcd5327a3cb439ce1bd8660004aa6aff79570ba73173621938bade7b0dabfd5",
|
| 1247 |
+
"layer8.router_w": "5f09a5e356890db8a388fe26c6f62cdf83d45a87fc9260343b805a24057bf969",
|
| 1248 |
+
"layer8.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1249 |
+
"layer9.V_read": "e52b198e7d50aea801a5a7933354d4337bc803e8e1b8eb189a1531e808e5dae2",
|
| 1250 |
+
"layer9.g_read": "bb0cfe4441ad5f0949e595ccd96aa88c057c85f61cd20bb9e2c1d39bf87e0449",
|
| 1251 |
+
"layer9.router_w": "305877528ccea774050a4371a19881d4ff4132df431dad61a92e743f758bc432",
|
| 1252 |
+
"layer9.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1253 |
+
"layer10.V_read": "b5a8b920d24aa7826191ab90da769226bc8d259dda30f9706799edbd7f718546",
|
| 1254 |
+
"layer10.g_read": "ae15ddbad1412dfa1a3ce90c7add4ac857b551daba6e9020121465a735bd56ab",
|
| 1255 |
+
"layer10.router_w": "0bdc2944213d7df1ab6827ffff224c7cd110e68795aad1ea5dfcc7f30e1d9708",
|
| 1256 |
+
"layer10.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1257 |
+
"layer11.V_read": "1b105e6f0ba0fd524614f741823fec6e822cd2a46dc74eaced5636858b65afb8",
|
| 1258 |
+
"layer11.g_read": "ff712f2f966ca299ceafb63832935f569fa562bb4b48263c7d9a9462f3b9569b",
|
| 1259 |
+
"layer11.router_w": "d132b46115bc43e4ca660d126b3110daaa98f6fb8ba26873d2f529734885b28f",
|
| 1260 |
+
"layer11.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1261 |
+
"layer12.V_read": "0fdd5af3782a2a165a6d82d0bac1bcbe946bd57a255e4f5e74777e0d5d90838f",
|
| 1262 |
+
"layer12.g_read": "5ba39a0f3d7a11dec51509cd9723b9770d0f29d54ff6bdd16ee8f0a853013769",
|
| 1263 |
+
"layer12.router_w": "98232121a4a8cafbfa7b3956dc43055a7f5830350d48a4bc5fbb6b2a5ea1260f",
|
| 1264 |
+
"layer12.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1265 |
+
"layer13.V_read": "74004cb52e34a5dfe0ee12092f7ddb2e21e5c8ff470aab1c4bc11bda2c99bd76",
|
| 1266 |
+
"layer13.g_read": "d774e150badd05bd200cee2089cf2ca2c1ae1581e8d991b6655a3522377b4ba7",
|
| 1267 |
+
"layer13.router_w": "fd7312d873043cd64718bc31895a07147e2f2e88821bb2f2e14a171a681b126e",
|
| 1268 |
+
"layer13.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1269 |
+
"layer14.V_read": "1de745e5fd11e59ce1c54f1f2a03d1feca64cfb607a52cec5a8cabdc83be8d11",
|
| 1270 |
+
"layer14.g_read": "b57ea67a980150db7c91687a61f76f68b6deb238fb1b570d3385cd6feb658db8",
|
| 1271 |
+
"layer14.router_w": "56430783c73b2250a9d9dcd3de173608f782e3b5c1308c74ad1fa03cee978f75",
|
| 1272 |
+
"layer14.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 1273 |
+
"layer15.V_read": "ad90c47d8f475c9cdbde883e9a59ee2b2c0e5c25d6348e8f965c3dbda258c326",
|
| 1274 |
+
"layer15.g_read": "ceae141021d9d934490a9dca0e655656b0e80d8df870da426beb829a365e56d3",
|
| 1275 |
+
"layer15.router_w": "4815b045f15450e69e2cce3332fe3994e075f3426f09e2fa334e057069004f67",
|
| 1276 |
+
"layer15.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1277 |
+
"layer16.V_read": "7adc0b9a51b6bd02d3e36aab417225423f8d66c0fb03a5b3e62b04841e7c4896",
|
| 1278 |
+
"layer16.g_read": "5e44144d2892ce7e7f48210350064383a0d9485101e76fc81b14805704ef7dff",
|
| 1279 |
+
"layer16.router_w": "f94c27b36b256ff81b8d5bf35d4289829c34723016f24802ccffba3fe12dc29a",
|
| 1280 |
+
"layer16.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1281 |
+
"layer17.V_read": "11009457018988fe510e65db34363a4d52972b05a0a655d5bd4be3f22aff80df",
|
| 1282 |
+
"layer17.g_read": "376799acab3b7b55eed58b79034eb8bfa861c073a446e88037145cd8206661b9",
|
| 1283 |
+
"layer17.router_w": "9638ef42133a5eedb14d8a569106fdee455c07da830fed4658881da15c883604",
|
| 1284 |
+
"layer17.router_b": "f585636255da971019a4ff67ee970b644ef4d4f493a597af54cd4390f07fd033",
|
| 1285 |
+
"layer18.V_read": "938246fa3a4a2c344dea5380b6ebf4fb2e0af50c9da754c4b0aa9fc3d9561c55",
|
| 1286 |
+
"layer18.g_read": "c981b84b1aae28ef5e8130d251edaa877352f4a97f6d7b02b855f993cf1b5f84",
|
| 1287 |
+
"layer18.router_w": "b25ee0021de616ccfd5ae3b94ea341f038494efc6a517b195a22f283b58f9bcf",
|
| 1288 |
+
"layer18.router_b": "f585636255da971019a4ff67ee970b644ef4d4f493a597af54cd4390f07fd033",
|
| 1289 |
+
"layer19.V_read": "b16fd7decc91f0009994fc313e841bbadb5bc7e037b28076cf54700aecd8fd6d",
|
| 1290 |
+
"layer19.g_read": "0ad360f92cf2ed90256ef5f8c1c38fba21829fa4de4a8478a1bc23ee8c832479",
|
| 1291 |
+
"layer19.router_w": "fcdbede22d6ede0bb3c26b6aa90bb0069bf7dd99b80b151173030010874d5939",
|
| 1292 |
+
"layer19.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1293 |
+
"layer20.V_read": "c2e8924803aecfec49945c0c482ed6d6edc6005bf993787ea40019fdd4705d57",
|
| 1294 |
+
"layer20.g_read": "de8a1d4c27fed3ed6e0a6343ba13a9c92f26f3b4ae4914355ece4c07a9ec3c99",
|
| 1295 |
+
"layer20.router_w": "69655bbd274b9d5d911ffdf01a05e1c0dc5d8fa8d652e9e7e36b3498ee37843c",
|
| 1296 |
+
"layer20.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 1297 |
+
"layer21.V_read": "84dd135a6f6832679102d93ba83efbbbe8686221c86a73e06e9ce478a1ccc8f8",
|
| 1298 |
+
"layer21.g_read": "59b7d74b02b75f627b40fff6d65e69c877dfb2dc2e11f1bff86970d33749f294",
|
| 1299 |
+
"layer21.router_w": "e3b0d6acd08da906697a1e014f8e1ca8456db581b290a2b01104115340e474ec",
|
| 1300 |
+
"layer21.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 1301 |
+
"layer22.V_read": "f78f4703e13c2e677a9b3dd606b983c0807be7987a8ad03baff3decc07972cdc",
|
| 1302 |
+
"layer22.g_read": "87bd57c65ababc859992824a45108805b7d2e909b9e1904ca309bb3b3528464f",
|
| 1303 |
+
"layer22.router_w": "b17d62b05781030730ec480cac4bf17c94d7f1b57460e66e9b115c15f1a5abf9",
|
| 1304 |
+
"layer22.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 1305 |
+
"layer23.V_read": "a3be6134cf4efd99f829250ca60b24523b686e30006a9f39fece1603dbd9d378",
|
| 1306 |
+
"layer23.g_read": "21e5f2238a3718502e0708e862d14b647f7d6065fa878e0d8f23bb9a29793acd",
|
| 1307 |
+
"layer23.router_w": "24f9f1bc48d369bb86b49a682fbfb36f58eefe9088213206701bb5e0a5d4bb25",
|
| 1308 |
+
"layer23.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 1309 |
+
"layer24.V_read": "2d2e712a2383e870754d1d748690ecf5e0d8a84e8669999da29b60543fd259a6",
|
| 1310 |
+
"layer24.g_read": "6927b95f16effae647ba7cd3f2d94498779004b1fc4d952e4105a49ace5ca7cd",
|
| 1311 |
+
"layer24.router_w": "aca8e53213586f7e59122b4d0523d11c23f95d063aad8cef8f780938ba90e310",
|
| 1312 |
+
"layer24.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1313 |
+
"layer25.V_read": "78ee73d73c34109e3c9935a5bc5b12e933a623218031845a84c3b1f444d6d415",
|
| 1314 |
+
"layer25.g_read": "0b6b84b951b187a2a611e41a98256f660a0e39841385bfc8334af413ceae88f6",
|
| 1315 |
+
"layer25.router_w": "17b2f32c6af11c8d5a115346ad1aef363b2cd5c8211ec7f5770bb440ab102860",
|
| 1316 |
+
"layer25.router_b": "ebbb11c8f59b403f66668cc95e0db57bd4e2497576522ddebec771d77670abd8",
|
| 1317 |
+
"layer26.V_read": "6267dd0f6a9077f2d0ced9151b6745a7372b18bc46ad22f4c0004c7738515ffe",
|
| 1318 |
+
"layer26.g_read": "dc3aa502134b682aa39d69d47c26d6b9f6dbe81bed70f86111eb885a7b559423",
|
| 1319 |
+
"layer26.router_w": "5fa67113ad4054a3f456a2ce3915d272d35345781a418d7d4c27b78127d63d86",
|
| 1320 |
+
"layer26.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1321 |
+
"layer27.V_read": "30649a9be6e9c316a602aa5d9af1f0b117c8b542a2d29e808be10e71761e4cfb",
|
| 1322 |
+
"layer27.g_read": "cf94e71c9666cd56a4e7f64e14e08a858098a5f5e4dae4b893ed46fc9bdf8383",
|
| 1323 |
+
"layer27.router_w": "fa5b088959330d66d107cca1cde24b210ae96a60669ba969500e13869ef082de",
|
| 1324 |
+
"layer27.router_b": "f47fe0dbb02326606c3752306eeb57abfa1b8370545ad7b23a91791074459042",
|
| 1325 |
+
"layer28.V_read": "01a98b0b2519586522e92b3c43b2c135c89c76711f004fc83501dcf3880ff2d9",
|
| 1326 |
+
"layer28.g_read": "ca83a526129c4d5caeddb1253ba750644a9ac3219dbfd0fec21c8d98647d4969",
|
| 1327 |
+
"layer28.router_w": "825cb808a2185545829272ab8d11f637a4d87ef59b93d37ffb718e01955e19e2",
|
| 1328 |
+
"layer28.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1329 |
+
"layer29.V_read": "79356640911705e471c44e9eff9dd676e9c04309bed974e6589f7bab88336c52",
|
| 1330 |
+
"layer29.g_read": "aeb0d1c4809e35e12982ba62667d351a320256a17e807a9fc598c941b271775a",
|
| 1331 |
+
"layer29.router_w": "46783f9d0b233ba23d703deaa1857c19f32a9e3ddf2ed215a6e6ddfb29998f12",
|
| 1332 |
+
"layer29.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1333 |
+
"layer30.V_read": "fff68081265f94aeaa96719f8bb5de2d8c534ab7b8755c394724aa299666fb59",
|
| 1334 |
+
"layer30.g_read": "e78c18028c02faeb85cb0f3061343a0f31770794d2d4eeef2f335522c3e2578e",
|
| 1335 |
+
"layer30.router_w": "6ea21eb8e0baf13e127901c7391cc1ee1985da1e27fc23b1094a4292be6e320d",
|
| 1336 |
+
"layer30.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1337 |
+
"layer31.V_read": "960b52288cae25aaa4526f589e522fcb43389aa6b84cbd41416ef2eeb584df31",
|
| 1338 |
+
"layer31.g_read": "80e213c9633dfcc4754066dd28d594b6697ee6e7f1d14ddc6f71a94a41c40628",
|
| 1339 |
+
"layer31.router_w": "736b61bc45aff35b77ef4dabc1471c7c25240319d4b71d3b11b0ca7b9d25c79b",
|
| 1340 |
+
"layer31.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1341 |
+
"layer32.V_read": "65d9afa6a2010e14ff7d659bcd4570576297026b2a13f12b98a8f57a2a6f2a64",
|
| 1342 |
+
"layer32.g_read": "9c02d68d5937b033abf4e9fe8a38a377d93066d7637e3f57bdbd581dcc735ba5",
|
| 1343 |
+
"layer32.router_w": "d24bcef9c4a2636330e698df6b644516c462b34ceb233923236a9b3794770a90",
|
| 1344 |
+
"layer32.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1345 |
+
"layer33.V_read": "f5b365067a71ae38e5aea7b1c0c08af1831a98cc24579ba89b5b5218bb0cd066",
|
| 1346 |
+
"layer33.g_read": "1b9cd0f48b40a423f1debc063e2ccff72b22e114cbcf557652e6ecad1c9192c4",
|
| 1347 |
+
"layer33.router_w": "2cb3289236db97b651ff1a0d917bb3267d567bbf65cbf67b3cdba67efb2f01a7",
|
| 1348 |
+
"layer33.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1349 |
+
"layer34.V_read": "fab302f6d5c2dcf2d11dd8d97f047dc5fc39377ccae00aae6074dc0d87bc718a",
|
| 1350 |
+
"layer34.g_read": "8fa3487358ddbd8d7f01ffcb96a83a643fde88701f78ee68196bbc27774fc132",
|
| 1351 |
+
"layer34.router_w": "4f9edad72ae32340f69aa22b371ad46b8866460e7e633910cdbeda92d7b09db0",
|
| 1352 |
+
"layer34.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1353 |
+
"layer35.V_read": "37dd7050a6c03b2d6c8576fe0976077a43ce258d159afbb15f9d3b6eae092187",
|
| 1354 |
+
"layer35.g_read": "8f79acf046acbca6f2e5108b7b53d283ff164b79d07ddd12091ad1e4ce619578",
|
| 1355 |
+
"layer35.router_w": "8f30c37d0784ea6d3347ec5e38d8763697ca8ebf61c8509e8e0f1c7bf0fc99e4",
|
| 1356 |
+
"layer35.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1357 |
+
"layer36.V_read": "5b9ab1226710787592b3fd515fa0c79197e12a2f1963000d8487b360728e2a44",
|
| 1358 |
+
"layer36.g_read": "f56be0851ba3de3be933beb07e49881add6c2d0501e36efc7f391cbc81331e60",
|
| 1359 |
+
"layer36.router_w": "0a64c450ae5e238b9e0cbb804fe44d13566f06934bc7ad07e5a7b33a4ebd9c55",
|
| 1360 |
+
"layer36.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1361 |
+
"layer37.V_read": "e4e630278f9cf7b4cc06d3a4572802ab09fd939931095ae8fd36bf84d1ef69cf",
|
| 1362 |
+
"layer37.g_read": "e9f179ce940df2e6a86ba4d6089d110abd27528e92a55eb88e0e678038eeebe2",
|
| 1363 |
+
"layer37.router_w": "be73e471e4217911977da45fb18cb1843bf7a07bc59a5b81d9c7ed32bffe303c",
|
| 1364 |
+
"layer37.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1365 |
+
"layer38.V_read": "0220e99471aac1a35ef7f17e0969481f9093be0ee5d68c8e1ba160b8475539fd",
|
| 1366 |
+
"layer38.g_read": "d0623a01efe8af6efd8afcc808f23efc24b071e69f9d85f28cf3f5a5f2a7c248",
|
| 1367 |
+
"layer38.router_w": "e35efac6a118f723f5628051aa40df067164d9a7a0422ccd9e25d6671eb61902",
|
| 1368 |
+
"layer38.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 1369 |
+
"layer39.V_read": "20db41719bffcf75bedfc9e2e8341d6857b003e7b0eb4267dfd31c86cc56300e",
|
| 1370 |
+
"layer39.g_read": "cb7801a184b1bc83380021fa67e160e00ecc33c695aed80bc974822493cb683c",
|
| 1371 |
+
"layer39.router_w": "2caa083f0ce4682a5eb31bfc27b54616beefea540d2d182f7bf09944f205acbc",
|
| 1372 |
+
"layer39.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1373 |
+
"layer40.V_read": "f26e7cbb00c98256fa43291a11ed335ef009b4ee96941c300a0079e976d1161e",
|
| 1374 |
+
"layer40.g_read": "9a4602baa58acc0b3f64c642ffdb395de49576b73860cc5cd0d6929ec59a789e",
|
| 1375 |
+
"layer40.router_w": "ced1c46686c5d2275481b692f8a23002c67e75408e255c148cb91ff693f4da7f",
|
| 1376 |
+
"layer40.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1377 |
+
"layer41.V_read": "9270e10c0381b11aee2cd22b9e8b89f6610eff8c6f14a5e09b2e4b60a1161641",
|
| 1378 |
+
"layer41.g_read": "dea048bfb55201b82e06d7ee1be785e6b8683a4dec14cf6bbcf57b07eb9402f4",
|
| 1379 |
+
"layer41.router_w": "877981b64edb45fda12d1114887f093726fc44b4eb4a7cf806290e048c05d51d",
|
| 1380 |
+
"layer41.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1381 |
+
"layer42.V_read": "2336f9a50806cc029dea41a23846369b69323b28793f605ca6512b8a809191b4",
|
| 1382 |
+
"layer42.g_read": "731ca130ed585cc26cee18f8944e438d85f9e3455adccb3f1e326769ecc25fae",
|
| 1383 |
+
"layer42.router_w": "2c2f4edb7b74f5ab9f53515a6eee49629e8ff372e2681dbdc22451b946fe065f",
|
| 1384 |
+
"layer42.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1385 |
+
"layer43.V_read": "be633b5c6a65efbcd08275bdde90edc5105a5923a7db06545227c5687b2ab084",
|
| 1386 |
+
"layer43.g_read": "6a0c1f2210e4cb409beb386ea1665929363785f4a2675fa79419b841b2f9a98e",
|
| 1387 |
+
"layer43.router_w": "897bef6f5fdcfb72f743f8937197460307cfe7fd0689e93a565ed9cb5a18c11c",
|
| 1388 |
+
"layer43.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1389 |
+
"layer44.V_read": "12240bc8f29cad323a0971a942e3236df552c9baa5d6b65225e603b32a105cfb",
|
| 1390 |
+
"layer44.g_read": "9ddb3a4153cb876deda7f13a3694a9dbcd75593f9bf9b2f624f943225009a42b",
|
| 1391 |
+
"layer44.router_w": "15852502c73f82718d3608f61506497038d55fefac716ea26ea30754fc4b6a44",
|
| 1392 |
+
"layer44.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1393 |
+
"layer45.V_read": "c6e259d615a04224c603ad1560673cead08ef91d700a7c68b7a4bf51c242bbd6",
|
| 1394 |
+
"layer45.g_read": "471b70bdf722bc78475ddb946bfb4fe5480640a9eea8ceaf763688a7061d3ce1",
|
| 1395 |
+
"layer45.router_w": "cea33b64eb22142e9a441fc52c04e79351fa32004c43869545ddbaa78991aee1",
|
| 1396 |
+
"layer45.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1397 |
+
"layer46.V_read": "febdf1a3c15e2a46038a9ddcaf8cdbfbab871bcb4133e0fe2eb3c3c8993eb4f0",
|
| 1398 |
+
"layer46.g_read": "3007c79fd9fd7f63340c19994495e9303decf9c56ebbe0d71d2514a6c3902e15",
|
| 1399 |
+
"layer46.router_w": "55e05ee5bd460dd3d42d88e67e63188faa078c93611c37cc5ca00520f2b2d657",
|
| 1400 |
+
"layer46.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1401 |
+
"layer47.V_read": "c9774380497d9338e6bb325a86fb7e9406080b96ffe66627c5730d2e746141fd",
|
| 1402 |
+
"layer47.g_read": "e3e2e19243ff02c0a536a6f3ee2e47e8a91abed16a0cf8bf0dd766f54eb59a99",
|
| 1403 |
+
"layer47.router_w": "83c101bcd7761d0f8b890eb22bc57b42b1b0cf74a163d3e520ad3a15fa460a87",
|
| 1404 |
+
"layer47.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1405 |
+
"layer48.V_read": "01413acae1c48270083817749978d3afe4f5190895d7c8ecaba3b99d6f838c31",
|
| 1406 |
+
"layer48.g_read": "69b08496e23f518458163e80a872a913373992a56b0edff3b5864392f3df002e",
|
| 1407 |
+
"layer48.router_w": "b71c2a0e5f4d476df429d72c971c3502dad832f4ef24400830aad1e90b9d1c21",
|
| 1408 |
+
"layer48.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1409 |
+
"layer49.V_read": "6e6a1fc67654e3902c9c28d31e6877c802ba95968913a265897ef4a40c06607a",
|
| 1410 |
+
"layer49.g_read": "5a3e9324e1243877ac164f9d0c0bd1ae9ece80e2f15c9acda6f295b1500308b1",
|
| 1411 |
+
"layer49.router_w": "36652898bd574b54803b72acce4c6818770e1ef70b9605512ca5601d4abea612",
|
| 1412 |
+
"layer49.router_b": "df8dac016c676e9f895ab0dbca95ddd64681d8816e2d1fcc6d44c3472e93eb49",
|
| 1413 |
+
"layer50.V_read": "10d5de7fb1e594cf7e44d009c0d08c7360aaa969b6cd6bb3abb20d0fe53784d0",
|
| 1414 |
+
"layer50.g_read": "8e2b9fdc0f9296084ceac9fe96e79f937293b653bf276d16be876e960faccd97",
|
| 1415 |
+
"layer50.router_w": "c37f8cd0776a306f5c0cc726384f23b19c94f499ed0cb95eaf156f8593584234",
|
| 1416 |
+
"layer50.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1417 |
+
"layer51.V_read": "d3787ce0b8489582692bbd2f4a67ef1380098ff784f18898e6077fb812f54e83",
|
| 1418 |
+
"layer51.g_read": "55359a2bb1cad616dfa055bd20da9c52142a0e37f9382eff9ee8dbedebf79977",
|
| 1419 |
+
"layer51.router_w": "6b0225c43ef7db1d0415f3b67fb627b2fb098626c427d3ff57b90d9a7fbde3fa",
|
| 1420 |
+
"layer51.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1421 |
+
"layer52.V_read": "4b5ec34ca0da5d988a58e59d4b6aea701e47c2d69314d3f77cf19c889740298e",
|
| 1422 |
+
"layer52.g_read": "951a8e8554f1ef867b5d9ba072cfc139782cd9e8a657871058d7590fc334a6a6",
|
| 1423 |
+
"layer52.router_w": "8fed96881d126a411d6a35d9bfd13282101dda66e90cdd063bd56136d3216472",
|
| 1424 |
+
"layer52.router_b": "82b5a0487992a93bbceefc7d417bb0a9ccfbbfd955b99d0784bc73a8f1de5b53",
|
| 1425 |
+
"layer53.V_read": "0cddc47ef8a9d869eb3f1302889e11203394ed4e56923addc654d38419238fd4",
|
| 1426 |
+
"layer53.g_read": "dffca0ba32359bd619327426d6d616e85db319b7316fae5628e05cf37960dff3",
|
| 1427 |
+
"layer53.router_w": "6d3a53968652f6f05b51b4e9e2de007e3b56cb691db022595e59608e37ee07e9",
|
| 1428 |
+
"layer53.router_b": "507f223d9339008322a491d3d4d9dd77e2e1dd99dd88b9a17e19e2bfdb235b65",
|
| 1429 |
+
"layer54.V_read": "a9dd4f7a3539ae26d868748682b65cca71dd135644a4c918571fc9ff930676c5",
|
| 1430 |
+
"layer54.g_read": "8e131fbe86a8a9e3215b4f3b381a255921f10b06566d020f330a276f8af8c4de",
|
| 1431 |
+
"layer54.router_w": "5f06feb297a625ab67df5ecf3990956681c60bee17f1e19e611629f41c9e7299",
|
| 1432 |
+
"layer54.router_b": "de6343c9dac0fa5e944b0260a84812bf8e0cc2efd38014a76d85709a524ad535",
|
| 1433 |
+
"layer55.V_read": "7d8e5fcbcf68199fd68b348b3f393c0a8728b765b600b68961ebef0c3090199e",
|
| 1434 |
+
"layer55.g_read": "a3c954b3221b64c08fdd9b27f918faebd17f5b9dc9b6c6b978943bba180f4309",
|
| 1435 |
+
"layer55.router_w": "48aa1e26ec214f74413e6763217efd089fbee7ecb6654c44ede62b948a1dcbbc",
|
| 1436 |
+
"layer55.router_b": "1eb9418e6a22f0af4c2d5edd254d59768916f24ea54c54ac9d4cf45f74862628"
|
| 1437 |
+
},
|
| 1438 |
+
"payload_bytes": 2308208,
|
| 1439 |
+
"parameters": 1154104,
|
| 1440 |
+
"tensor_count": 224
|
| 1441 |
+
},
|
| 1442 |
+
"frozen_base_check": {
|
| 1443 |
+
"tensors": 507,
|
| 1444 |
+
"parameters": 8236999680,
|
| 1445 |
+
"identity_version_gradients_unchanged": true,
|
| 1446 |
+
"actual_content_checked": true
|
| 1447 |
+
},
|
| 1448 |
+
"base_identity": {
|
| 1449 |
+
"complete": true,
|
| 1450 |
+
"tensors": 507,
|
| 1451 |
+
"parameters": 8236999680,
|
| 1452 |
+
"identity_version_gradients_unchanged": true
|
| 1453 |
+
},
|
| 1454 |
+
"native_restoration": {
|
| 1455 |
+
"complete": true,
|
| 1456 |
+
"modules": 454,
|
| 1457 |
+
"forward_methods_and_hooks_exact": true
|
| 1458 |
+
},
|
| 1459 |
+
"cuda_memory": {
|
| 1460 |
+
"peak_allocated_bytes": 16862959616,
|
| 1461 |
+
"peak_reserved_bytes": 17075011584,
|
| 1462 |
+
"scope": "Resident decoded FP16 weights plus runtime allocations; not encoded model size."
|
| 1463 |
+
}
|
| 1464 |
+
}
|
reports/wt2_test_v1/weight_ledger.json
ADDED
|
@@ -0,0 +1,515 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"provenance": {
|
| 3 |
+
"method": "Verified pinned BF16 source cast to FP16 in bounded CPU row chunks",
|
| 4 |
+
"source_checkpoint_sha256": "47c2766f6aad89d73beafbeaecb334aab902d7370906d081764a90bb7a8bbbcb"
|
| 5 |
+
},
|
| 6 |
+
"tensor_sha256": {
|
| 7 |
+
"backbone.embedding.weight": "615d80463f9c7a53b614944805e3307409a25549dc3ba428eddaad26cdfb6239",
|
| 8 |
+
"backbone.layers.0.mixer.dt_bias": "575d992f7d30726a3ed0ef4404363b3bb05517d6c5c9919fe0ce6f71e82a2f95",
|
| 9 |
+
"backbone.layers.0.mixer.A_log": "29392b76e84485e739e55ec93ac1f3c9a5fdd773f1507e05a646594b3318d509",
|
| 10 |
+
"backbone.layers.0.mixer.D": "f403f7ff1c7b8bf49948587eba321a8b88d4e251298068f2bc7e7f81dac3aeaf",
|
| 11 |
+
"backbone.layers.0.mixer.in_proj.weight": "af1c4d73a86b263940dcda4a851b6bd9d2a78fd631ebcf816425b251fe921dc7",
|
| 12 |
+
"backbone.layers.0.mixer.conv1d.weight": "6c20cbe43da8393c6b58312e7d08d28d6f11f2c27917f763c2ad43a400947a01",
|
| 13 |
+
"backbone.layers.0.mixer.conv1d.bias": "2f8af760ccc9b5c4a9571e6b2d7911341ae2de5a20f78d8a7e91ab78e2cb45c8",
|
| 14 |
+
"backbone.layers.0.mixer.norm.weight": "2dc7d90e518356c594e0b8ccc49246f2b0538d11176dc515d3dbbcf728b9d399",
|
| 15 |
+
"backbone.layers.0.mixer.out_proj.weight": "548d4391dc23a5e3b97e47148ab5e1f532acfaf1c0210ee915b1bf6693fcf971",
|
| 16 |
+
"backbone.layers.0.norm.weight": "5562a9691038b64dc2b252531869400abc26cbeb3c6c2fcc20ddf3f2dcebae4f",
|
| 17 |
+
"backbone.layers.1.mixer.dt_bias": "ae3bc85972c86c363f53effd0dc73bbaf26e8e1dc3712b93a2b55e355c272d2a",
|
| 18 |
+
"backbone.layers.1.mixer.A_log": "c3e07ff4f6828621d19e6f9f485707a4e1a0dbc1523846859e3977c96a5c9955",
|
| 19 |
+
"backbone.layers.1.mixer.D": "3f8d560bb4c20630d019b88b266febddecb796b6107dc1383fedc5b8a940f93b",
|
| 20 |
+
"backbone.layers.1.mixer.in_proj.weight": "4213986e1c80d8a410d8168a3828c26bf08aa6ad6dd18b1946f352e9ef80464b",
|
| 21 |
+
"backbone.layers.1.mixer.conv1d.weight": "1466752dc9510ffa8f81980222280bd544cacfdf18b4828c44323993fc7259df",
|
| 22 |
+
"backbone.layers.1.mixer.conv1d.bias": "1bafe69217bc1f0b2897cbf9025a6566173295d880ffadf59fc34954598e6453",
|
| 23 |
+
"backbone.layers.1.mixer.norm.weight": "c67bd1ff390b579f7baa69cd9f2de1d2b6ce72c73371481b9de9addbe340506f",
|
| 24 |
+
"backbone.layers.1.mixer.out_proj.weight": "3dce4500efdca328be0700b40ca84d921aaf1d6c1712ceb4f1e6fd2f286c845b",
|
| 25 |
+
"backbone.layers.1.norm.weight": "9e29e2f3de238898a191e6bbba0b0852e2f15dd41466879da36be8d518b2199d",
|
| 26 |
+
"backbone.layers.2.mixer.dt_bias": "15cde4f788f1ff9485dc227f4e9ebce8086db1708b906f1f978da5f91575c0ee",
|
| 27 |
+
"backbone.layers.2.mixer.A_log": "5d489b5a3aea0568892de8ffe6bdb9f66393f63fd9950d46e68bce5890d8cc22",
|
| 28 |
+
"backbone.layers.2.mixer.D": "a8e6d5bb5ad6b6f5b2c348c29942d5257c8ba57ac1d5911979a267a9a7cc7c61",
|
| 29 |
+
"backbone.layers.2.mixer.in_proj.weight": "de358e3311e6524b58965bc0b7628fc8a995bc91209b290438a554183a2b0493",
|
| 30 |
+
"backbone.layers.2.mixer.conv1d.weight": "6b9e2f4ade006643ecd590ab25ca3589a261eb00a96cd7d7335206b80dba5524",
|
| 31 |
+
"backbone.layers.2.mixer.conv1d.bias": "4f456087ed19f4e735c6d6255b9cf1625cdee2bb5bf2445b3a8c17df6969e2d9",
|
| 32 |
+
"backbone.layers.2.mixer.norm.weight": "51b44f3709aef25e046659951ac9c318492f578e513ff84816fb41ef36119108",
|
| 33 |
+
"backbone.layers.2.mixer.out_proj.weight": "809f642a6c155a0c3009eaf47116d4e042b1adfce057d6f8478666abdf25b704",
|
| 34 |
+
"backbone.layers.2.norm.weight": "bab85a52f73b19bfe8ebfa48bf15b30bee7ecb683bbfcd7d34eb75ee30b53fd2",
|
| 35 |
+
"backbone.layers.3.mixer.dt_bias": "269a5fbcce1bc2c51819b1e7a5558a29775161ca550099a8a105961d11842405",
|
| 36 |
+
"backbone.layers.3.mixer.A_log": "0fcecbdfef16ee9a572cfbec4cedfab376b4c779389acaf0f52f548dc764ed73",
|
| 37 |
+
"backbone.layers.3.mixer.D": "1554e5ad7649f1544d2f41cde1e36eb3d68b19396feea8a3c84d8f3b9dd6ba57",
|
| 38 |
+
"backbone.layers.3.mixer.in_proj.weight": "e6778b07920b586958514750c590cd6b554e59ff14800fb71dd68bee62a568c2",
|
| 39 |
+
"backbone.layers.3.mixer.conv1d.weight": "aa8686394d656efac69db1c9167911b2a3c34fc47e5abaf54730ab2dbf932e15",
|
| 40 |
+
"backbone.layers.3.mixer.conv1d.bias": "428fd49dc59e1387638268ad2a2009b78e311ccde29ae0ef25a5a45ea392db40",
|
| 41 |
+
"backbone.layers.3.mixer.norm.weight": "bc0e2fdc620a26d59c1dae62a1596ce4d03a75c78231c884cfbd13e6535b8ad7",
|
| 42 |
+
"backbone.layers.3.mixer.out_proj.weight": "4dc61971d9de628b96dfcf73bafe379f74363c3cf54a5c20f800a7bac1cb5aad",
|
| 43 |
+
"backbone.layers.3.norm.weight": "01ed7860131cd8bcefabff40c3228132fc222b7a5eeddec03e7a4d13fd137758",
|
| 44 |
+
"backbone.layers.4.mixer.dt_bias": "330c9ada10d50b5b1395a49c669a6c6866cb89c67b5dab528d216bd183d29306",
|
| 45 |
+
"backbone.layers.4.mixer.A_log": "81b2cb9c623779e72fc6f0baee4ce062d421bc3b2ee092df00b2da922752a18b",
|
| 46 |
+
"backbone.layers.4.mixer.D": "d60e0ec52a650f6df590923518c337ed37af616802d683947cfb8b2be9da3c04",
|
| 47 |
+
"backbone.layers.4.mixer.in_proj.weight": "e19868105d85df12c26529286614ec54f75df5e638e93fca952df8d1d21b4d97",
|
| 48 |
+
"backbone.layers.4.mixer.conv1d.weight": "5d2e1b198cf100fcece9605aaed3bc93b5141a8a17311eb4b1942bb41826addc",
|
| 49 |
+
"backbone.layers.4.mixer.conv1d.bias": "020308daf8003db276077de20db5bf9b015ba5379195b4550cbc12f21d989f5b",
|
| 50 |
+
"backbone.layers.4.mixer.norm.weight": "3b91251395dfbf0093dacfacd7b2c2c83758057a0a0280e0c4e46c08c6d37e34",
|
| 51 |
+
"backbone.layers.4.mixer.out_proj.weight": "705050573b90e641e8abee30ef93034af67df4235777c66d92e4e4ec99de86b7",
|
| 52 |
+
"backbone.layers.4.norm.weight": "f2ce50fb37126ddd30ba3beb7b213b331c1746bdc0c60ec6c5c26fd82e1ecd0c",
|
| 53 |
+
"backbone.layers.5.mixer.dt_bias": "63264b7f6a174f9440dff8dc8ea288244c1bfd1d57521e59bddda1e1fa431610",
|
| 54 |
+
"backbone.layers.5.mixer.A_log": "1b53fda9c3153fec7f9df48545658001dde1edb43e6f459845b02cd0b77c5147",
|
| 55 |
+
"backbone.layers.5.mixer.D": "4172d001644056bd136afdf0858a678923d470657a551a643b765360d04ba519",
|
| 56 |
+
"backbone.layers.5.mixer.in_proj.weight": "a8951d14a3f523951cfae6f9fc198eb041b6911c9e66ced7aebb4914da8ea591",
|
| 57 |
+
"backbone.layers.5.mixer.conv1d.weight": "79c906ba95e398316653dea595561039a13c17225d83ad84b6875ce1a33fc9e7",
|
| 58 |
+
"backbone.layers.5.mixer.conv1d.bias": "27c9d3e0cde45c9092fb572d44ce3a1af4a279679805dda4941eb7d63441c1bb",
|
| 59 |
+
"backbone.layers.5.mixer.norm.weight": "1e5d36f08245f889d3509d61eaf571c5aadd5fc230e98eb021241e81fdbeb7e2",
|
| 60 |
+
"backbone.layers.5.mixer.out_proj.weight": "f5fbbed2adfcc52982304dc6c6ce81bfbddcfc0d50a0d7f8d3685a9ec0d9c644",
|
| 61 |
+
"backbone.layers.5.norm.weight": "ad44f74b2f588a6b5f106e3ed44aab1e31ee3366352869d7d7f17011236e9b60",
|
| 62 |
+
"backbone.layers.6.mixer.dt_bias": "08a8f542317c36767a974f0703ead327a5750c2a2e99bf44ff78b6660763f564",
|
| 63 |
+
"backbone.layers.6.mixer.A_log": "0bb4b3272855d54a1554309ab0e12861b80f0c555c87fff6db317748c46d1d94",
|
| 64 |
+
"backbone.layers.6.mixer.D": "005598e55cbaf612df49f91a6bf3073ab4ce625a34fafd657c12ca61f12332d1",
|
| 65 |
+
"backbone.layers.6.mixer.in_proj.weight": "508c6ceb7c454c4ed76e42414709a01a8f6a318d6ac2ba5e081eedd9a19c6f92",
|
| 66 |
+
"backbone.layers.6.mixer.conv1d.weight": "c223dade29ec079d10aa9644114e22b65e4288de0a68c3fa4ead894f065fc2ad",
|
| 67 |
+
"backbone.layers.6.mixer.conv1d.bias": "bcab2f7846e401324b5d9deacfb94eb7dfbb2e0af81c212ae6d4b5b696e93467",
|
| 68 |
+
"backbone.layers.6.mixer.norm.weight": "1c912801c88b08ee0a2fa7db0f57a9b16368da6ff49c796d8209b618473a1141",
|
| 69 |
+
"backbone.layers.6.mixer.out_proj.weight": "f96fa782e93ece71ea9777b4a9c4730d564f53e4b759e2b6f54a96c28d45a523",
|
| 70 |
+
"backbone.layers.6.norm.weight": "7f520131c6694231b0fd73c6b1dc2a7700af71abbd9837f5c3abfe00f310d423",
|
| 71 |
+
"backbone.layers.7.mixer.dt_bias": "8121037a3fb7644e5e255a153151ad5e2d8c378c65571fb8e6bbcd758538fd21",
|
| 72 |
+
"backbone.layers.7.mixer.A_log": "a12135bb157b5ea0af3ee12034386adffcec65ec397100025981285b0fc13efb",
|
| 73 |
+
"backbone.layers.7.mixer.D": "2d7fdf8f427733a7c61baf62e757550b9e036a9a055eb6fc14a31917d0d4300c",
|
| 74 |
+
"backbone.layers.7.mixer.in_proj.weight": "f464cce6c57898ca328b34ad7a7a54d16cc1122a1fde52413cc1ca324e396cee",
|
| 75 |
+
"backbone.layers.7.mixer.conv1d.weight": "de368d0203ed30e65c31f983fd693d38c67540bc7548ec15aadf7a1f9dc6b36a",
|
| 76 |
+
"backbone.layers.7.mixer.conv1d.bias": "936c9391061cdba320210737ca1ce22fd4dad9b7dc013986ae62a466762a6cd8",
|
| 77 |
+
"backbone.layers.7.mixer.norm.weight": "cbde1a4a5b92ebacdf70c419a6665b14e385e6664ef8be0431ee7bcbbb39bfc2",
|
| 78 |
+
"backbone.layers.7.mixer.out_proj.weight": "f1ec00454cce8c74ee9ef16c20a1d5f36940d3d5570863216cbccd2d33eb2535",
|
| 79 |
+
"backbone.layers.7.norm.weight": "bf6fb10ee195f462fc56f2ebdf5bc90868903e11ad112be734047f8b22840778",
|
| 80 |
+
"backbone.layers.8.mixer.dt_bias": "d09c4c2c4fb2bf1ff7f998b49b31b971106e406d3a708e8a7eba2cff0496a9a5",
|
| 81 |
+
"backbone.layers.8.mixer.A_log": "ba4517666ce89351f5468bcc2645a1ea9fa211d23b6570b7cd39a28e0430bc5b",
|
| 82 |
+
"backbone.layers.8.mixer.D": "07c8e5a49539b5d3fb17eb46648f82c1f7fdeb64fce7688e9b4ea8b025254834",
|
| 83 |
+
"backbone.layers.8.mixer.in_proj.weight": "f8b1913567adb94b4fa423ea926fbfcea2fbc5ed66d86d1de71af74eb7308d42",
|
| 84 |
+
"backbone.layers.8.mixer.conv1d.weight": "41bae34c06ae430d1f39c44e9d671e7a1de097564cd57f1fefb4a9f31a838a8f",
|
| 85 |
+
"backbone.layers.8.mixer.conv1d.bias": "8a9dcea999f18211d615924e737c2d1d5f97b71be36fa95c1ca30d68ba4b6e35",
|
| 86 |
+
"backbone.layers.8.mixer.norm.weight": "ab220986a488e55cee281f84419c114813490af4b37eb867dc455a6a4fe7205a",
|
| 87 |
+
"backbone.layers.8.mixer.out_proj.weight": "2d7015e55349a82c73e73e46208b1b82edb3c8299c422446f87df0038a861767",
|
| 88 |
+
"backbone.layers.8.norm.weight": "5f27b8fa0c87417559de141558cfc2c1c26b157e54bf882da11253f8f714960d",
|
| 89 |
+
"backbone.layers.9.mixer.dt_bias": "47f82d0956f1410de8d2e3028613d87744fff36c40e01f101499275d9a1f246c",
|
| 90 |
+
"backbone.layers.9.mixer.A_log": "d12ee15e981b77f5d6ce618e0e954264ad591d5030feeb54df7bb6ae85041a14",
|
| 91 |
+
"backbone.layers.9.mixer.D": "45b1e4c49adf0de86e746ec6dbd3df451fc49923be6d147b64c98e3f509d0886",
|
| 92 |
+
"backbone.layers.9.mixer.in_proj.weight": "fa93f8a3379ba1f0bdc8f8c1ffda49a7b75e9694162b9152342116c8d29834b9",
|
| 93 |
+
"backbone.layers.9.mixer.conv1d.weight": "2dc1137b6df8f51a8a518450ee93cc6bffce8a611691b3e66d6be38b71486391",
|
| 94 |
+
"backbone.layers.9.mixer.conv1d.bias": "4717f86a354f0c59d1e9b87d4b7585a7cea98b0bf55de0a21836ebc926ea7ba6",
|
| 95 |
+
"backbone.layers.9.mixer.norm.weight": "d5d1b32718b3dbc5b52ce69864b34f87dbefdc438c11db6c3ae77f5d7dcbe1b1",
|
| 96 |
+
"backbone.layers.9.mixer.out_proj.weight": "aab881f7ca290ef35d5ad857959191c24da119c2e2e07f2ab90391592132f4a1",
|
| 97 |
+
"backbone.layers.9.norm.weight": "c44b6d08369d2d42e241d4e82632dbb6fd7b6d3cfae96bfb40205a2effb3b2d9",
|
| 98 |
+
"backbone.layers.10.mixer.dt_bias": "d324777e66f0e6e4d99a1ef833e038402303c7a973c79b7e9eec5de728ee0994",
|
| 99 |
+
"backbone.layers.10.mixer.A_log": "549d1d9132b408b025c666aec65ba1a172b21f3fdaf7b0bc4f08e22c1c41e068",
|
| 100 |
+
"backbone.layers.10.mixer.D": "7d5dbad8b46b07489ac9840757e480904f6a0b0c333ea8c9be87fb7ce2be8369",
|
| 101 |
+
"backbone.layers.10.mixer.in_proj.weight": "3dd6f8cb4dfc3221d565ccdbea323e3d852cc0459fe18f1b0527336c14385116",
|
| 102 |
+
"backbone.layers.10.mixer.conv1d.weight": "ab3baa93bfcc166d21ecfdd6a8020435d14cc8e9ac1298a561d8978a4a1ee3ab",
|
| 103 |
+
"backbone.layers.10.mixer.conv1d.bias": "6c10a33e19ca3fffd3811eea63e1cfbfff164474e6c7cd03fea1d060c2838f68",
|
| 104 |
+
"backbone.layers.10.mixer.norm.weight": "93a7cdeafbebfa45e2bacf2c853000d821c55c481ed9e95a9edff4c1dfeaa237",
|
| 105 |
+
"backbone.layers.10.mixer.out_proj.weight": "94d94194690573cc92a15a1365128d99c20ea3b06d76a9fcf1115d4d6782e118",
|
| 106 |
+
"backbone.layers.10.norm.weight": "81d316cd8e282e0da65590159338cc18f24d79687577225b90a5d87ed493539d",
|
| 107 |
+
"backbone.layers.11.mixer.dt_bias": "398a57d4a84fad84fe793c252c36689eb7222bf2c7dae842bd769aea8d132a8d",
|
| 108 |
+
"backbone.layers.11.mixer.A_log": "a49a4097c8b0b93687d7bca54c448edfd336695887835643c937a193b7645f8e",
|
| 109 |
+
"backbone.layers.11.mixer.D": "d37062a9603ed53aaf07f227312eac2896aa86a645b0a4013cd15be935d3176b",
|
| 110 |
+
"backbone.layers.11.mixer.in_proj.weight": "18fb3c249741271dd0ac8de734e6f03cc21c0f49c8547a73592a0685ca278faf",
|
| 111 |
+
"backbone.layers.11.mixer.conv1d.weight": "ee0b2719bd2f2b62edbe785d2147b03546d6d3041b79074ccd798695f13203de",
|
| 112 |
+
"backbone.layers.11.mixer.conv1d.bias": "cea5365c08c89d985b9e355ab8cb317a1ccdb27a625d68fb45a685916d5f508a",
|
| 113 |
+
"backbone.layers.11.mixer.norm.weight": "1ef490c7f176e9579103f56505be102504700a07a3eb3cfc84cb35988d0cbd4b",
|
| 114 |
+
"backbone.layers.11.mixer.out_proj.weight": "f76828e9dc0ad2846e3f6b0040a4eb8cf20b57a9f7d728d2a208508975498b8c",
|
| 115 |
+
"backbone.layers.11.norm.weight": "2fabc03b5ba35332dde5bdab5a195ee2063b7e28e464f3c5b404f74632d1e184",
|
| 116 |
+
"backbone.layers.12.mixer.dt_bias": "8c838c8c0b2d299b3b927c826c5bd7cd94b062276337a426e327ce8d2e5d9615",
|
| 117 |
+
"backbone.layers.12.mixer.A_log": "c37e8e70f7036f00d6a6e775a55ffc34f672edce0d72c573dc703039381c8e72",
|
| 118 |
+
"backbone.layers.12.mixer.D": "2420796f486bf19ace80997a6b522feae3ff158e2eb7f971208ececf872d39aa",
|
| 119 |
+
"backbone.layers.12.mixer.in_proj.weight": "e0940d550ba016dd074fdbc3cf7c9c7b1197e703e54d1c5a2aa04c734237e9e7",
|
| 120 |
+
"backbone.layers.12.mixer.conv1d.weight": "bebbfe698927e5d6d94173d8cdc0004119b53051867e443a7434c63bbb4f5e97",
|
| 121 |
+
"backbone.layers.12.mixer.conv1d.bias": "34af93b560b686b68ca77a875da62ed0900dd584a668991b714b70b32ecc5fcc",
|
| 122 |
+
"backbone.layers.12.mixer.norm.weight": "c78f9b596a2729d871881b54081c09469c26fdd0b67468b83ecd5211ba926c00",
|
| 123 |
+
"backbone.layers.12.mixer.out_proj.weight": "5a4b382b291b2d5a8370f39de51ba7cfed99756f43a8c13c512472cb6e7a2596",
|
| 124 |
+
"backbone.layers.12.norm.weight": "18979a2dd8e980e4e4fcc04e43e8e03ad79e7e984f7f80300cf16dac1ad561de",
|
| 125 |
+
"backbone.layers.13.mixer.dt_bias": "18197d06a34ed3676a0bf74347d6d97dc6e7c35f5295f92a2b5e93b628b408f6",
|
| 126 |
+
"backbone.layers.13.mixer.A_log": "4d441f8ab4caa2078589fcf57fb5a5a28679a641ded4cfef767c632971cf9ab9",
|
| 127 |
+
"backbone.layers.13.mixer.D": "57f2216c11bd9372c4195f4629809b66288a3f656f2f37e02a955bbbb9c10544",
|
| 128 |
+
"backbone.layers.13.mixer.in_proj.weight": "c31cae1a3a7d4fdd8c426d40a529596384a5844865b1e0dec46a8da39c74c87c",
|
| 129 |
+
"backbone.layers.13.mixer.conv1d.weight": "3acf8890e7923c823748643a95d17ca047634a4c01f251939d6ec431ca925e68",
|
| 130 |
+
"backbone.layers.13.mixer.conv1d.bias": "497b7c5ac2401cca830c0178689ab7ff534a6efff0da08f47e66f834ac430d4e",
|
| 131 |
+
"backbone.layers.13.mixer.norm.weight": "84e3df57faa920d2c26e13dfae0edf677c4e887f4c1e708e0a5d18b8cdd6011e",
|
| 132 |
+
"backbone.layers.13.mixer.out_proj.weight": "79bea421c2088b1cc8a5440785978006073a7516ed04ab71424711539baac780",
|
| 133 |
+
"backbone.layers.13.norm.weight": "3d6c40cbdd1a5583b6a7cd541f7e6fa4e1c129471ed87f8735f072f01688c5c6",
|
| 134 |
+
"backbone.layers.14.mixer.dt_bias": "d7ba3283e57c1d6667dafc6a5557bc0283b6083ca738eb24bc16e7bc5ebbbfe9",
|
| 135 |
+
"backbone.layers.14.mixer.A_log": "612e1d1902aff391bac31159c8cb257d87d8b45007a5caa20fc0d0f86877affe",
|
| 136 |
+
"backbone.layers.14.mixer.D": "8608fca8fa70a7abe2a4f30750450782f8bcfc6e1c328d3fca2c446ef7ff5750",
|
| 137 |
+
"backbone.layers.14.mixer.in_proj.weight": "b81ff7915be4754101169e8ee8bf227ea3182435d6b11f60271be75045f54587",
|
| 138 |
+
"backbone.layers.14.mixer.conv1d.weight": "207d18b40a9ae29b5dd59fae18785e75f4a55546c1251dadeefcdfec5e1061ae",
|
| 139 |
+
"backbone.layers.14.mixer.conv1d.bias": "9b7a78d13c5cf9dc2bdbba42e8dfb7dcd36dccfa188608eded62fc5e4120f9c1",
|
| 140 |
+
"backbone.layers.14.mixer.norm.weight": "acf3b8e8c612b8aa54cc0131925e527de166cd6177c2178b2610d9169da4d0e8",
|
| 141 |
+
"backbone.layers.14.mixer.out_proj.weight": "e37333775c2ba6284468ab0681c1d59dca4b8faee1e579e5e9279af1a22718a3",
|
| 142 |
+
"backbone.layers.14.norm.weight": "4c5734dd29096b6224aaafe294714a1b5312503bf11faa53c524d14c2e0daa97",
|
| 143 |
+
"backbone.layers.15.mixer.dt_bias": "03d06adfb765cd1c63511a591af2b922594ca984d9d02191990aed377ab87a7f",
|
| 144 |
+
"backbone.layers.15.mixer.A_log": "5570e0a435b5edcbd1128dbdbc536a59e5a65cd2ca20d6e17a97fe2f02630172",
|
| 145 |
+
"backbone.layers.15.mixer.D": "fef0f1d04b18aacc04c7e1ddcf0effed779c6e95dbbe769ec656c35d176ca803",
|
| 146 |
+
"backbone.layers.15.mixer.in_proj.weight": "24c69914dcfca119fa43bc0b16f1292a944bad97a077097a0109c30b05600d34",
|
| 147 |
+
"backbone.layers.15.mixer.conv1d.weight": "302cea25cf918972d29affd7d73b0fca03dd237e736f7da89a066aecb90d8557",
|
| 148 |
+
"backbone.layers.15.mixer.conv1d.bias": "d89d957103a8bc734d9b4f1c092b25f1560e785f1919db467c222f9353e2750c",
|
| 149 |
+
"backbone.layers.15.mixer.norm.weight": "26ebe92cc7717d77c45c7e0cb8cbb5fab5ea878ed77d0736842cfcb817056187",
|
| 150 |
+
"backbone.layers.15.mixer.out_proj.weight": "78aa94783b559734a32b1d7b6a5009aa099ff6e3222d8168d74f2f4fab92b785",
|
| 151 |
+
"backbone.layers.15.norm.weight": "f5776f81ddb0776e09eba6807b30abccedbdbed440a16de704005b0768d48c85",
|
| 152 |
+
"backbone.layers.16.mixer.dt_bias": "56bbfd60436a32ec7d72a51ea63f95e86b0ba57016422a180dc28e8edaba05d0",
|
| 153 |
+
"backbone.layers.16.mixer.A_log": "28dc31e731f08a54d3863b59a5771ac77087b99d7f95f9a0060b8bb826b6892c",
|
| 154 |
+
"backbone.layers.16.mixer.D": "e8083fd430522a2bacf26ce7fe30348102c101e811927f4543db403724f7bd94",
|
| 155 |
+
"backbone.layers.16.mixer.in_proj.weight": "d0d5c0d0e0816a952472b2f64cd72af0c0e1ba9a3ce335869d2aa8227b4e0314",
|
| 156 |
+
"backbone.layers.16.mixer.conv1d.weight": "c5e1703d843e11393887d7031286f517bbbd31794018ec275d761470b494600f",
|
| 157 |
+
"backbone.layers.16.mixer.conv1d.bias": "78af0ceaf23cb00b0536a85962eac2db567fbb670d70191d2d182fa53ac1688c",
|
| 158 |
+
"backbone.layers.16.mixer.norm.weight": "aec366d8c69394679d6cd4e1c72b72b98213c5c63c839d6ba3e0d4ab86d2c644",
|
| 159 |
+
"backbone.layers.16.mixer.out_proj.weight": "107be4fc5b24072d9df203f3960c0b1bb06249f9ad84e324e0cf2b68576074cd",
|
| 160 |
+
"backbone.layers.16.norm.weight": "d6910544ccd022e809030c347fddd6153bfccdc0e70a9884f8b7660e8ced8ffa",
|
| 161 |
+
"backbone.layers.17.mixer.dt_bias": "da30edbdcb7abd589e0f5be03acb09f8e313d50cf6e2895ea510e1bbd866e10b",
|
| 162 |
+
"backbone.layers.17.mixer.A_log": "19fb22c89739c883c9b4f8194b23f158e73ed8977319e97490934013d78992fd",
|
| 163 |
+
"backbone.layers.17.mixer.D": "d3c6e51127a62f41bdcc8c25feab179e952572a107647d25a9119c70a23de797",
|
| 164 |
+
"backbone.layers.17.mixer.in_proj.weight": "3659af936667d578972dc6a1a51b5cd98abc8faefe261dd18874d3b1da2637cc",
|
| 165 |
+
"backbone.layers.17.mixer.conv1d.weight": "2d8f6e155e96d9180819637800d896672ddbc91f82c158eab2fb2d5082ec0b8e",
|
| 166 |
+
"backbone.layers.17.mixer.conv1d.bias": "9b1677920accdb43d90bada8e0506cf83c8a18eb26f398710e6f5bae2cf4b9c6",
|
| 167 |
+
"backbone.layers.17.mixer.norm.weight": "2961d4c1a59b3497f57d0249d5fae9aa8aa79bfb14747711102a44334cb64162",
|
| 168 |
+
"backbone.layers.17.mixer.out_proj.weight": "7ad02096239919bbf69311148bd50b89d542098f2a319ab88f68c2fb5d5153f0",
|
| 169 |
+
"backbone.layers.17.norm.weight": "0dbda8b78265d62ed2101015cceac59fa93d7ff5a1d1a9a255b063eb5fb46b71",
|
| 170 |
+
"backbone.layers.18.mixer.dt_bias": "caf798ab038bbe6db8ecabd667b6b8f9832f81e578660c9de7e5645ad0d4bc5d",
|
| 171 |
+
"backbone.layers.18.mixer.A_log": "5c35bd6083c3feff4bead9e2dc49ba8982161ad7e0d97d8ae7de42647c69688d",
|
| 172 |
+
"backbone.layers.18.mixer.D": "abb5386c7f4d69371ac5c5ca3102958b09f9f8ae1a3a9fa0aa30c6265092cdb8",
|
| 173 |
+
"backbone.layers.18.mixer.in_proj.weight": "1cd1478278704dea33ade789993cbd5877016a2309c102765412ea756050034e",
|
| 174 |
+
"backbone.layers.18.mixer.conv1d.weight": "e995e5de14b4ee1032fff9dcb5dc8b1237d99c2de0ce5faff9e8f139d674df97",
|
| 175 |
+
"backbone.layers.18.mixer.conv1d.bias": "0d2ffc81c63e92124c27558f04bc3000ee08cd253e81932d87d9cd45e002aded",
|
| 176 |
+
"backbone.layers.18.mixer.norm.weight": "eb30150bd7cba908da56f4f0fd6199f06a5941645a067c0f9fe64da2cfb21ffc",
|
| 177 |
+
"backbone.layers.18.mixer.out_proj.weight": "203ee13389f584b504a4f8cc9b4b0af1d7f9039d836cf228c2cd568c3e3fe905",
|
| 178 |
+
"backbone.layers.18.norm.weight": "995ca90d9a99a38c557cecf8ae6262cdbd785a8d2dc080c5676711203a5c789c",
|
| 179 |
+
"backbone.layers.19.mixer.dt_bias": "fafee561285a386364dd8bd49ca8b045c843e15d969b711630a193a3c86fe037",
|
| 180 |
+
"backbone.layers.19.mixer.A_log": "10865c8e3bb5785b848efb7a887909e5dd5e008f8a275d8655fda2b7edb84000",
|
| 181 |
+
"backbone.layers.19.mixer.D": "f4c1bf7c3dd76270406c68c1ccc17c30a699106d618dcd40b2b8078dcaed76b7",
|
| 182 |
+
"backbone.layers.19.mixer.in_proj.weight": "a0cddaa43c0928fafc23db3cc5ef1e14729b2647328816a8ba56cac14b9d0100",
|
| 183 |
+
"backbone.layers.19.mixer.conv1d.weight": "e10180afd81ce19d7d22072984fcf64e730dfe3bb477ada00205d5846e33ac44",
|
| 184 |
+
"backbone.layers.19.mixer.conv1d.bias": "072c36c348e2c3d96c4c6c172f3d0b1fd55c75876c75a9e18bcc4d113aece5dd",
|
| 185 |
+
"backbone.layers.19.mixer.norm.weight": "1f72d6a7157d56fc4ae4524c131f3f6874a7855ba37eea2eb71dc285b64fb2c4",
|
| 186 |
+
"backbone.layers.19.mixer.out_proj.weight": "f63599c730f64cf3536c3d6996181403f82861ab542a07c6e1cd4e31fca28bec",
|
| 187 |
+
"backbone.layers.19.norm.weight": "1d073b4d344e004814677ed176b3a9b62a5126b78e7c9f9fc37f0133bda6607c",
|
| 188 |
+
"backbone.layers.20.mixer.dt_bias": "26134afcf405687849544834016e542c1c58a6d5adfbc8240aac49638608a861",
|
| 189 |
+
"backbone.layers.20.mixer.A_log": "b0b072bf883570b33eba9fa410a2d2cbe622db9ab7a6bc563c504c5d716352e5",
|
| 190 |
+
"backbone.layers.20.mixer.D": "a238032bdaada9b645f91e1c63b5e0b3db1eab1ad64486f99e7d8fe7806cde1c",
|
| 191 |
+
"backbone.layers.20.mixer.in_proj.weight": "d6fc646592fdbfccbda2ef753282f80d89bef7cc092e112fd07894804b8f638f",
|
| 192 |
+
"backbone.layers.20.mixer.conv1d.weight": "2dd7a3dd802dc375cf3d5ae9ff7ec4dcb5941ae7fd25557ac1d6e2d646511be7",
|
| 193 |
+
"backbone.layers.20.mixer.conv1d.bias": "400d3fa9aa1336b93699fe8af01d48f8f706f3796c265408ecf9a15a559bb69a",
|
| 194 |
+
"backbone.layers.20.mixer.norm.weight": "dad789985cfc12c5b2064e81e9bbb2710553ba912b5cb87b8ff3a08f1c87d3a8",
|
| 195 |
+
"backbone.layers.20.mixer.out_proj.weight": "5ee066897993c7f296109560b88efe43ac239eaa17e4e676d9e502b245e0ef98",
|
| 196 |
+
"backbone.layers.20.norm.weight": "5599a9e958c94b2ed441e2dd1d404df9db1c0c169471849b7f7f51eb133bef16",
|
| 197 |
+
"backbone.layers.21.mixer.dt_bias": "b20c56a2c64efcff68b3e93d6b8514034ca5d5546856071a9398095d275b8b4c",
|
| 198 |
+
"backbone.layers.21.mixer.A_log": "5239beb13dbce70934dc6919697a07f3b6dd77f90d4a2583129e2a779d20f3f5",
|
| 199 |
+
"backbone.layers.21.mixer.D": "2851559df57450185bae0f482b9be52e42265df4d1c4c88e16357a59f6fa3e15",
|
| 200 |
+
"backbone.layers.21.mixer.in_proj.weight": "754db329600100dc31f59f8bb1a9c3ce7783429d780040537151af05a3737b10",
|
| 201 |
+
"backbone.layers.21.mixer.conv1d.weight": "bcf01d6be9d30e54c1db3d97970864950267452d52498b5bb16eea5e0a7b08be",
|
| 202 |
+
"backbone.layers.21.mixer.conv1d.bias": "5fcf3613c6df9a4f2d99129a88a6dbd88d807ba3f7631fd1287fa0856b57a993",
|
| 203 |
+
"backbone.layers.21.mixer.norm.weight": "63cb1c9f7c34ec2611e2fef89b755637d42985bf9874e4981b96a5b9330b5851",
|
| 204 |
+
"backbone.layers.21.mixer.out_proj.weight": "ab44bc8dde985df2a32c49dff37ebd9fcfc068aa6e47081ce3c1c3d68b4deb22",
|
| 205 |
+
"backbone.layers.21.norm.weight": "09eac8ee9f6cd1828078d4d5e700ffda829e6a2bbdfe45f7a381cf2dca5bdd42",
|
| 206 |
+
"backbone.layers.22.mixer.dt_bias": "1629d1e393ad3c37f1da0f068d94e157e63c37eb7548b3e44125308b3a7ccd62",
|
| 207 |
+
"backbone.layers.22.mixer.A_log": "845e54a627c0e0a25e0ffe725aba0113e43b9ed454fd09faa0542f379d06ebe7",
|
| 208 |
+
"backbone.layers.22.mixer.D": "560c88a129a0fc7887f022437ff18351d4647a800bffe402e39633331571a145",
|
| 209 |
+
"backbone.layers.22.mixer.in_proj.weight": "77249cfea935e4c0701b488f97b8779975e2322a8a7252457a4712d9c2c77149",
|
| 210 |
+
"backbone.layers.22.mixer.conv1d.weight": "cec6dfc0500b9cbc4bf6c83ffe60b0d916876a190ae13e623cf72947a5104935",
|
| 211 |
+
"backbone.layers.22.mixer.conv1d.bias": "25028a667430f08788bcf140e2ff926e903cd621ccfc7ace87bcf0b65765b101",
|
| 212 |
+
"backbone.layers.22.mixer.norm.weight": "6bc7d193ad54fe2ee961e7fbda0898b083e56a08286d433e8ea83c9791b8f2d5",
|
| 213 |
+
"backbone.layers.22.mixer.out_proj.weight": "99aebb52c12941ceeacdce77532486a9ca48752ff23dc5fdd5c41f601deef59a",
|
| 214 |
+
"backbone.layers.22.norm.weight": "6f74012a93ec2813c4b9542f7082b1fcf1591331aa333412f2718430f5217c1e",
|
| 215 |
+
"backbone.layers.23.mixer.dt_bias": "62460522d608fcb2491b08c60a786297e7f35abf22250529d8a17417b73a075a",
|
| 216 |
+
"backbone.layers.23.mixer.A_log": "256b45c7be3a4c714ebd96c5bd329c7f8ecc56b0826e2ce974d839e266d47ef3",
|
| 217 |
+
"backbone.layers.23.mixer.D": "ea60d8fd560951d90850a68d2e5d7f02fc49c12841c7bbd0d9411e3696e4afa1",
|
| 218 |
+
"backbone.layers.23.mixer.in_proj.weight": "db9433408734c18ffd8801cff175c5a2d22461672a3b374658095a3e3946804e",
|
| 219 |
+
"backbone.layers.23.mixer.conv1d.weight": "9e799b05ab5bb8001a337a71abf442faa52aba1ef1d1fa61f6fb588a030113e7",
|
| 220 |
+
"backbone.layers.23.mixer.conv1d.bias": "ca5342e86d03bec00d41086f66b480f4cc5e2fe509293acb59ffc2289becf02c",
|
| 221 |
+
"backbone.layers.23.mixer.norm.weight": "26b3e10dd4898678e40eaa535f4f200ae096a7e566e24d3cbbf7b527b2f67e5e",
|
| 222 |
+
"backbone.layers.23.mixer.out_proj.weight": "a04b8696d8955f16a7cc5874be48c613ed25fc00c47a4b652e1defe0743b4f6e",
|
| 223 |
+
"backbone.layers.23.norm.weight": "c817ced803c1d381cb9c0eb3322e5cf18a3701772b00fee4997f8dab9c2cc6cc",
|
| 224 |
+
"backbone.layers.24.mixer.dt_bias": "39582e76d30b090ba934dfb12c1e9352eeb31e98ff91cbebc7adf05866e9ea0d",
|
| 225 |
+
"backbone.layers.24.mixer.A_log": "4c8064143205ada0e96b336941ac55f7197848aa7918b2e840fc970cfcd2bb7a",
|
| 226 |
+
"backbone.layers.24.mixer.D": "c277c3b967a492735fbcbcec26e532f4b6607282f664c10cd72bc859206219c1",
|
| 227 |
+
"backbone.layers.24.mixer.in_proj.weight": "ab1fcb786bbaec6e77a6eec2ed0a04a5e2e3a5bef5920db66badbbd314e41d88",
|
| 228 |
+
"backbone.layers.24.mixer.conv1d.weight": "7a2d3eeaf6a70a17b79ae7aaede083dd03dee250a00399009ea451cc42fe7d03",
|
| 229 |
+
"backbone.layers.24.mixer.conv1d.bias": "1ffbd7d1b80471daf04af1993702bd8dad51936492fbc9bfb45b997fa59e1171",
|
| 230 |
+
"backbone.layers.24.mixer.norm.weight": "defd9fddb23b67208233e7b657e55960cba45003d5636e7a1ddca64e64de00a4",
|
| 231 |
+
"backbone.layers.24.mixer.out_proj.weight": "b768be602d22616e2e3ab8220f2a5f626668bc31400147b6ba4326aa15c15f43",
|
| 232 |
+
"backbone.layers.24.norm.weight": "5a85ce8fc16972fd0219563bc5465d1bbd2d238b8f49d4f7e32fdb7ee287d5b5",
|
| 233 |
+
"backbone.layers.25.mixer.dt_bias": "33269b569cd020ff52d7a8a4b57f59259e6f20d6ff929caf74006f2f79ff7a70",
|
| 234 |
+
"backbone.layers.25.mixer.A_log": "634260bcad92801b7d10fcc8fa0fd32d7ba6303155adf2521328e6f665cc2b51",
|
| 235 |
+
"backbone.layers.25.mixer.D": "920a89b75261ef527ce072f36e935475f37a3ead7cac42c33e5a38dcf5318119",
|
| 236 |
+
"backbone.layers.25.mixer.in_proj.weight": "6b5a0aaa04a0b4b290edb3dd6525794132be22a51655a203ee86aef2ffd88cbb",
|
| 237 |
+
"backbone.layers.25.mixer.conv1d.weight": "c417ed3d8092ca914285332d84b0c204ada28b804677dda6463472e6c59a8ab1",
|
| 238 |
+
"backbone.layers.25.mixer.conv1d.bias": "ba65e8274dd4aaf833500e40f74e273ecf86d2edb73bc7376e0311b3bf7d73fc",
|
| 239 |
+
"backbone.layers.25.mixer.norm.weight": "c8635dde354623fbfc1f101bd23dd06aea895dcbe0638c423a78fcc0928d904b",
|
| 240 |
+
"backbone.layers.25.mixer.out_proj.weight": "075206e5bad70824927f9bf57baa200a04cf55361e79999eba098cde0a01541a",
|
| 241 |
+
"backbone.layers.25.norm.weight": "428fe2cd8377e08f72255b42a74501984e919bdee4907f9d7457b5c4becc8547",
|
| 242 |
+
"backbone.layers.26.mixer.dt_bias": "50f2caf9766bd297c1f9f16b96b3990a628752bc545e16dd1c84042633921bfe",
|
| 243 |
+
"backbone.layers.26.mixer.A_log": "09c4b20ab9cb3e3d9e2c7fd24fd8cb66352eb085bc1a83c89affc955b59c54f3",
|
| 244 |
+
"backbone.layers.26.mixer.D": "220c27d710e3cb3e3621d6511cf85a26e1623fe3e178189570c6c6c37144652b",
|
| 245 |
+
"backbone.layers.26.mixer.in_proj.weight": "c02b085288e5acdac7175fad5d700f6ec3df21c501be5a6626bf819613d05d29",
|
| 246 |
+
"backbone.layers.26.mixer.conv1d.weight": "ca1ed45a5ddbaa3625d5ac91eba345c4af8feae39a92481574eca76d69a15990",
|
| 247 |
+
"backbone.layers.26.mixer.conv1d.bias": "1f0eeb3761cf264e20b57e2823a807a337ed335783767d81b7d43e305cac3ebd",
|
| 248 |
+
"backbone.layers.26.mixer.norm.weight": "5895f01973b6ac66ca6dc1d4239e9cbb56dfe8a9b052959ca904456b1c28be67",
|
| 249 |
+
"backbone.layers.26.mixer.out_proj.weight": "c8b85be9fc55681359abff1f56847e4c88afe3ae342129fdefd08fb0ae602266",
|
| 250 |
+
"backbone.layers.26.norm.weight": "dd50e679bda5b158354eca362245dad141a3c21ae5741698f588a587fb12c80d",
|
| 251 |
+
"backbone.layers.27.mixer.dt_bias": "fea735358e84b4fa3d1724dd2a6e03006ac8a27e484d1804b24a6aef42ffab09",
|
| 252 |
+
"backbone.layers.27.mixer.A_log": "a3a4ea97dd46282a53ce070e5db10954d794a466331889afaf52c411d3a06942",
|
| 253 |
+
"backbone.layers.27.mixer.D": "505141a60590d7933d9ab19de1c35a6ac0e9c953f5126a9b4e2109e0dd510c94",
|
| 254 |
+
"backbone.layers.27.mixer.in_proj.weight": "c1f930bd91f13917c354ee5c23aeb0108ee85527f8fa280352afb1726308ae1c",
|
| 255 |
+
"backbone.layers.27.mixer.conv1d.weight": "fb9fccc4a87f7096335d33581c48715b9274c6ea6087d6843104253e4fd32eef",
|
| 256 |
+
"backbone.layers.27.mixer.conv1d.bias": "c5ab9a61189144f5fa46cb277a73157ff2c46c6799bab8906da7e4a6e0f1df6f",
|
| 257 |
+
"backbone.layers.27.mixer.norm.weight": "50a06f98f38f67642de181721c9e55f55349fd1688fdcfa32366c4ebb877bdf6",
|
| 258 |
+
"backbone.layers.27.mixer.out_proj.weight": "77405e0c2a693c316b54de060ec67f34fe7010bcd975ee56cc76e8033909b4e6",
|
| 259 |
+
"backbone.layers.27.norm.weight": "f52a1de1e8c66b9ec8019b87d0c65d58a49f7b0c72f7ac5a76a2c1be033409d2",
|
| 260 |
+
"backbone.layers.28.mixer.dt_bias": "58ca53e6860d866c746ce73692e7ba63c1cb4a784422fa8e177644a901c7f77e",
|
| 261 |
+
"backbone.layers.28.mixer.A_log": "a75d221bb3297d1041bb5589d1106b4bbdfec488e34255de0e1cd03f20290bc7",
|
| 262 |
+
"backbone.layers.28.mixer.D": "f080d472359a57018234ee32dc9e99d9f4181fa59a88d1b7e22a3ed313593dac",
|
| 263 |
+
"backbone.layers.28.mixer.in_proj.weight": "a0416025c2f5b996b3ecf7f903020e84ac57cd7e6d8ec6dcb8df183ff96dae4f",
|
| 264 |
+
"backbone.layers.28.mixer.conv1d.weight": "f0c9a06697227bb14f3915a6350dd99914317e1a61ba19644c1b678da57f0296",
|
| 265 |
+
"backbone.layers.28.mixer.conv1d.bias": "72d1f45edd6593d5f99049026ecf2741ba6df2efe71e078e10a93d4668a84e81",
|
| 266 |
+
"backbone.layers.28.mixer.norm.weight": "0aedafd4438f82aae40dceb70dc90eb793717a9a3003926877312d043ab67739",
|
| 267 |
+
"backbone.layers.28.mixer.out_proj.weight": "c374e8b339c6801d05361dde2aa644484bec691cc353dbb2203b69adf2f4d80e",
|
| 268 |
+
"backbone.layers.28.norm.weight": "3e885238ffc01b721de1579a8d0e7fb624779ea1f8281f3188265de4913e0074",
|
| 269 |
+
"backbone.layers.29.mixer.dt_bias": "af38750da64c057b7a71009edd4eb40a3af42162b6c7efac9eeaad380d430357",
|
| 270 |
+
"backbone.layers.29.mixer.A_log": "9c6b2a26a52931a7226d3e1d94ae1d7d6941f25f5d20cac02978f7d7903a4b32",
|
| 271 |
+
"backbone.layers.29.mixer.D": "34ec92f2f0417360ad2bcec676346ef5c903905e71dc91f21d5f7dd3a412217b",
|
| 272 |
+
"backbone.layers.29.mixer.in_proj.weight": "88cb32df10e8c13ae6a2efa9e2703772c0b3eb8c592e0255aac52dad6ba3135a",
|
| 273 |
+
"backbone.layers.29.mixer.conv1d.weight": "1b49521951ccecdb84c437b260b352f4d9002bf9f83b33eae8071243814c9820",
|
| 274 |
+
"backbone.layers.29.mixer.conv1d.bias": "b1a093af7c4f5b43e3b4168ab301e8fd4f8eb4945b1fd18be55d320253c57958",
|
| 275 |
+
"backbone.layers.29.mixer.norm.weight": "d08ffea1f5a5e6f7e677687923490e5a44d612f324519d1d401026a70c42dca9",
|
| 276 |
+
"backbone.layers.29.mixer.out_proj.weight": "f0f3548d3068c0623972762c006f5833edea5ec85e9b4f1a4267f797f1a9f76a",
|
| 277 |
+
"backbone.layers.29.norm.weight": "9a2aa638bc4c696d19960f272c53c465af7e1aefcbed57ec73910c9bcebe0650",
|
| 278 |
+
"backbone.layers.30.mixer.dt_bias": "60af789d442f0a8809967c8dc2f804aabdef826126615deda6e174c9e4d4ddb6",
|
| 279 |
+
"backbone.layers.30.mixer.A_log": "be73553b772adc0c015324fc0bfc2ce358ca250c18cdaa885e351905ace5936c",
|
| 280 |
+
"backbone.layers.30.mixer.D": "2cc12741fa7e45dc342c0fe275669dd06525ba8f38b4b9c9fc88569bf8dc51f8",
|
| 281 |
+
"backbone.layers.30.mixer.in_proj.weight": "2a1a97ac192b10b0da9769232b7c946914a6c56665b299a57f6e3cffa1de6526",
|
| 282 |
+
"backbone.layers.30.mixer.conv1d.weight": "afa0c6eaee3d7b8631e6b3b0da76a4a4735f1077409c0da0bb95a003717f7761",
|
| 283 |
+
"backbone.layers.30.mixer.conv1d.bias": "3f0d8d9f7dadf6957251a1032c26a3124e84a9994d2303581a270f7009eb51ac",
|
| 284 |
+
"backbone.layers.30.mixer.norm.weight": "9e2ab3e210498b2e112d80a0047d41080e13929a5a222b31c9f1883d6a558d72",
|
| 285 |
+
"backbone.layers.30.mixer.out_proj.weight": "268da84a7af1f1539b884dc98da08580cbd66b8dbebe8caaecb734eb72cd9fe7",
|
| 286 |
+
"backbone.layers.30.norm.weight": "77dfff6725bf5fb1bdf61246549174ad9dc8cada01e5ae5b236cf47b32289b83",
|
| 287 |
+
"backbone.layers.31.mixer.dt_bias": "7d5c2871ec08174f6b13748229c1da78b992e7b8cacf72b0eebf0e0f6127e3be",
|
| 288 |
+
"backbone.layers.31.mixer.A_log": "9b00d81564617e621f95aa7f3d67a1cd534632d66fe99da77f529b00635292cc",
|
| 289 |
+
"backbone.layers.31.mixer.D": "d377cc4a701f89da641a04033892ced50d7ec6f2edb2f341ade6a926e704daa1",
|
| 290 |
+
"backbone.layers.31.mixer.in_proj.weight": "b8dd62a43638f14235aa8fee8f5f2ab830874a5be8881c8fed30c3931b5c2725",
|
| 291 |
+
"backbone.layers.31.mixer.conv1d.weight": "72aced7ecb6f3308ae1d4d2bc15ae25b38b5d01dbfcea8119d0f275f27f89c0c",
|
| 292 |
+
"backbone.layers.31.mixer.conv1d.bias": "7528dc3029445c57ba5d667b3e777b0546256f6904268b0e5189cc8ab0a630e8",
|
| 293 |
+
"backbone.layers.31.mixer.norm.weight": "ddd20046c075ded99e1a929bd20f0c8a5487c39f3a4fd14fb9d96d67762987f0",
|
| 294 |
+
"backbone.layers.31.mixer.out_proj.weight": "36cad1aef6d8b73293d3c2c4a2516fa44162d66af435093beb2280b651084f5e",
|
| 295 |
+
"backbone.layers.31.norm.weight": "91021b027b364e8c80684278e5d8ae72dd4ce447f9a3b32169f466b277e1d4b6",
|
| 296 |
+
"backbone.layers.32.mixer.dt_bias": "1291bf7326684dfeef78b2fe3a1c2320f2b22fe6592cff6dbe716d688811fe8b",
|
| 297 |
+
"backbone.layers.32.mixer.A_log": "d3ee4375ad337d0cc017c59e93c1a275a5649770276cefcf9058a43697359af9",
|
| 298 |
+
"backbone.layers.32.mixer.D": "9b91843c8d97e58473eb99bc1fd10540a3ba3440d4d79a82577b46579e6c6c04",
|
| 299 |
+
"backbone.layers.32.mixer.in_proj.weight": "ea9b2caa12a3512fcab9eb8cf192fef052e740ffb2960cc0121d176f03361e4a",
|
| 300 |
+
"backbone.layers.32.mixer.conv1d.weight": "9677ef22cc6d7c9ad6379350d4e6d4f37728b0336b1dc3ecf9c7c6edb7b0a22c",
|
| 301 |
+
"backbone.layers.32.mixer.conv1d.bias": "db603f363889d727bc5b14cd0d9a132f63c547f33b9e2ca874a186f1cd519e50",
|
| 302 |
+
"backbone.layers.32.mixer.norm.weight": "59128fc8319125515a5e3d7a641a875a1790e78eb3572ca6805b51c4bee08fc3",
|
| 303 |
+
"backbone.layers.32.mixer.out_proj.weight": "9ad7cfeff241f7b524148216f7243017f24a3b49e40f9b79d8a0e1c059b86aeb",
|
| 304 |
+
"backbone.layers.32.norm.weight": "85ee33d860c5d68a96301e435c588a6f326169d9b6ff81e49a17d2a95fd96455",
|
| 305 |
+
"backbone.layers.33.mixer.dt_bias": "671034faf837a3a064b45420ab7b2195bc42fbc3c3ca1f710af4fc531e3ecb1f",
|
| 306 |
+
"backbone.layers.33.mixer.A_log": "e3ec48e2b880768adc1527050e6bc556ab22fdfec36c494f855a4d6b100ee5ea",
|
| 307 |
+
"backbone.layers.33.mixer.D": "586b436e12f5fb752878435d9f470fd002d9a0e2fb3d65409402b4eed401b5d5",
|
| 308 |
+
"backbone.layers.33.mixer.in_proj.weight": "a61c84b8915c6fbfdb31b67633ea6135ae544c41132c80163787e39e1c27527c",
|
| 309 |
+
"backbone.layers.33.mixer.conv1d.weight": "87c254c6107ab19ae344bc8a5ffcc457dd7032986ae4dde7718629081d6da398",
|
| 310 |
+
"backbone.layers.33.mixer.conv1d.bias": "38a580623b2f6f19eed16867388bc37aadfc02387aec79aeb9c71505f9088e51",
|
| 311 |
+
"backbone.layers.33.mixer.norm.weight": "35846c570f491ec0cb8f86e46decb88a0734dd431cd51419c03031767abf34f5",
|
| 312 |
+
"backbone.layers.33.mixer.out_proj.weight": "f7c3d4b77036a460966050a8ee9a2d8fb4f709a7a9d2dff1d6574065ff3e376c",
|
| 313 |
+
"backbone.layers.33.norm.weight": "b553ea94ea3b619e0c0a5c605f5fc959329ba42fca542b60926ae48e326b6fe1",
|
| 314 |
+
"backbone.layers.34.mixer.dt_bias": "35cd68aeead431973ef70bdbf0e1124941455a7de3564290fc1079aaa51210b9",
|
| 315 |
+
"backbone.layers.34.mixer.A_log": "2ba267a4fc86aefb6c5f6353d571eaee07489767f7391d41b61a6550786c835e",
|
| 316 |
+
"backbone.layers.34.mixer.D": "0d7f5f91c5d2e7b54508698836425801c8c120236cc34499e437f88817d80d5c",
|
| 317 |
+
"backbone.layers.34.mixer.in_proj.weight": "3072cd7efefe9d8aec184a0994a876e4db0609eb44aaae80f76b281e2b4315cf",
|
| 318 |
+
"backbone.layers.34.mixer.conv1d.weight": "76538a7b0362f623d67f30f6bd5b4da5911f9536ae4d605983e536d890b49175",
|
| 319 |
+
"backbone.layers.34.mixer.conv1d.bias": "05b25f96883dbc27fd1baab476435d88ef21f0466177a5971e9873bf1e2aac0c",
|
| 320 |
+
"backbone.layers.34.mixer.norm.weight": "3fa1221d75daa25405a0e88bcd2e1c56f56da5d401a0d924bd765df43ad09238",
|
| 321 |
+
"backbone.layers.34.mixer.out_proj.weight": "7b68731e3e15af850c49b1b1b7af5b8ebbba68493df24b1d1a23bfee0953e2af",
|
| 322 |
+
"backbone.layers.34.norm.weight": "d85bb56c4bbf9d032530af16540b5e0730cec734d1df58893e16a5025bb23775",
|
| 323 |
+
"backbone.layers.35.mixer.dt_bias": "4e271173698295e02fe3f47e43faf1e05338db38ac697bc891f0e61eabf1f159",
|
| 324 |
+
"backbone.layers.35.mixer.A_log": "97e5d6bfebbba2843243da0471a042b4ef92b3c55beba5272f2982c1152483a5",
|
| 325 |
+
"backbone.layers.35.mixer.D": "0caeea446d35bfe9dbf5e9b26a84a5cf6c5f1ce1445ec58af1ba43e9ada65324",
|
| 326 |
+
"backbone.layers.35.mixer.in_proj.weight": "a8bd1db9d7f44a6f248686a7d2956586d60f70a0e5b96d542428ddad29c8be65",
|
| 327 |
+
"backbone.layers.35.mixer.conv1d.weight": "2a55721dba0ea86203c2f4bbdcc2cc6823a0c625733620e0c0c60e7a12a09fe3",
|
| 328 |
+
"backbone.layers.35.mixer.conv1d.bias": "b277e9c1d7de7d7fc88447a26801203743700a6a4901310b298ed1817faf4ff2",
|
| 329 |
+
"backbone.layers.35.mixer.norm.weight": "b349e1cfde0daf0a0489e5a6a1493488dca663d85f59a9b0c444ef18f1d7cf61",
|
| 330 |
+
"backbone.layers.35.mixer.out_proj.weight": "28b65371e175398ba55bd82a646bd1e66af0097e9397c5fee1b306569be96203",
|
| 331 |
+
"backbone.layers.35.norm.weight": "02d45b99deb96f5b2ba06fb445e676ab517ae9de09e3ab073a89c300d1d68d89",
|
| 332 |
+
"backbone.layers.36.mixer.dt_bias": "39d4835ea0cdc827b0968deafc0c0d45f524819048cffa165179614d1b50b8e4",
|
| 333 |
+
"backbone.layers.36.mixer.A_log": "d05e16e2f76329bc1b64c70736b6cb0476d64c74933bb50798248b880683dc29",
|
| 334 |
+
"backbone.layers.36.mixer.D": "c6143b54d67d933de4d0afcaa98681230cc5132666655412d9359ff27e35ae64",
|
| 335 |
+
"backbone.layers.36.mixer.in_proj.weight": "7885b302a14d51e58fe625d0e91e5192e39e0704b6f2b3116ecb3dd197346aae",
|
| 336 |
+
"backbone.layers.36.mixer.conv1d.weight": "0b7402d2a4922e5c9fbae8fce224580d40fbbc5927b0ac1f646a2e5e5f7b40b4",
|
| 337 |
+
"backbone.layers.36.mixer.conv1d.bias": "5a3bb5587f05a1904f12779f363090d45b21cd6ca94064d5688f46b9520a4316",
|
| 338 |
+
"backbone.layers.36.mixer.norm.weight": "7a91711f5ddf6d7e6eeef1cbae70d6e51df9f26dd8e3681180c456df8249d1c2",
|
| 339 |
+
"backbone.layers.36.mixer.out_proj.weight": "9a6176237b6907d4f7908cca27f399bf82fed7a5cf091d560cade1dc79f8198e",
|
| 340 |
+
"backbone.layers.36.norm.weight": "5b0463dab176c1aea4336ffb6e748c23c104f5f6636d0fd1ec2d28d12dce1875",
|
| 341 |
+
"backbone.layers.37.mixer.dt_bias": "7fa97f465f46d7d49c6e250dc670484f0540ffc579589d7a7c82835dba869191",
|
| 342 |
+
"backbone.layers.37.mixer.A_log": "ca5d2d0168cb1a01d76d27ca191a6adf183817c190970a1a33779da8848f5668",
|
| 343 |
+
"backbone.layers.37.mixer.D": "d0a32750ff9b5e827e137f239acdbd8d28ecde58dda42ab9555ba390962ff626",
|
| 344 |
+
"backbone.layers.37.mixer.in_proj.weight": "ba165ec5131d5052415dd967c5a995360d14d024728371d6e2fdffe692959c41",
|
| 345 |
+
"backbone.layers.37.mixer.conv1d.weight": "19345ca24b3930cf1001d8784b4ab119048d15d04b02ba9dcc645653a809ae0d",
|
| 346 |
+
"backbone.layers.37.mixer.conv1d.bias": "17d828fcec482e4c5887d8eb3fe1459333b45cdd85cc88ce52cc97606c33e71a",
|
| 347 |
+
"backbone.layers.37.mixer.norm.weight": "cc1413622366432ad32d0f312abdb6635487b213144dfe7bc16f52ea57e7fd9f",
|
| 348 |
+
"backbone.layers.37.mixer.out_proj.weight": "426f5a0c340be8bc13b210c98a6ab3a7a7089cec9069ac8f16ba0bdd8f8c58f2",
|
| 349 |
+
"backbone.layers.37.norm.weight": "a89b7145356590ddbbff47a18beca350004e67d3ebf99a5906d0b5078a604b07",
|
| 350 |
+
"backbone.layers.38.mixer.dt_bias": "061ccf5c8f95a1663f665cad6f4d39abc5a10cc3aa5ee489da261fe361a9176d",
|
| 351 |
+
"backbone.layers.38.mixer.A_log": "5d5a09e494b52a60a0ba0337277b666eaa48e98ab976da5c5b94bfe2fe24f5ed",
|
| 352 |
+
"backbone.layers.38.mixer.D": "98ba09fe907a61a443e167e3353c7ca8ee40fc2d8e47a6d41500684870c41827",
|
| 353 |
+
"backbone.layers.38.mixer.in_proj.weight": "a2f48ac67a8d177b7cb2084c0ede48ffb47157d1d09e9fe1dca3d9a813b003b9",
|
| 354 |
+
"backbone.layers.38.mixer.conv1d.weight": "57fa0b20a5450868cda63c18dabb3b657360cfd2d27633ac0da77e79b6e93a28",
|
| 355 |
+
"backbone.layers.38.mixer.conv1d.bias": "9caf82d89fd8f0920149a77cae16a7ba72dba299f6ff06423c0a5596dae8d5fc",
|
| 356 |
+
"backbone.layers.38.mixer.norm.weight": "b049e6e4da87e179f36a61396813fd540aacb6b73ffed3282c91a44e3edcb068",
|
| 357 |
+
"backbone.layers.38.mixer.out_proj.weight": "1cb5465d3c5691ee2f1c850abf433451b3b14944391979ce1851038a1fde077c",
|
| 358 |
+
"backbone.layers.38.norm.weight": "19c5ebc7de9cee118883b267470901048adffffe3424a2793aedcf127332b204",
|
| 359 |
+
"backbone.layers.39.mixer.dt_bias": "9b380fb631b17078a624dc88c90de4d115963e9355a987fb69faa1637f72d1ec",
|
| 360 |
+
"backbone.layers.39.mixer.A_log": "18284f0941bccbdcc05602676b480fab3cd41e84b733ee3f563362c9cb76739e",
|
| 361 |
+
"backbone.layers.39.mixer.D": "84250a71646d0589a032f35c222f7b3fedaf1d472d97d3b9e2adbb11b53cd97d",
|
| 362 |
+
"backbone.layers.39.mixer.in_proj.weight": "5cb5884fa0adbfae65b01fa4188c0f20003903b2e29c9d1e344256ead52ac5b9",
|
| 363 |
+
"backbone.layers.39.mixer.conv1d.weight": "51e38a6d49af9d572fa7f08d7b65df02def1c759f33de6cf51ddc592736fca12",
|
| 364 |
+
"backbone.layers.39.mixer.conv1d.bias": "8f498e795748a1e114e52715e9a63160f66c4eace76b9d1ff67b3da4a4a8968a",
|
| 365 |
+
"backbone.layers.39.mixer.norm.weight": "16e3e991c07b727359ef69516c31853398fd5827735d0f416f7077ffe9211de7",
|
| 366 |
+
"backbone.layers.39.mixer.out_proj.weight": "a08f1f6f025730f9022b908a521dead8367c3948ede19ef7c9ba67e8fe3821b1",
|
| 367 |
+
"backbone.layers.39.norm.weight": "3327514c1bef70a69f384698244da77c203b86b2c869cbbe4fd364cecab11199",
|
| 368 |
+
"backbone.layers.40.mixer.dt_bias": "5ee07eefa9ba88c43d0c9b43280d5ec57fb2a16d8b9368f677d6fb3baf0cb60d",
|
| 369 |
+
"backbone.layers.40.mixer.A_log": "f57bf6e3709d4ec6c7fa4947380357faf5ea6d05d5914856979e564c6dea1948",
|
| 370 |
+
"backbone.layers.40.mixer.D": "affb722ec6de655ab2bbad726b2e0120e66b6531133b4ae01fb9f1edf90c0e1d",
|
| 371 |
+
"backbone.layers.40.mixer.in_proj.weight": "52578ed6992261b74a0939a87e2540fa3724d0e46e7484d9256996b93a63b049",
|
| 372 |
+
"backbone.layers.40.mixer.conv1d.weight": "ae927d8741236fca965fe2028dceedc1811c1ff166acf4d7a7ec29e26d2eb09a",
|
| 373 |
+
"backbone.layers.40.mixer.conv1d.bias": "9313d66bf10b2cb23cde8723c2665feeff7dfd53bae4aa2dc7fce805e2b09713",
|
| 374 |
+
"backbone.layers.40.mixer.norm.weight": "99059b40a20237f88a69cb9aacaaaa3dc13fd8a8fcdd88d72a71cdd80552d70e",
|
| 375 |
+
"backbone.layers.40.mixer.out_proj.weight": "4f387bf2777ab0f512e2a16aba347253bd8a1a2c63b451835ba73f89959863bf",
|
| 376 |
+
"backbone.layers.40.norm.weight": "bf30af844cea24801de8f4723be1c268d072034dee59f66d84be4aa68fed4391",
|
| 377 |
+
"backbone.layers.41.mixer.dt_bias": "47ee2a26e5060ee6a24d9880f5919c3837647d82a9a1cbb5def674e3691e642f",
|
| 378 |
+
"backbone.layers.41.mixer.A_log": "f6879fea619a2be2cfcae46cd05377528545eb6a62bf6fac70a1cf4f465cb74f",
|
| 379 |
+
"backbone.layers.41.mixer.D": "a0dd498f98283e33106675c0148d8c470de0ffb263f44c132cc3acd606930721",
|
| 380 |
+
"backbone.layers.41.mixer.in_proj.weight": "fbac14842c9ae5d08387365621aabd9565a5151212e415fdf5b16120fa4dc878",
|
| 381 |
+
"backbone.layers.41.mixer.conv1d.weight": "89152e5d2616cf6ddcacb51bd4b1f1b054f26fc08cd6624ef5b431bde424a360",
|
| 382 |
+
"backbone.layers.41.mixer.conv1d.bias": "a8993f0a36875a223544a35b9553bb64b97309df6e762c44517bfeb5386ee841",
|
| 383 |
+
"backbone.layers.41.mixer.norm.weight": "5c587ed32028a0286770adbe32849399ff1c4196593ca9e844170cf51bcc5b38",
|
| 384 |
+
"backbone.layers.41.mixer.out_proj.weight": "1e071395749592be5dd3a148f08f695578dc886eea7806b8e9c51db7d5bdfbcb",
|
| 385 |
+
"backbone.layers.41.norm.weight": "40d6612ceed7df0429106b726925c4a45ea83efe961d6140875f9543a9a6ec71",
|
| 386 |
+
"backbone.layers.42.mixer.dt_bias": "440ded94260ab51228f5094c6985f2cde93cc90e0cfa4e83bec65064482f7649",
|
| 387 |
+
"backbone.layers.42.mixer.A_log": "249fc5a91c1f15e8305b72eb61af7f4ccf5be63aa457ebdfeb38f60a8e4a3b6f",
|
| 388 |
+
"backbone.layers.42.mixer.D": "f60eddf54194b085a47c0e48782358d33d32c9858645b52bc8d010d86fd68ba9",
|
| 389 |
+
"backbone.layers.42.mixer.in_proj.weight": "1927bc38b4a91405721fbc4fad8088a1400242945ddfa15909c8a698e75474db",
|
| 390 |
+
"backbone.layers.42.mixer.conv1d.weight": "da13483bc073d89009c8e9d05cf70ca89f031e162cc5404a0ac28c1d5b540bb6",
|
| 391 |
+
"backbone.layers.42.mixer.conv1d.bias": "dbeac231e6485daff825a874cd55f8da8384ebecd337348a6ed47424ae9e41c9",
|
| 392 |
+
"backbone.layers.42.mixer.norm.weight": "6667f30a637e350682ecc8a3cf2198548ab78469911dde3b1ae4943abfe9e3aa",
|
| 393 |
+
"backbone.layers.42.mixer.out_proj.weight": "e890233c8be5855bb5b73e8caa3d57ac77f9d34aef10dea7224b3545d34ac2ca",
|
| 394 |
+
"backbone.layers.42.norm.weight": "aa9fae823de02810653c52b050070f50da962cefbdc1e9decbe39b0b6d8bb5d3",
|
| 395 |
+
"backbone.layers.43.mixer.dt_bias": "2c03c0820d17bff6a4406000d60b1649789e3c0a6967c257e38f402343db9b74",
|
| 396 |
+
"backbone.layers.43.mixer.A_log": "49bbfab5bef14c2bf238df459af7681750d714aac8cfd8ee9146dbe68a729f93",
|
| 397 |
+
"backbone.layers.43.mixer.D": "5c3ff72ef67a19d151b873fdb28f0357913e45538d309d35fad864e6d50b3b39",
|
| 398 |
+
"backbone.layers.43.mixer.in_proj.weight": "bbba0b3369fb7096a260e5a3a41b02b1f50daedffad38ac88ce155226134b1dd",
|
| 399 |
+
"backbone.layers.43.mixer.conv1d.weight": "5e0bfabdfb68b8820d4083d0425c83fc2779b31b0aaf6d649114918bc04856c1",
|
| 400 |
+
"backbone.layers.43.mixer.conv1d.bias": "7de4a7e545e348907d4f154ed408d91ab8f8420be4002e5fefe3c59cfce36303",
|
| 401 |
+
"backbone.layers.43.mixer.norm.weight": "58e1e3b0ec08587101e6763aee3d97a3a0b447177fcdd0f9545b558814e1e252",
|
| 402 |
+
"backbone.layers.43.mixer.out_proj.weight": "f74a498e47f86ee30e5e16edb0da8fc888b93572cfa182ad2cb3aaf409ab08c7",
|
| 403 |
+
"backbone.layers.43.norm.weight": "5370a0ef81e6a56b40e9fc15379fab1041af9820be02686c83502a57c453875b",
|
| 404 |
+
"backbone.layers.44.mixer.dt_bias": "27ff0f9825db835728f903b85e28332ebb74fcdb63b029568360ec9d7cfeb35d",
|
| 405 |
+
"backbone.layers.44.mixer.A_log": "155c3d7eca7a9ddb33712b26487b95b04f523b219b231d41bc6e9f6d066987f9",
|
| 406 |
+
"backbone.layers.44.mixer.D": "c68b64b06ed794383384f9002c6ba0b58275105a20cd8cc226faffb885d9a535",
|
| 407 |
+
"backbone.layers.44.mixer.in_proj.weight": "3834cbd6817c4ad7cedc9bea1f7e1787e4155dec18626ba4c91de5bf7f078a73",
|
| 408 |
+
"backbone.layers.44.mixer.conv1d.weight": "b12ad8a8bb084785018d153dd71a6172d78d422535f741ffd68d0c89ffc120cf",
|
| 409 |
+
"backbone.layers.44.mixer.conv1d.bias": "938a1bca6d7e41a5581f6ee2312775489a4e6243bdfc3e8fd09316daece438e6",
|
| 410 |
+
"backbone.layers.44.mixer.norm.weight": "21c3fb89f54f8b7fba8b6b0fbb1eff24b4a13f4f05c9a94465ef0b76654c7e0b",
|
| 411 |
+
"backbone.layers.44.mixer.out_proj.weight": "595bb17ca854d9df9f5c5f07990662323a92f252975741611d1552f205dbddc6",
|
| 412 |
+
"backbone.layers.44.norm.weight": "c5a14436d4a811481c4a2721084397fe1bf4b8a4718ea9fb60e053fe6454d3b7",
|
| 413 |
+
"backbone.layers.45.mixer.dt_bias": "73f042043e635f13a0d122a4961e62978223a846164ff40005d62a8c1bb8d45f",
|
| 414 |
+
"backbone.layers.45.mixer.A_log": "6b29bb3ec4d760b328693ec2075273e2f87fcb477198ea99d2355ba1fbe3f2fe",
|
| 415 |
+
"backbone.layers.45.mixer.D": "dd647c522d145274b89d9e82275a14a09b893c18494bdc6b28e4ffb67bb6c29d",
|
| 416 |
+
"backbone.layers.45.mixer.in_proj.weight": "f5bb86e5849f2a24c636afde66ab46133543c4a718e05593f75e9e3d14ff269b",
|
| 417 |
+
"backbone.layers.45.mixer.conv1d.weight": "5c544fdc99c8c4d3ced0a4bfad4a32b7ead3ebab0eb60254a6f8307bedf0625d",
|
| 418 |
+
"backbone.layers.45.mixer.conv1d.bias": "f2f43f61ebc03bc0c8a7d6ff1259451c62d27d9a807e1ba304aa191c141764e1",
|
| 419 |
+
"backbone.layers.45.mixer.norm.weight": "b092e6e4324da326fb861227817c38f2b09ed66c2f8cca03089569c760bce4cd",
|
| 420 |
+
"backbone.layers.45.mixer.out_proj.weight": "a3c0c5c70ab9fc43e9614a419ea71999a5d5cbace798db173be0431295750b5e",
|
| 421 |
+
"backbone.layers.45.norm.weight": "49f30d35192740e313602ffba85390aed06821c36ff1c65e6be293f3ae9ac972",
|
| 422 |
+
"backbone.layers.46.mixer.dt_bias": "f8caf32dddf92186fac1ba5b1ff4f181ccc0040ab1b38c461489cc356c50d33a",
|
| 423 |
+
"backbone.layers.46.mixer.A_log": "8a989920334bc92393686bbf2b2efa3611732fe53f84956d4de40fb861dc64e3",
|
| 424 |
+
"backbone.layers.46.mixer.D": "4834b609243d4475ae5e30871ee255334dada8949d02dac96a483f26cb192fa7",
|
| 425 |
+
"backbone.layers.46.mixer.in_proj.weight": "dbc7934c21cc6415a33b5fde480f97d7c1748b9d0da677c08c160bc73394607c",
|
| 426 |
+
"backbone.layers.46.mixer.conv1d.weight": "579dff7d8c5eac6701e7ceed5299beb4c6f1529227374f63825128f5aedf1a6b",
|
| 427 |
+
"backbone.layers.46.mixer.conv1d.bias": "a2ab3e03f854fc1a0a7c71e50b9801925508836cb7a02e076805fcdc774b2a88",
|
| 428 |
+
"backbone.layers.46.mixer.norm.weight": "69a4f3b5a185a9a35c0f231f67dc5a465ead6fb5404953af562bed10f7539d50",
|
| 429 |
+
"backbone.layers.46.mixer.out_proj.weight": "d95caeebc75a926eab40041b34e92d31f2423d2823aeb02bfcfc13f4f88a6c89",
|
| 430 |
+
"backbone.layers.46.norm.weight": "c421c4c3824bfa538102a680c5cadfff3ee40c0639f3a0e727348a541a58f1b1",
|
| 431 |
+
"backbone.layers.47.mixer.dt_bias": "bd0142a676b162c063ef0d3aac69e28a84b66a05477fba6035639dcf9b850b65",
|
| 432 |
+
"backbone.layers.47.mixer.A_log": "4f7d891c22784aad2cdaee0c5ce5fd1e33d6e71c3fbb845698ef84a717090b46",
|
| 433 |
+
"backbone.layers.47.mixer.D": "66718e3f205fd5d58d8e0a80d8c2d869a1e8af2172430de746bc20ec433200d2",
|
| 434 |
+
"backbone.layers.47.mixer.in_proj.weight": "667897945c7c5a8ea66a29937f34ff657d5a4c0630bcb5f5b8e544fa4b841c16",
|
| 435 |
+
"backbone.layers.47.mixer.conv1d.weight": "420cb29e3b2893903f8c64866acfeb93954a739cc2b24b4281c1ad1924847c2d",
|
| 436 |
+
"backbone.layers.47.mixer.conv1d.bias": "8d5882044063d76d03d24bb0477e712f16213820012221f51672188558175259",
|
| 437 |
+
"backbone.layers.47.mixer.norm.weight": "a153d49d4f0eca3739ff3bb2aa3e3bd71d6bc87fa616bfef9890b8989dd55f59",
|
| 438 |
+
"backbone.layers.47.mixer.out_proj.weight": "75afddcaff56c8e53d11fd4f1087f275f2feaa5ae45ca0443587dd455e2e6d3a",
|
| 439 |
+
"backbone.layers.47.norm.weight": "c5ab9dc43617118fd6eb9f7cf6edec79d6d6e7a6d15e7b9a1d10f4c8e8f824d1",
|
| 440 |
+
"backbone.layers.48.mixer.dt_bias": "23a9a99e94e54323779b8093cc5fcf089d2574c528bd896078d50ef52ff587db",
|
| 441 |
+
"backbone.layers.48.mixer.A_log": "a78fcbaf3ffdb027087186e2582b931121c2eab3ae481b06729f4bba2e7d973b",
|
| 442 |
+
"backbone.layers.48.mixer.D": "6cb037bd585ae7cf7f27b7d8e6e6364b148438c39a13669d6c8e666569675975",
|
| 443 |
+
"backbone.layers.48.mixer.in_proj.weight": "5fd1f74e4276ee334c58d7a06a54e449898c57ffb3a68e4273f2cf8e7794c972",
|
| 444 |
+
"backbone.layers.48.mixer.conv1d.weight": "59d397baded0c2b3d29c1544f9981f8e2144eeaed1b39640333e97b62271c1f8",
|
| 445 |
+
"backbone.layers.48.mixer.conv1d.bias": "068d0a9c792c2082e49c3e9c991fbab2cdc8dbf4647e5d56d480d21496d32b20",
|
| 446 |
+
"backbone.layers.48.mixer.norm.weight": "f7b8841dfe8b9774ee6ca50c96b5ba0e23da626bb3fea597c69520b00c050b87",
|
| 447 |
+
"backbone.layers.48.mixer.out_proj.weight": "896124e9f2a1cd3468589ea18e8c4439de1ac70266d1ce241190cbc8f29a935c",
|
| 448 |
+
"backbone.layers.48.norm.weight": "ac7f7955b75579f0638d3e64e6c3380386515b8cb1ed160a1578c20df13d1175",
|
| 449 |
+
"backbone.layers.49.mixer.dt_bias": "0fcd19bb99e92f5cff77e6078429ce3a6d8857344b2a843c8d6a7e1f696925e5",
|
| 450 |
+
"backbone.layers.49.mixer.A_log": "4d44157348c9ddd310695dc14c864bbc15868da6dd3dd9b221c08ef6acac025a",
|
| 451 |
+
"backbone.layers.49.mixer.D": "e633566bf14b4724b48810efe18360b9ac69e95b4491dded550cff85ce11857a",
|
| 452 |
+
"backbone.layers.49.mixer.in_proj.weight": "712f187ac1142b8063aa9597b8a13e0c1e7b35ed2e43d08e550e06bc99c65a94",
|
| 453 |
+
"backbone.layers.49.mixer.conv1d.weight": "833fd245bb69b6bf561ce8e8a3f8271bc90f7536c74e7815b400d89bdcb1334d",
|
| 454 |
+
"backbone.layers.49.mixer.conv1d.bias": "23c9fba917b74b1a2ae7366f54565a3d96d2d0fda4068dd1f24cb74d36fd0a1b",
|
| 455 |
+
"backbone.layers.49.mixer.norm.weight": "f255dc5c52a27300e359207e369a61ee8fc793825933f3e183c9013b04e998cb",
|
| 456 |
+
"backbone.layers.49.mixer.out_proj.weight": "a3c75fa89ca051e22b6d406421d14cdb4c5d069e46d2a52adc47d303db4ce0d3",
|
| 457 |
+
"backbone.layers.49.norm.weight": "f5b08cea96d8f46f81c7488b8d62cd4daa861c3a9c720fc3826850c2295e6153",
|
| 458 |
+
"backbone.layers.50.mixer.dt_bias": "aadca8ec16a867cc51108504122036d3180b4a49cefeefdfc214a87ee6a04922",
|
| 459 |
+
"backbone.layers.50.mixer.A_log": "5c1fd8241fbcc1c27e66f4e9bf1a790783ec51e387b88605354f1d17f5310253",
|
| 460 |
+
"backbone.layers.50.mixer.D": "ae310a7f1e2aa98cd9fd8a9270af18a84c07b799cef3429a134e874b02e053b5",
|
| 461 |
+
"backbone.layers.50.mixer.in_proj.weight": "7afd3f3624c3c6127d65bd20efdbe091022013713fd8f6f5374442c0e2236c4e",
|
| 462 |
+
"backbone.layers.50.mixer.conv1d.weight": "f84769bba2fee2a7098187bc41a2831446c872100f661a8d0d939d43ca4ca77b",
|
| 463 |
+
"backbone.layers.50.mixer.conv1d.bias": "0b205aeb8ee73e55fa2e78fbda95a0cf92a43f9a0b80444f5778dc083d1bb4d0",
|
| 464 |
+
"backbone.layers.50.mixer.norm.weight": "cdcf9fb2e68b7eb03f98839d8f612a1ce94e25590813f9f29b4f44bb68e6dee8",
|
| 465 |
+
"backbone.layers.50.mixer.out_proj.weight": "b46fda55f8ac69d0f9d2078a70b0853df06146ba0a83281e743df87043cec01a",
|
| 466 |
+
"backbone.layers.50.norm.weight": "6436551798c81da55ca6367216278a121ae1720616f9610a5c687a4a0e06fef3",
|
| 467 |
+
"backbone.layers.51.mixer.dt_bias": "809ad982abbd2d789f5987a7f2cc7f8b3117e668c96af026c493e34743b3cbbd",
|
| 468 |
+
"backbone.layers.51.mixer.A_log": "49e32c8d2c13558901db8e4b04bdc2742b08e7202e72b286c722b18d47c9aa97",
|
| 469 |
+
"backbone.layers.51.mixer.D": "def5cfc13027bf23767a3350da1b773efb9fcb03142d328c0d6e7ea8e7f749c4",
|
| 470 |
+
"backbone.layers.51.mixer.in_proj.weight": "5afe56636df8c9e4354ded74c20a98535b459f2303549b8f90dd8d1c4b111336",
|
| 471 |
+
"backbone.layers.51.mixer.conv1d.weight": "b003794db7bf6307c667045ffa915c53b7080c82cabe3c55d4b20339b33b5a8a",
|
| 472 |
+
"backbone.layers.51.mixer.conv1d.bias": "aee66d98ac1223d5b694da5a799baa9e0da8664a588288f3364f3bc37d1682ef",
|
| 473 |
+
"backbone.layers.51.mixer.norm.weight": "f62951febc3c24829e24fa5a9f4c316f5ab0df289dd473db02ded6e1e3487e3a",
|
| 474 |
+
"backbone.layers.51.mixer.out_proj.weight": "6dc9fe6134915e1d72d15b851b13bb1ee514971bed4974975a178dca551b15ce",
|
| 475 |
+
"backbone.layers.51.norm.weight": "dc862c6c47dc5ad41303a2baa6b173c25c3affed57f49205b0aa04412749226e",
|
| 476 |
+
"backbone.layers.52.mixer.dt_bias": "f74bef10810b0cb3a93c1f7c014da3f7443cec9852ac0edc274c8f481d8b0a49",
|
| 477 |
+
"backbone.layers.52.mixer.A_log": "fb590d261a373571292c894afcd7e71cab9ae39780ebca350c35581f0df6416d",
|
| 478 |
+
"backbone.layers.52.mixer.D": "40eae124031d35334db04160927348808b35efaccf64cc40116e67c4febe3909",
|
| 479 |
+
"backbone.layers.52.mixer.in_proj.weight": "644302079359bf94e3aa7513da4f904fe415ffc55dac4db492e013d11a092749",
|
| 480 |
+
"backbone.layers.52.mixer.conv1d.weight": "9535472952b188f805d443ec6d38680a2a15db861a907fbed6fd3521a9560acf",
|
| 481 |
+
"backbone.layers.52.mixer.conv1d.bias": "e183dc4d8a06c4110da78d6f6db7f3d38f0c75e24062eda4d6f550dbbbbfb056",
|
| 482 |
+
"backbone.layers.52.mixer.norm.weight": "e33b3417a787b99f5c0649a7aa3123204099f411a93ee44fc07a84bb03545ffe",
|
| 483 |
+
"backbone.layers.52.mixer.out_proj.weight": "e264f23433d82a43ab159640fcf111f26576728304e30ccaf5de3c11c971dbb0",
|
| 484 |
+
"backbone.layers.52.norm.weight": "50817ac8acd636b72e33ed1e9de0d29d58c4bbb6d7751a55997f4ccaf40887d9",
|
| 485 |
+
"backbone.layers.53.mixer.dt_bias": "54c22bf7fb4a2652393889934c697e92dd607d863ca4342022a665dfdbfeca45",
|
| 486 |
+
"backbone.layers.53.mixer.A_log": "ec349f324dca3159cca46a3711c22df5e39bb4ce531f2d8c08ea20660a9cb783",
|
| 487 |
+
"backbone.layers.53.mixer.D": "02e9bc75761e09e3555dea47f2d592b82f0be4032b023ef2a966fa100e0c4195",
|
| 488 |
+
"backbone.layers.53.mixer.in_proj.weight": "05a5ed90d4e709f5e1fe31a454bb651d0fdbc4d9ffe9926c51632d5c067c70e7",
|
| 489 |
+
"backbone.layers.53.mixer.conv1d.weight": "cdee9ce12fdceab78e4c346057cb4cf946e646b0c46134b0605739f6385a7a77",
|
| 490 |
+
"backbone.layers.53.mixer.conv1d.bias": "61645607fed62f1d7afe76581081d9b3711b553fbd2d6edabec8737e962885d4",
|
| 491 |
+
"backbone.layers.53.mixer.norm.weight": "880d1f7d6901bfee82c5bd36f45dfbe367700596558fa7da2481c39a466f5e0d",
|
| 492 |
+
"backbone.layers.53.mixer.out_proj.weight": "bf63f65cccdc6bbed2e881f293146b669e3cad753d2d413391e3107981eb48fd",
|
| 493 |
+
"backbone.layers.53.norm.weight": "c72b82592f1abaadc353befea342fe8b220c3f64ed8aa4d5044110dcafcc6776",
|
| 494 |
+
"backbone.layers.54.mixer.dt_bias": "17947be77cb8cc01a782b111ac37b7c9f36feac7394e952dd4b65b241b10fddd",
|
| 495 |
+
"backbone.layers.54.mixer.A_log": "23998d19aa4a789eed263f7c89f64ce81c5e19581f185441dd72f075cc440da6",
|
| 496 |
+
"backbone.layers.54.mixer.D": "8e30d52e7ddf898f766aa1d730fbe8153198193cb6287955210b5f63d1e2c75e",
|
| 497 |
+
"backbone.layers.54.mixer.in_proj.weight": "8e94e6d25fd4cedcdfe1787486cb850c0cfb59a4e22ddf704d8df230a8904a13",
|
| 498 |
+
"backbone.layers.54.mixer.conv1d.weight": "ddb2b508247f0d0af9bc93fcc95893d538ccc9817c6aaf75566d4e98c95aa1c0",
|
| 499 |
+
"backbone.layers.54.mixer.conv1d.bias": "86f246f7939f535033a6ae09b8978c199f15b0191b86c2a1cd2713e430edd4f3",
|
| 500 |
+
"backbone.layers.54.mixer.norm.weight": "643eaf1bf34c823b7f1edf43a4446c766b91589a5103916d837cc21a7b5e3f99",
|
| 501 |
+
"backbone.layers.54.mixer.out_proj.weight": "b59108b7ad1a9c31a2ba76046584d1fa4a18c7a7b49c602cdff4cd80d5c2121a",
|
| 502 |
+
"backbone.layers.54.norm.weight": "34f792867f2eb4849503b03959193ccd9e2db3017e61fc1f8ae2fed08f18d5fa",
|
| 503 |
+
"backbone.layers.55.mixer.dt_bias": "c62e1181d17ae9066f1580fc55bb69327cc3fd10a2dc2cdea946874edaaf83e3",
|
| 504 |
+
"backbone.layers.55.mixer.A_log": "6e0edce166fba392ad2e1052dc11cf69f4f99de6f0f9d15f1987c7e91bc7de8c",
|
| 505 |
+
"backbone.layers.55.mixer.D": "3213cef8067aff9ed891bae84b92ae1f2732b3bfc43a6c54725a9fae2e436c18",
|
| 506 |
+
"backbone.layers.55.mixer.in_proj.weight": "526fe3bed402df73bc8828a01b6fd7ff14448763da864180122e85491eb6f9ef",
|
| 507 |
+
"backbone.layers.55.mixer.conv1d.weight": "bcfe867de02764bc60add117d9e0835da99c653430372810c179dc7a2c71a360",
|
| 508 |
+
"backbone.layers.55.mixer.conv1d.bias": "8ecf0804842bb5da41cca59add2c0d47cfda3a2b12bbd38e972c6bd1f2290b10",
|
| 509 |
+
"backbone.layers.55.mixer.norm.weight": "e6b57a00377e4f6d0ed915979dfdb47d5094c80b8943029641deb38d479c886d",
|
| 510 |
+
"backbone.layers.55.mixer.out_proj.weight": "5911652f3ea8d51be49565444d46ae01a93e7ea465c34f0014c1cf5eae7790b6",
|
| 511 |
+
"backbone.layers.55.norm.weight": "0cebaa98b5f5c71290d4f5855504bd69063e5122d4b22c097a915536ee54de32",
|
| 512 |
+
"backbone.norm_f.weight": "faaec522c44fb91b13d39994ddcab415ca0f106eefd127a2fd437b2303759860",
|
| 513 |
+
"lm_head.weight": "9d76b8d52d842e671a9807fb4eae2e124c6a89b36ad3a72538d83f9362cac9ca"
|
| 514 |
+
}
|
| 515 |
+
}
|
reports/wt2_test_v1/without_resurface.json
ADDED
|
@@ -0,0 +1,1226 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "RECALL_NATIVE_PREFILL_WT2_TEST_V1",
|
| 3 |
+
"arm": "without_resurface",
|
| 4 |
+
"model_kind": "fp16",
|
| 5 |
+
"complete": true,
|
| 6 |
+
"adapter_loaded": false,
|
| 7 |
+
"adapter_sha256": null,
|
| 8 |
+
"dataset": {
|
| 9 |
+
"dataset": "Salesforce/wikitext",
|
| 10 |
+
"configuration": "wikitext-2-raw-v1",
|
| 11 |
+
"split": "test",
|
| 12 |
+
"revision_argument": "b08601e04326c79dfdd32d625aee71d232d685c3",
|
| 13 |
+
"dataset_fingerprint": "a46124b21ac53738",
|
| 14 |
+
"document_join": "two newline characters",
|
| 15 |
+
"text_sha256": "696cca6b65a171b0a358a4be6732cdfdf2dd6164a32e20fd70e3c13fc4dfae83",
|
| 16 |
+
"token_stream_sha256_int64le": "5b82bd46e833e77fcfc0af62bafeaac62e70e68cfdf214d375f0b7b132d4b608",
|
| 17 |
+
"total_tokens": 300964,
|
| 18 |
+
"tokenizer_sha256": "5862e2f71caf762bc9845662be5fec2867deb58d874568235a02a36c5111cd09",
|
| 19 |
+
"automatic_special_tokens": false
|
| 20 |
+
},
|
| 21 |
+
"ppl": {
|
| 22 |
+
"ppl": 7.244532175944678,
|
| 23 |
+
"nll": 595981.0783724785,
|
| 24 |
+
"target_tokens": 300963,
|
| 25 |
+
"execution": "prefill",
|
| 26 |
+
"windows": [
|
| 27 |
+
{
|
| 28 |
+
"start": 0,
|
| 29 |
+
"input_tokens": 2048,
|
| 30 |
+
"target_tokens": 2048,
|
| 31 |
+
"token_sha256_int64le": "1a2d6b883799adf5ffc7f16d48b0460668675e47f0f434ff5a718f9dc4b4df46",
|
| 32 |
+
"nll": 3494.153772354126,
|
| 33 |
+
"ppl": 5.507604489076998
|
| 34 |
+
},
|
| 35 |
+
{
|
| 36 |
+
"start": 2048,
|
| 37 |
+
"input_tokens": 2048,
|
| 38 |
+
"target_tokens": 2048,
|
| 39 |
+
"token_sha256_int64le": "7697abe4489bc66e2c85a387f701d5fed7488ad33bf58c10222d23c1183126a4",
|
| 40 |
+
"nll": 4594.441719055176,
|
| 41 |
+
"ppl": 9.42513206683709
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"start": 4096,
|
| 45 |
+
"input_tokens": 2048,
|
| 46 |
+
"target_tokens": 2048,
|
| 47 |
+
"token_sha256_int64le": "7e16a843f8aaf1e2785a41a80563825a702e36684f88bea80334ea1d24d103ca",
|
| 48 |
+
"nll": 4974.796684265137,
|
| 49 |
+
"ppl": 11.348663043770731
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"start": 6144,
|
| 53 |
+
"input_tokens": 2048,
|
| 54 |
+
"target_tokens": 2048,
|
| 55 |
+
"token_sha256_int64le": "1a1e313f7e4842994100c1c05d95a74161f38ce648ee8492cb965cc90d1e42e0",
|
| 56 |
+
"nll": 4199.120769500732,
|
| 57 |
+
"ppl": 7.770635408819106
|
| 58 |
+
},
|
| 59 |
+
{
|
| 60 |
+
"start": 8192,
|
| 61 |
+
"input_tokens": 2048,
|
| 62 |
+
"target_tokens": 2048,
|
| 63 |
+
"token_sha256_int64le": "36e892d798f0575cdf0d2c213de6e9741cbd1b6b2f9559300637bddcbd1054bb",
|
| 64 |
+
"nll": 3496.2670974731445,
|
| 65 |
+
"ppl": 5.51329070308505
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"start": 10240,
|
| 69 |
+
"input_tokens": 2048,
|
| 70 |
+
"target_tokens": 2048,
|
| 71 |
+
"token_sha256_int64le": "861b7f6f1b60b293c5c29869f740bca014bac01991b7ea49194c4d849fbfc56c",
|
| 72 |
+
"nll": 3522.128894805908,
|
| 73 |
+
"ppl": 5.5833530414961805
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"start": 12288,
|
| 77 |
+
"input_tokens": 2048,
|
| 78 |
+
"target_tokens": 2048,
|
| 79 |
+
"token_sha256_int64le": "0fa99db0d520d1f840220596695f453b79857e1d58bcfa1621b961029ab5627b",
|
| 80 |
+
"nll": 2881.9797687530518,
|
| 81 |
+
"ppl": 4.084570916364946
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"start": 14336,
|
| 85 |
+
"input_tokens": 2048,
|
| 86 |
+
"target_tokens": 2048,
|
| 87 |
+
"token_sha256_int64le": "81e4e56a99d407898ed96c05d2761d1d8a259687d425a947a136a5aad43db69a",
|
| 88 |
+
"nll": 3706.147300720215,
|
| 89 |
+
"ppl": 6.108261729274788
|
| 90 |
+
},
|
| 91 |
+
{
|
| 92 |
+
"start": 16384,
|
| 93 |
+
"input_tokens": 2048,
|
| 94 |
+
"target_tokens": 2048,
|
| 95 |
+
"token_sha256_int64le": "529d7b8692a2930dfdf62f3433cd902ce6efc941a682650be224f7681b04d55d",
|
| 96 |
+
"nll": 4021.177474975586,
|
| 97 |
+
"ppl": 7.123972918347191
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"start": 18432,
|
| 101 |
+
"input_tokens": 2048,
|
| 102 |
+
"target_tokens": 2048,
|
| 103 |
+
"token_sha256_int64le": "e8d680b5ad89bbe9fc06e27c8e266813e3384e309d24c635bd8097b75957bd0c",
|
| 104 |
+
"nll": 4270.329284667969,
|
| 105 |
+
"ppl": 8.045570737249781
|
| 106 |
+
},
|
| 107 |
+
{
|
| 108 |
+
"start": 20480,
|
| 109 |
+
"input_tokens": 2048,
|
| 110 |
+
"target_tokens": 2048,
|
| 111 |
+
"token_sha256_int64le": "b55804a097552f93e7a3ada5d3e5ca88b193da7602e08f60043164f7d276f26a",
|
| 112 |
+
"nll": 3937.443672180176,
|
| 113 |
+
"ppl": 6.83857870281585
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"start": 22528,
|
| 117 |
+
"input_tokens": 2048,
|
| 118 |
+
"target_tokens": 2048,
|
| 119 |
+
"token_sha256_int64le": "79ac58154608ca6c4faa60a26006cb25a52ffee8ebed64c9b3a1aadab5966456",
|
| 120 |
+
"nll": 4059.571937561035,
|
| 121 |
+
"ppl": 7.258787908435264
|
| 122 |
+
},
|
| 123 |
+
{
|
| 124 |
+
"start": 24576,
|
| 125 |
+
"input_tokens": 2048,
|
| 126 |
+
"target_tokens": 2048,
|
| 127 |
+
"token_sha256_int64le": "5a4bd71e5725cff95a16854b008ca69d89f687a1c72829b37d9db72cf2ca092c",
|
| 128 |
+
"nll": 4080.9905166625977,
|
| 129 |
+
"ppl": 7.335100777131728
|
| 130 |
+
},
|
| 131 |
+
{
|
| 132 |
+
"start": 26624,
|
| 133 |
+
"input_tokens": 2048,
|
| 134 |
+
"target_tokens": 2048,
|
| 135 |
+
"token_sha256_int64le": "0dff6eb969a20082d5454585184896f13c5f82644d509724add1fb00df67d640",
|
| 136 |
+
"nll": 4459.779159545898,
|
| 137 |
+
"ppl": 8.825334826213203
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"start": 28672,
|
| 141 |
+
"input_tokens": 2048,
|
| 142 |
+
"target_tokens": 2048,
|
| 143 |
+
"token_sha256_int64le": "05cf104fcf3f4da284adc1a9c0f75188010b8d99da8a339d6ad6bf67ea63179a",
|
| 144 |
+
"nll": 4411.475830078125,
|
| 145 |
+
"ppl": 8.61961941798709
|
| 146 |
+
},
|
| 147 |
+
{
|
| 148 |
+
"start": 30720,
|
| 149 |
+
"input_tokens": 2048,
|
| 150 |
+
"target_tokens": 2048,
|
| 151 |
+
"token_sha256_int64le": "5bcb87c98edf5bd5e2cc1dc8b9cb6035e1039da343a4c1e0d5bd2315bd301d2f",
|
| 152 |
+
"nll": 4519.329086303711,
|
| 153 |
+
"ppl": 9.085717321533327
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"start": 32768,
|
| 157 |
+
"input_tokens": 2048,
|
| 158 |
+
"target_tokens": 2048,
|
| 159 |
+
"token_sha256_int64le": "e5572a47b0f3190b6d5fe26f1851a8cebe56c94568f8d0bff7507bc82b3ee874",
|
| 160 |
+
"nll": 4901.790435791016,
|
| 161 |
+
"ppl": 10.951236379146472
|
| 162 |
+
},
|
| 163 |
+
{
|
| 164 |
+
"start": 34816,
|
| 165 |
+
"input_tokens": 2048,
|
| 166 |
+
"target_tokens": 2048,
|
| 167 |
+
"token_sha256_int64le": "9594c2d6d8c598ef18b85af375fd588e8ed0d894378c4bfb38203b0e795c36f5",
|
| 168 |
+
"nll": 4350.239295959473,
|
| 169 |
+
"ppl": 8.36570224438543
|
| 170 |
+
},
|
| 171 |
+
{
|
| 172 |
+
"start": 36864,
|
| 173 |
+
"input_tokens": 2048,
|
| 174 |
+
"target_tokens": 2048,
|
| 175 |
+
"token_sha256_int64le": "523e0fc4c878920a442d2a0dd7056700c83983dc91a3d324e9487e2897267a22",
|
| 176 |
+
"nll": 4169.271186828613,
|
| 177 |
+
"ppl": 7.658199827691167
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"start": 38912,
|
| 181 |
+
"input_tokens": 2048,
|
| 182 |
+
"target_tokens": 2048,
|
| 183 |
+
"token_sha256_int64le": "79fb5ecc92a0428cf4226e62ff4cacfd4e9c10ac87d9459be06606cee84ecd56",
|
| 184 |
+
"nll": 3419.7476625442505,
|
| 185 |
+
"ppl": 5.31109837078257
|
| 186 |
+
},
|
| 187 |
+
{
|
| 188 |
+
"start": 40960,
|
| 189 |
+
"input_tokens": 2048,
|
| 190 |
+
"target_tokens": 2048,
|
| 191 |
+
"token_sha256_int64le": "9f1b10da5f0e0a8f3038e97313456ca3654c01ebcdb7c2b9af430f567522fe12",
|
| 192 |
+
"nll": 3953.276336669922,
|
| 193 |
+
"ppl": 6.891651222617239
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"start": 43008,
|
| 197 |
+
"input_tokens": 2048,
|
| 198 |
+
"target_tokens": 2048,
|
| 199 |
+
"token_sha256_int64le": "c298e4f30124d7dd5b2acd1af1c20fd10d5c760f914bccf6596222b9af6a4817",
|
| 200 |
+
"nll": 4715.557823181152,
|
| 201 |
+
"ppl": 9.999333775754987
|
| 202 |
+
},
|
| 203 |
+
{
|
| 204 |
+
"start": 45056,
|
| 205 |
+
"input_tokens": 2048,
|
| 206 |
+
"target_tokens": 2048,
|
| 207 |
+
"token_sha256_int64le": "1ee4f672f4aa05e0bca489aa34e611d5c3672fd45d35ecaa80650172f517e1f3",
|
| 208 |
+
"nll": 4255.512184143066,
|
| 209 |
+
"ppl": 7.98757179964033
|
| 210 |
+
},
|
| 211 |
+
{
|
| 212 |
+
"start": 47104,
|
| 213 |
+
"input_tokens": 2048,
|
| 214 |
+
"target_tokens": 2048,
|
| 215 |
+
"token_sha256_int64le": "a32fa420db87e3b71bbd34feabf530fa3f10459413f5d4f5dc362971ae6c713c",
|
| 216 |
+
"nll": 4370.436614990234,
|
| 217 |
+
"ppl": 8.448612725662782
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"start": 49152,
|
| 221 |
+
"input_tokens": 2048,
|
| 222 |
+
"target_tokens": 2048,
|
| 223 |
+
"token_sha256_int64le": "3a080801aee2913550da2ef76116d30b8544f72f3e415efeb32a28c853f885b0",
|
| 224 |
+
"nll": 4400.240097045898,
|
| 225 |
+
"ppl": 8.572459962495888
|
| 226 |
+
},
|
| 227 |
+
{
|
| 228 |
+
"start": 51200,
|
| 229 |
+
"input_tokens": 2048,
|
| 230 |
+
"target_tokens": 2048,
|
| 231 |
+
"token_sha256_int64le": "9a31a733e7fd3359f73439a67d44ebb8b226ade7db57b6d8dfdc365e38ab8578",
|
| 232 |
+
"nll": 4664.821754455566,
|
| 233 |
+
"ppl": 9.754658807455437
|
| 234 |
+
},
|
| 235 |
+
{
|
| 236 |
+
"start": 53248,
|
| 237 |
+
"input_tokens": 2048,
|
| 238 |
+
"target_tokens": 2048,
|
| 239 |
+
"token_sha256_int64le": "00ab7d803922a2919b32b0a911ea2c785a0c10b1daf6a8a41822ee3fd4dbbbfe",
|
| 240 |
+
"nll": 4223.743103027344,
|
| 241 |
+
"ppl": 7.864622689724836
|
| 242 |
+
},
|
| 243 |
+
{
|
| 244 |
+
"start": 55296,
|
| 245 |
+
"input_tokens": 2048,
|
| 246 |
+
"target_tokens": 2048,
|
| 247 |
+
"token_sha256_int64le": "ea0361a9b10122b903c6c0c566223c34ba6777438ad9760a4fd2780df416bb4c",
|
| 248 |
+
"nll": 5251.601356506348,
|
| 249 |
+
"ppl": 12.991021627993668
|
| 250 |
+
},
|
| 251 |
+
{
|
| 252 |
+
"start": 57344,
|
| 253 |
+
"input_tokens": 2048,
|
| 254 |
+
"target_tokens": 2048,
|
| 255 |
+
"token_sha256_int64le": "273931d43814c4fc37e89a18cdf28ad55c4b4a7838eb730bf1c50524dd63b07b",
|
| 256 |
+
"nll": 4795.1335372924805,
|
| 257 |
+
"ppl": 10.395508080475063
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"start": 59392,
|
| 261 |
+
"input_tokens": 2048,
|
| 262 |
+
"target_tokens": 2048,
|
| 263 |
+
"token_sha256_int64le": "504bf8092c24e386bb14d7fd516f3b9b87de76cbed8d5b04e77b5b648959e721",
|
| 264 |
+
"nll": 4159.1624755859375,
|
| 265 |
+
"ppl": 7.62049289913323
|
| 266 |
+
},
|
| 267 |
+
{
|
| 268 |
+
"start": 61440,
|
| 269 |
+
"input_tokens": 2048,
|
| 270 |
+
"target_tokens": 2048,
|
| 271 |
+
"token_sha256_int64le": "5ae8981c50c7d44bdf981826af279d76b58672a60b39ab9f42208d5ca4e05f58",
|
| 272 |
+
"nll": 4096.841087341309,
|
| 273 |
+
"ppl": 7.392091312749801
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"start": 63488,
|
| 277 |
+
"input_tokens": 2048,
|
| 278 |
+
"target_tokens": 2048,
|
| 279 |
+
"token_sha256_int64le": "d83e6672b7bc41afde8ee4f57a55a33ded70664969f6b10f23fcccc465cb318a",
|
| 280 |
+
"nll": 3812.442581176758,
|
| 281 |
+
"ppl": 6.433664155724488
|
| 282 |
+
},
|
| 283 |
+
{
|
| 284 |
+
"start": 65536,
|
| 285 |
+
"input_tokens": 2048,
|
| 286 |
+
"target_tokens": 2048,
|
| 287 |
+
"token_sha256_int64le": "dce022be768611664ee67ddfdae6abc4af0dadfd9643d691f67ab6bcc1673795",
|
| 288 |
+
"nll": 3209.4397888183594,
|
| 289 |
+
"ppl": 4.792773544278978
|
| 290 |
+
},
|
| 291 |
+
{
|
| 292 |
+
"start": 67584,
|
| 293 |
+
"input_tokens": 2048,
|
| 294 |
+
"target_tokens": 2048,
|
| 295 |
+
"token_sha256_int64le": "3a1c84c593afe2049a4686370c7f15fc16c23acb44d0e6d80117025044d099cc",
|
| 296 |
+
"nll": 3941.0531997680664,
|
| 297 |
+
"ppl": 6.850642083502963
|
| 298 |
+
},
|
| 299 |
+
{
|
| 300 |
+
"start": 69632,
|
| 301 |
+
"input_tokens": 2048,
|
| 302 |
+
"target_tokens": 2048,
|
| 303 |
+
"token_sha256_int64le": "d7c822f494cbef0ca81ba2c79bc8038895e9c80c4fd293a97f0a02ef6295c082",
|
| 304 |
+
"nll": 3829.544334411621,
|
| 305 |
+
"ppl": 6.487613182203689
|
| 306 |
+
},
|
| 307 |
+
{
|
| 308 |
+
"start": 71680,
|
| 309 |
+
"input_tokens": 2048,
|
| 310 |
+
"target_tokens": 2048,
|
| 311 |
+
"token_sha256_int64le": "39af145da34bb3e1d399335c388ca897f7b8015b915dffd9126d06a66ca9ab69",
|
| 312 |
+
"nll": 4200.018852233887,
|
| 313 |
+
"ppl": 7.774043711476979
|
| 314 |
+
},
|
| 315 |
+
{
|
| 316 |
+
"start": 73728,
|
| 317 |
+
"input_tokens": 2048,
|
| 318 |
+
"target_tokens": 2048,
|
| 319 |
+
"token_sha256_int64le": "f92c643e9edc615dbea357215ad6d0dd830b12807354551edae764294e8c1f1c",
|
| 320 |
+
"nll": 3638.6435585021973,
|
| 321 |
+
"ppl": 5.9102103621082485
|
| 322 |
+
},
|
| 323 |
+
{
|
| 324 |
+
"start": 75776,
|
| 325 |
+
"input_tokens": 2048,
|
| 326 |
+
"target_tokens": 2048,
|
| 327 |
+
"token_sha256_int64le": "c888968354fe6914e5290adaf9ea6fe52f04ca26a15fcb1eed68a0bd391e3830",
|
| 328 |
+
"nll": 2438.1566276550293,
|
| 329 |
+
"ppl": 3.2887454367797195
|
| 330 |
+
},
|
| 331 |
+
{
|
| 332 |
+
"start": 77824,
|
| 333 |
+
"input_tokens": 2048,
|
| 334 |
+
"target_tokens": 2048,
|
| 335 |
+
"token_sha256_int64le": "bdea806e668159c6cb35589ad033df517b7f254734d9f51944f13c97c9d3ae68",
|
| 336 |
+
"nll": 2823.446002960205,
|
| 337 |
+
"ppl": 3.9694825399937046
|
| 338 |
+
},
|
| 339 |
+
{
|
| 340 |
+
"start": 79872,
|
| 341 |
+
"input_tokens": 2048,
|
| 342 |
+
"target_tokens": 2048,
|
| 343 |
+
"token_sha256_int64le": "4b07638d19b5c9ad746568806beb439065d4cb78383216c80f9a3e4d27d463e5",
|
| 344 |
+
"nll": 2865.2736625671387,
|
| 345 |
+
"ppl": 4.051387461555422
|
| 346 |
+
},
|
| 347 |
+
{
|
| 348 |
+
"start": 81920,
|
| 349 |
+
"input_tokens": 2048,
|
| 350 |
+
"target_tokens": 2048,
|
| 351 |
+
"token_sha256_int64le": "63d59d1b1701e47518b37c6dbe96328f74b55cfa0c39fa777572550316c4e30b",
|
| 352 |
+
"nll": 2843.3174591064453,
|
| 353 |
+
"ppl": 4.008185331386159
|
| 354 |
+
},
|
| 355 |
+
{
|
| 356 |
+
"start": 83968,
|
| 357 |
+
"input_tokens": 2048,
|
| 358 |
+
"target_tokens": 2048,
|
| 359 |
+
"token_sha256_int64le": "728708cfa01a3dace51ee3914cd1b7df8e8965b3be89c074ec1d526e26ffe9ca",
|
| 360 |
+
"nll": 3246.5053520202637,
|
| 361 |
+
"ppl": 4.880304870692085
|
| 362 |
+
},
|
| 363 |
+
{
|
| 364 |
+
"start": 86016,
|
| 365 |
+
"input_tokens": 2048,
|
| 366 |
+
"target_tokens": 2048,
|
| 367 |
+
"token_sha256_int64le": "415a1604f02c5b8d7be0dcf28cacd543e9d7cae35f1a096ac2bd9bc558397e80",
|
| 368 |
+
"nll": 4319.6020584106445,
|
| 369 |
+
"ppl": 8.241486206988096
|
| 370 |
+
},
|
| 371 |
+
{
|
| 372 |
+
"start": 88064,
|
| 373 |
+
"input_tokens": 2048,
|
| 374 |
+
"target_tokens": 2048,
|
| 375 |
+
"token_sha256_int64le": "9f1a7dfd05a3fe0f23f3240717e27ce520d3bb2ef23d9563ba3c1d0c1c886fc6",
|
| 376 |
+
"nll": 5180.963424682617,
|
| 377 |
+
"ppl": 12.550585240608235
|
| 378 |
+
},
|
| 379 |
+
{
|
| 380 |
+
"start": 90112,
|
| 381 |
+
"input_tokens": 2048,
|
| 382 |
+
"target_tokens": 2048,
|
| 383 |
+
"token_sha256_int64le": "c3e4dd63053d31e9664902b623588bc69bc8db1825922f50a5317d6883611927",
|
| 384 |
+
"nll": 5577.9566650390625,
|
| 385 |
+
"ppl": 15.235247424707776
|
| 386 |
+
},
|
| 387 |
+
{
|
| 388 |
+
"start": 92160,
|
| 389 |
+
"input_tokens": 2048,
|
| 390 |
+
"target_tokens": 2048,
|
| 391 |
+
"token_sha256_int64le": "176f55c4882d5853d613dce487101474627fd410c11a8c65b0fab479657da2f1",
|
| 392 |
+
"nll": 4683.435928344727,
|
| 393 |
+
"ppl": 9.843722569062209
|
| 394 |
+
},
|
| 395 |
+
{
|
| 396 |
+
"start": 94208,
|
| 397 |
+
"input_tokens": 2048,
|
| 398 |
+
"target_tokens": 2048,
|
| 399 |
+
"token_sha256_int64le": "5f0e51c563b7f36ec336f5e58d9da80d5afbf4a3ddfad34e24f0c9a7fb34aa3f",
|
| 400 |
+
"nll": 4291.830490112305,
|
| 401 |
+
"ppl": 8.130483204164673
|
| 402 |
+
},
|
| 403 |
+
{
|
| 404 |
+
"start": 96256,
|
| 405 |
+
"input_tokens": 2048,
|
| 406 |
+
"target_tokens": 2048,
|
| 407 |
+
"token_sha256_int64le": "cc5d6b982df6a3e63856f9558eadd8ff4ed07cf87028d7dd5dd35c07088f404e",
|
| 408 |
+
"nll": 4590.746429443359,
|
| 409 |
+
"ppl": 9.408141251317593
|
| 410 |
+
},
|
| 411 |
+
{
|
| 412 |
+
"start": 98304,
|
| 413 |
+
"input_tokens": 2048,
|
| 414 |
+
"target_tokens": 2048,
|
| 415 |
+
"token_sha256_int64le": "d95820340c62af23bd4070605867072e382dd287fad90f51a4212ede697981a2",
|
| 416 |
+
"nll": 4814.766464233398,
|
| 417 |
+
"ppl": 10.495642674089316
|
| 418 |
+
},
|
| 419 |
+
{
|
| 420 |
+
"start": 100352,
|
| 421 |
+
"input_tokens": 2048,
|
| 422 |
+
"target_tokens": 2048,
|
| 423 |
+
"token_sha256_int64le": "7ca14adfb3be5680cd954b14482135c6578e679f2c4dfca98ea175a5ab31423b",
|
| 424 |
+
"nll": 4387.232940673828,
|
| 425 |
+
"ppl": 8.518187507199547
|
| 426 |
+
},
|
| 427 |
+
{
|
| 428 |
+
"start": 102400,
|
| 429 |
+
"input_tokens": 2048,
|
| 430 |
+
"target_tokens": 2048,
|
| 431 |
+
"token_sha256_int64le": "3e21ad94cfbf0d1252f4ac9f5379feb0bb59325c6de00fda03d4ed492f49d4a1",
|
| 432 |
+
"nll": 3626.4562759399414,
|
| 433 |
+
"ppl": 5.875144194573961
|
| 434 |
+
},
|
| 435 |
+
{
|
| 436 |
+
"start": 104448,
|
| 437 |
+
"input_tokens": 2048,
|
| 438 |
+
"target_tokens": 2048,
|
| 439 |
+
"token_sha256_int64le": "fb345a3fee185cb82cfd3740826a0e8b1e5e0f97b2625e9a2e863c4e9a1c6665",
|
| 440 |
+
"nll": 4172.224967956543,
|
| 441 |
+
"ppl": 7.669253034027615
|
| 442 |
+
},
|
| 443 |
+
{
|
| 444 |
+
"start": 106496,
|
| 445 |
+
"input_tokens": 2048,
|
| 446 |
+
"target_tokens": 2048,
|
| 447 |
+
"token_sha256_int64le": "93d9382d0ac84a8987e86a0aee19a0897ba27521df2cf6a24e84759d473db170",
|
| 448 |
+
"nll": 3456.4198112487793,
|
| 449 |
+
"ppl": 5.407057182363128
|
| 450 |
+
},
|
| 451 |
+
{
|
| 452 |
+
"start": 108544,
|
| 453 |
+
"input_tokens": 2048,
|
| 454 |
+
"target_tokens": 2048,
|
| 455 |
+
"token_sha256_int64le": "1d47fee2ba8b7a2c694cd0963df65056d2bbeb7e71f1aeb6038178384b831e14",
|
| 456 |
+
"nll": 3579.0213012695312,
|
| 457 |
+
"ppl": 5.740630197214967
|
| 458 |
+
},
|
| 459 |
+
{
|
| 460 |
+
"start": 110592,
|
| 461 |
+
"input_tokens": 2048,
|
| 462 |
+
"target_tokens": 2048,
|
| 463 |
+
"token_sha256_int64le": "31c43559c09f97b1d8b8ee46d021d266d93c3bd48661da1d0a8edf8b95aeeeba",
|
| 464 |
+
"nll": 3498.8253021240234,
|
| 465 |
+
"ppl": 5.520181786329111
|
| 466 |
+
},
|
| 467 |
+
{
|
| 468 |
+
"start": 112640,
|
| 469 |
+
"input_tokens": 2048,
|
| 470 |
+
"target_tokens": 2048,
|
| 471 |
+
"token_sha256_int64le": "80967f476ea0f604ed9e5e4876994ee60e6a9520f700aa4cd4d06c025ab8555e",
|
| 472 |
+
"nll": 3347.4490661621094,
|
| 473 |
+
"ppl": 5.126876551198251
|
| 474 |
+
},
|
| 475 |
+
{
|
| 476 |
+
"start": 114688,
|
| 477 |
+
"input_tokens": 2048,
|
| 478 |
+
"target_tokens": 2048,
|
| 479 |
+
"token_sha256_int64le": "526722d86264dd0327b8134a217c3dbc0cfeada6cf152f86990e45619cd211a4",
|
| 480 |
+
"nll": 3495.8827743530273,
|
| 481 |
+
"ppl": 5.512256188297487
|
| 482 |
+
},
|
| 483 |
+
{
|
| 484 |
+
"start": 116736,
|
| 485 |
+
"input_tokens": 2048,
|
| 486 |
+
"target_tokens": 2048,
|
| 487 |
+
"token_sha256_int64le": "f408a1b708dab5847c6d5199f41519e0b1350cc9c110762285620a836ecc1489",
|
| 488 |
+
"nll": 3067.2318382263184,
|
| 489 |
+
"ppl": 4.471266921110348
|
| 490 |
+
},
|
| 491 |
+
{
|
| 492 |
+
"start": 118784,
|
| 493 |
+
"input_tokens": 2048,
|
| 494 |
+
"target_tokens": 2048,
|
| 495 |
+
"token_sha256_int64le": "6052f506f3ab2f7b602857548485e766b10b516b92aaa3c0ad2218eaeca681c2",
|
| 496 |
+
"nll": 3550.278953552246,
|
| 497 |
+
"ppl": 5.660626897114106
|
| 498 |
+
},
|
| 499 |
+
{
|
| 500 |
+
"start": 120832,
|
| 501 |
+
"input_tokens": 2048,
|
| 502 |
+
"target_tokens": 2048,
|
| 503 |
+
"token_sha256_int64le": "d4d38109cafdc61e186b9115a258de568de94086ad37b46b4bc112630f79337d",
|
| 504 |
+
"nll": 3289.985987663269,
|
| 505 |
+
"ppl": 4.985025259681643
|
| 506 |
+
},
|
| 507 |
+
{
|
| 508 |
+
"start": 122880,
|
| 509 |
+
"input_tokens": 2048,
|
| 510 |
+
"target_tokens": 2048,
|
| 511 |
+
"token_sha256_int64le": "b4ab40620ac2cfe91e2b35e849c7e440aa3aceea004a2908c56a2cc9c831e2e6",
|
| 512 |
+
"nll": 4318.97908782959,
|
| 513 |
+
"ppl": 8.238979652956441
|
| 514 |
+
},
|
| 515 |
+
{
|
| 516 |
+
"start": 124928,
|
| 517 |
+
"input_tokens": 2048,
|
| 518 |
+
"target_tokens": 2048,
|
| 519 |
+
"token_sha256_int64le": "8fdaa7359f3f50d044792b3753b413fa458cf415b9d6a9ffe43c024a3ebeda8b",
|
| 520 |
+
"nll": 4525.313682556152,
|
| 521 |
+
"ppl": 9.11230612657042
|
| 522 |
+
},
|
| 523 |
+
{
|
| 524 |
+
"start": 126976,
|
| 525 |
+
"input_tokens": 2048,
|
| 526 |
+
"target_tokens": 2048,
|
| 527 |
+
"token_sha256_int64le": "c252332b02aa8e79b37bd54c086c701397bf47a88463840dfe39dbcfe96717dc",
|
| 528 |
+
"nll": 4697.724822998047,
|
| 529 |
+
"ppl": 9.912642362053022
|
| 530 |
+
},
|
| 531 |
+
{
|
| 532 |
+
"start": 129024,
|
| 533 |
+
"input_tokens": 2048,
|
| 534 |
+
"target_tokens": 2048,
|
| 535 |
+
"token_sha256_int64le": "a25db88b3e46febc7fe2b128cad4112e43c7c0a85fe4357cf3b76fe2c0bedb18",
|
| 536 |
+
"nll": 5034.206169128418,
|
| 537 |
+
"ppl": 11.68269259259181
|
| 538 |
+
},
|
| 539 |
+
{
|
| 540 |
+
"start": 131072,
|
| 541 |
+
"input_tokens": 2048,
|
| 542 |
+
"target_tokens": 2048,
|
| 543 |
+
"token_sha256_int64le": "53fbaab0079ba77dae982f6d960b73bb0a15c32599c1b5b9b1fc5ebae01e4508",
|
| 544 |
+
"nll": 4305.268623352051,
|
| 545 |
+
"ppl": 8.184007499385592
|
| 546 |
+
},
|
| 547 |
+
{
|
| 548 |
+
"start": 133120,
|
| 549 |
+
"input_tokens": 2048,
|
| 550 |
+
"target_tokens": 2048,
|
| 551 |
+
"token_sha256_int64le": "5bdbdf09ceef09d6aa431db8277b89196813a7a38af6c6273ecb7f8e14d6a706",
|
| 552 |
+
"nll": 2861.4199714660645,
|
| 553 |
+
"ppl": 4.0437711940583245
|
| 554 |
+
},
|
| 555 |
+
{
|
| 556 |
+
"start": 135168,
|
| 557 |
+
"input_tokens": 2048,
|
| 558 |
+
"target_tokens": 2048,
|
| 559 |
+
"token_sha256_int64le": "c234d0ef8f6ce3509e1113c4bf0597f4be1c2a93cc7a62dffd29db050897b3df",
|
| 560 |
+
"nll": 4257.87614440918,
|
| 561 |
+
"ppl": 7.996796997032621
|
| 562 |
+
},
|
| 563 |
+
{
|
| 564 |
+
"start": 137216,
|
| 565 |
+
"input_tokens": 2048,
|
| 566 |
+
"target_tokens": 2048,
|
| 567 |
+
"token_sha256_int64le": "b464cbc80b68bae974a72bd7611c7fcd3b51915fc071646cd3aee2c819c4ca89",
|
| 568 |
+
"nll": 3488.4750175476074,
|
| 569 |
+
"ppl": 5.492353994183106
|
| 570 |
+
},
|
| 571 |
+
{
|
| 572 |
+
"start": 139264,
|
| 573 |
+
"input_tokens": 2048,
|
| 574 |
+
"target_tokens": 2048,
|
| 575 |
+
"token_sha256_int64le": "57e524a37ce26addc3be29f5865763cd6c6e43362c9093b7031f92c9a5447444",
|
| 576 |
+
"nll": 3572.947334289551,
|
| 577 |
+
"ppl": 5.723629834460816
|
| 578 |
+
},
|
| 579 |
+
{
|
| 580 |
+
"start": 141312,
|
| 581 |
+
"input_tokens": 2048,
|
| 582 |
+
"target_tokens": 2048,
|
| 583 |
+
"token_sha256_int64le": "456dd4d2463d87007d5c12aaff4b37037dff22e884aece61620063b2b8d47eb3",
|
| 584 |
+
"nll": 3527.0460052490234,
|
| 585 |
+
"ppl": 5.596774402580811
|
| 586 |
+
},
|
| 587 |
+
{
|
| 588 |
+
"start": 143360,
|
| 589 |
+
"input_tokens": 2048,
|
| 590 |
+
"target_tokens": 2048,
|
| 591 |
+
"token_sha256_int64le": "e605eedd4934713a8d1cca99200965930b02475c2ea7526ea5dd7c83c73e0c2d",
|
| 592 |
+
"nll": 3007.1626205444336,
|
| 593 |
+
"ppl": 4.342026277240921
|
| 594 |
+
},
|
| 595 |
+
{
|
| 596 |
+
"start": 145408,
|
| 597 |
+
"input_tokens": 2048,
|
| 598 |
+
"target_tokens": 2048,
|
| 599 |
+
"token_sha256_int64le": "7d67022273bc70fb56deb220e15ba87a371679cfe5bfe6c69ec2c3053eb06285",
|
| 600 |
+
"nll": 3557.816089630127,
|
| 601 |
+
"ppl": 5.681497756596443
|
| 602 |
+
},
|
| 603 |
+
{
|
| 604 |
+
"start": 147456,
|
| 605 |
+
"input_tokens": 2048,
|
| 606 |
+
"target_tokens": 2048,
|
| 607 |
+
"token_sha256_int64le": "326f43859059e3233ce0cfb7e01af3b9e5890edacad013a48b79926520aecfd6",
|
| 608 |
+
"nll": 4075.175567626953,
|
| 609 |
+
"ppl": 7.314303540247181
|
| 610 |
+
},
|
| 611 |
+
{
|
| 612 |
+
"start": 149504,
|
| 613 |
+
"input_tokens": 2048,
|
| 614 |
+
"target_tokens": 2048,
|
| 615 |
+
"token_sha256_int64le": "1f1dcb8d4ed2394e11ee7a01ab57f51f99795aff8d15fd18bb9be12229b715f2",
|
| 616 |
+
"nll": 4570.942901611328,
|
| 617 |
+
"ppl": 9.317605859571216
|
| 618 |
+
},
|
| 619 |
+
{
|
| 620 |
+
"start": 151552,
|
| 621 |
+
"input_tokens": 2048,
|
| 622 |
+
"target_tokens": 2048,
|
| 623 |
+
"token_sha256_int64le": "efcdcd786b861a83480e50dc169dbe42cbdd0ce5703be0200852c9a5a35d45dc",
|
| 624 |
+
"nll": 4159.648002624512,
|
| 625 |
+
"ppl": 7.622299732124244
|
| 626 |
+
},
|
| 627 |
+
{
|
| 628 |
+
"start": 153600,
|
| 629 |
+
"input_tokens": 2048,
|
| 630 |
+
"target_tokens": 2048,
|
| 631 |
+
"token_sha256_int64le": "0435f9685afc422dd9b3361859e2a5b8c5aeff77430443ab114adf8eac40ec6a",
|
| 632 |
+
"nll": 3725.0904083251953,
|
| 633 |
+
"ppl": 6.16502259059718
|
| 634 |
+
},
|
| 635 |
+
{
|
| 636 |
+
"start": 155648,
|
| 637 |
+
"input_tokens": 2048,
|
| 638 |
+
"target_tokens": 2048,
|
| 639 |
+
"token_sha256_int64le": "7812055c909040c52b02e4d86ce7b0008243027c6db97e16b343c8677bfa869c",
|
| 640 |
+
"nll": 4093.5602798461914,
|
| 641 |
+
"ppl": 7.380258981889792
|
| 642 |
+
},
|
| 643 |
+
{
|
| 644 |
+
"start": 157696,
|
| 645 |
+
"input_tokens": 2048,
|
| 646 |
+
"target_tokens": 2048,
|
| 647 |
+
"token_sha256_int64le": "81df4320e9b14dc3b1510df717f280150752a0f66d5be4cc3e7699044375f231",
|
| 648 |
+
"nll": 4041.5487213134766,
|
| 649 |
+
"ppl": 7.195187944772973
|
| 650 |
+
},
|
| 651 |
+
{
|
| 652 |
+
"start": 159744,
|
| 653 |
+
"input_tokens": 2048,
|
| 654 |
+
"target_tokens": 2048,
|
| 655 |
+
"token_sha256_int64le": "61756936a0aab088ae5e8596ac465e24f73203487adf4c0c78fff1491d1a027b",
|
| 656 |
+
"nll": 4010.4755859375,
|
| 657 |
+
"ppl": 7.086843466680223
|
| 658 |
+
},
|
| 659 |
+
{
|
| 660 |
+
"start": 161792,
|
| 661 |
+
"input_tokens": 2048,
|
| 662 |
+
"target_tokens": 2048,
|
| 663 |
+
"token_sha256_int64le": "17ec9a2e4d188258766d53057c8db2aa19a95c56224b72c427ef9bbd9ecab588",
|
| 664 |
+
"nll": 4301.985973358154,
|
| 665 |
+
"ppl": 8.17090021759285
|
| 666 |
+
},
|
| 667 |
+
{
|
| 668 |
+
"start": 163840,
|
| 669 |
+
"input_tokens": 2048,
|
| 670 |
+
"target_tokens": 2048,
|
| 671 |
+
"token_sha256_int64le": "2ae8b819a850b1b8400abc174d036cd916bc71a4f85301a45c325a28af686590",
|
| 672 |
+
"nll": 4039.36466217041,
|
| 673 |
+
"ppl": 7.187518833624461
|
| 674 |
+
},
|
| 675 |
+
{
|
| 676 |
+
"start": 165888,
|
| 677 |
+
"input_tokens": 2048,
|
| 678 |
+
"target_tokens": 2048,
|
| 679 |
+
"token_sha256_int64le": "075d9bf13dd08dca31814252bf3dfae2cd0105b5c292104fdeffe4e31aa3aa06",
|
| 680 |
+
"nll": 4992.637908935547,
|
| 681 |
+
"ppl": 11.447959207623606
|
| 682 |
+
},
|
| 683 |
+
{
|
| 684 |
+
"start": 167936,
|
| 685 |
+
"input_tokens": 2048,
|
| 686 |
+
"target_tokens": 2048,
|
| 687 |
+
"token_sha256_int64le": "033950cf80d4d6474d828d07d37130bf6195302e6edfd28dad24cb9d8c96350d",
|
| 688 |
+
"nll": 3591.561849594116,
|
| 689 |
+
"ppl": 5.7758897243992395
|
| 690 |
+
},
|
| 691 |
+
{
|
| 692 |
+
"start": 169984,
|
| 693 |
+
"input_tokens": 2048,
|
| 694 |
+
"target_tokens": 2048,
|
| 695 |
+
"token_sha256_int64le": "994ac15313bb5f36f32f68aa4539d765a0e97c57cdf66085322b084221a5f627",
|
| 696 |
+
"nll": 3794.161666870117,
|
| 697 |
+
"ppl": 6.376491352940107
|
| 698 |
+
},
|
| 699 |
+
{
|
| 700 |
+
"start": 172032,
|
| 701 |
+
"input_tokens": 2048,
|
| 702 |
+
"target_tokens": 2048,
|
| 703 |
+
"token_sha256_int64le": "b7cf37e2d3ae4533b02c832a041399d763ff8280f4baa7d21b74870aa97f0acf",
|
| 704 |
+
"nll": 4035.235420227051,
|
| 705 |
+
"ppl": 7.173041731835192
|
| 706 |
+
},
|
| 707 |
+
{
|
| 708 |
+
"start": 174080,
|
| 709 |
+
"input_tokens": 2048,
|
| 710 |
+
"target_tokens": 2048,
|
| 711 |
+
"token_sha256_int64le": "82ca3f6ee68359b0d695bc304b36b96eced80e978228697fbe648e91bea4c191",
|
| 712 |
+
"nll": 3442.7013244628906,
|
| 713 |
+
"ppl": 5.370959154570809
|
| 714 |
+
},
|
| 715 |
+
{
|
| 716 |
+
"start": 176128,
|
| 717 |
+
"input_tokens": 2048,
|
| 718 |
+
"target_tokens": 2048,
|
| 719 |
+
"token_sha256_int64le": "77095ab5a4ceffb0d157e5ca439d1e77fb930046e06f494f66ed1ec652dfe2f0",
|
| 720 |
+
"nll": 3652.5721855163574,
|
| 721 |
+
"ppl": 5.950543217560218
|
| 722 |
+
},
|
| 723 |
+
{
|
| 724 |
+
"start": 178176,
|
| 725 |
+
"input_tokens": 2048,
|
| 726 |
+
"target_tokens": 2048,
|
| 727 |
+
"token_sha256_int64le": "860303bd273211d412d6ae9b6601861e3a02e88dca9d49e59424550e52b6da25",
|
| 728 |
+
"nll": 3139.655143737793,
|
| 729 |
+
"ppl": 4.632213067092136
|
| 730 |
+
},
|
| 731 |
+
{
|
| 732 |
+
"start": 180224,
|
| 733 |
+
"input_tokens": 2048,
|
| 734 |
+
"target_tokens": 2048,
|
| 735 |
+
"token_sha256_int64le": "ff2154ce3f53bef7bfaf3bbc5e80c89db57da2ca9db5452df04e24ae798327e0",
|
| 736 |
+
"nll": 3548.070453643799,
|
| 737 |
+
"ppl": 5.6545259421365905
|
| 738 |
+
},
|
| 739 |
+
{
|
| 740 |
+
"start": 182272,
|
| 741 |
+
"input_tokens": 2048,
|
| 742 |
+
"target_tokens": 2048,
|
| 743 |
+
"token_sha256_int64le": "205014bcf6efc0ae1b54c5fde5bbbe6eef5cca032624f9209a0fbfb63f12d0b4",
|
| 744 |
+
"nll": 4230.248916625977,
|
| 745 |
+
"ppl": 7.889645699327152
|
| 746 |
+
},
|
| 747 |
+
{
|
| 748 |
+
"start": 184320,
|
| 749 |
+
"input_tokens": 2048,
|
| 750 |
+
"target_tokens": 2048,
|
| 751 |
+
"token_sha256_int64le": "5c93807a4025356a713b59385ae3b9026d317d5895ea297ecb81ece4bea5a24b",
|
| 752 |
+
"nll": 4224.665458679199,
|
| 753 |
+
"ppl": 7.868165469469171
|
| 754 |
+
},
|
| 755 |
+
{
|
| 756 |
+
"start": 186368,
|
| 757 |
+
"input_tokens": 2048,
|
| 758 |
+
"target_tokens": 2048,
|
| 759 |
+
"token_sha256_int64le": "a540fbca609950b4f89ee7aacf16647200ac4662aed4a435d34fa5a591506767",
|
| 760 |
+
"nll": 4899.46036529541,
|
| 761 |
+
"ppl": 10.938783917003315
|
| 762 |
+
},
|
| 763 |
+
{
|
| 764 |
+
"start": 188416,
|
| 765 |
+
"input_tokens": 2048,
|
| 766 |
+
"target_tokens": 2048,
|
| 767 |
+
"token_sha256_int64le": "def9d1107a3f9217e45385d23ad448d07071e08afc3f8fdc4d215719dda5e676",
|
| 768 |
+
"nll": 5703.1156005859375,
|
| 769 |
+
"ppl": 16.195354054511498
|
| 770 |
+
},
|
| 771 |
+
{
|
| 772 |
+
"start": 190464,
|
| 773 |
+
"input_tokens": 2048,
|
| 774 |
+
"target_tokens": 2048,
|
| 775 |
+
"token_sha256_int64le": "f85f70be3d7c0aaf1f1672276af223b53c8babb4665b6669d9c50500e26ce06c",
|
| 776 |
+
"nll": 5362.338844299316,
|
| 777 |
+
"ppl": 13.712797402896241
|
| 778 |
+
},
|
| 779 |
+
{
|
| 780 |
+
"start": 192512,
|
| 781 |
+
"input_tokens": 2048,
|
| 782 |
+
"target_tokens": 2048,
|
| 783 |
+
"token_sha256_int64le": "9f149a2ac78574d25c8884f3eb8f885354612ca1a0a7c52d3c2e4a9c87681d6d",
|
| 784 |
+
"nll": 4913.429962158203,
|
| 785 |
+
"ppl": 11.013653426480516
|
| 786 |
+
},
|
| 787 |
+
{
|
| 788 |
+
"start": 194560,
|
| 789 |
+
"input_tokens": 2048,
|
| 790 |
+
"target_tokens": 2048,
|
| 791 |
+
"token_sha256_int64le": "31f093666f3973d82cd9f91b24eeee8d8f85ea118a69c23466b8fee222e8b095",
|
| 792 |
+
"nll": 5117.261344909668,
|
| 793 |
+
"ppl": 12.166214003898428
|
| 794 |
+
},
|
| 795 |
+
{
|
| 796 |
+
"start": 196608,
|
| 797 |
+
"input_tokens": 2048,
|
| 798 |
+
"target_tokens": 2048,
|
| 799 |
+
"token_sha256_int64le": "30cf2b9c3370fee885bec22db9331bd4102cf9692790988fae9b6fae6546e633",
|
| 800 |
+
"nll": 5244.8347244262695,
|
| 801 |
+
"ppl": 12.948169868804177
|
| 802 |
+
},
|
| 803 |
+
{
|
| 804 |
+
"start": 198656,
|
| 805 |
+
"input_tokens": 2048,
|
| 806 |
+
"target_tokens": 2048,
|
| 807 |
+
"token_sha256_int64le": "b2137f528db02ad66f5baba2608cd170f1ea8da93d055f5cd5bd663cd1cbf38f",
|
| 808 |
+
"nll": 4298.105186462402,
|
| 809 |
+
"ppl": 8.155431712697306
|
| 810 |
+
},
|
| 811 |
+
{
|
| 812 |
+
"start": 200704,
|
| 813 |
+
"input_tokens": 2048,
|
| 814 |
+
"target_tokens": 2048,
|
| 815 |
+
"token_sha256_int64le": "8617a5ef9748481a079ebace75ffecf5d73dbc71fc0f31277ed397a8b2117214",
|
| 816 |
+
"nll": 4386.3925857543945,
|
| 817 |
+
"ppl": 8.514692960155594
|
| 818 |
+
},
|
| 819 |
+
{
|
| 820 |
+
"start": 202752,
|
| 821 |
+
"input_tokens": 2048,
|
| 822 |
+
"target_tokens": 2048,
|
| 823 |
+
"token_sha256_int64le": "119e062f2d3c08a3edffd75b84f2ae1bdfbf8df8921b4bbaaedb4f0d8d49e622",
|
| 824 |
+
"nll": 4348.884414672852,
|
| 825 |
+
"ppl": 8.3601696345259
|
| 826 |
+
},
|
| 827 |
+
{
|
| 828 |
+
"start": 204800,
|
| 829 |
+
"input_tokens": 2048,
|
| 830 |
+
"target_tokens": 2048,
|
| 831 |
+
"token_sha256_int64le": "219c6bee49173e63ab46fba96fb2eebf64b936c66625821dd0ab3887e0c3fe84",
|
| 832 |
+
"nll": 4481.14599609375,
|
| 833 |
+
"ppl": 8.917891755755335
|
| 834 |
+
},
|
| 835 |
+
{
|
| 836 |
+
"start": 206848,
|
| 837 |
+
"input_tokens": 2048,
|
| 838 |
+
"target_tokens": 2048,
|
| 839 |
+
"token_sha256_int64le": "8fd9219793088ab936c3a2a3591cdb99b0494706f761f7a0a8477f9a9517e948",
|
| 840 |
+
"nll": 3393.9326362609863,
|
| 841 |
+
"ppl": 5.244572173364035
|
| 842 |
+
},
|
| 843 |
+
{
|
| 844 |
+
"start": 208896,
|
| 845 |
+
"input_tokens": 2048,
|
| 846 |
+
"target_tokens": 2048,
|
| 847 |
+
"token_sha256_int64le": "956e5ccaf62e8a667c920d83a905e3a5a5c5370f4c4a392c88db1883330f36a0",
|
| 848 |
+
"nll": 4470.857444763184,
|
| 849 |
+
"ppl": 8.873203228166856
|
| 850 |
+
},
|
| 851 |
+
{
|
| 852 |
+
"start": 210944,
|
| 853 |
+
"input_tokens": 2048,
|
| 854 |
+
"target_tokens": 2048,
|
| 855 |
+
"token_sha256_int64le": "e0c9b5ceaf6f8f8ca27d14e0cba95c7069cf85bb6e82a275fa825fa2af9874c9",
|
| 856 |
+
"nll": 4595.567184448242,
|
| 857 |
+
"ppl": 9.430313011752315
|
| 858 |
+
},
|
| 859 |
+
{
|
| 860 |
+
"start": 212992,
|
| 861 |
+
"input_tokens": 2048,
|
| 862 |
+
"target_tokens": 2048,
|
| 863 |
+
"token_sha256_int64le": "65db3f6e716e7d8b73ca309b59c4d3b6fdef7dae19c3d55afe6a2444b42c574f",
|
| 864 |
+
"nll": 4150.576377868652,
|
| 865 |
+
"ppl": 7.588611389284182
|
| 866 |
+
},
|
| 867 |
+
{
|
| 868 |
+
"start": 215040,
|
| 869 |
+
"input_tokens": 2048,
|
| 870 |
+
"target_tokens": 2048,
|
| 871 |
+
"token_sha256_int64le": "a549a37af9e6d60435c3a8b1120e8626444ae5a2a19af4d07a065cb5b507dd5e",
|
| 872 |
+
"nll": 4211.133659362793,
|
| 873 |
+
"ppl": 7.816349323371917
|
| 874 |
+
},
|
| 875 |
+
{
|
| 876 |
+
"start": 217088,
|
| 877 |
+
"input_tokens": 2048,
|
| 878 |
+
"target_tokens": 2048,
|
| 879 |
+
"token_sha256_int64le": "24c630974ce5a2b74bc1e865952e1996cacc28c84402c1d82e1a434acf2a407d",
|
| 880 |
+
"nll": 4007.637451171875,
|
| 881 |
+
"ppl": 7.0770292642436665
|
| 882 |
+
},
|
| 883 |
+
{
|
| 884 |
+
"start": 219136,
|
| 885 |
+
"input_tokens": 2048,
|
| 886 |
+
"target_tokens": 2048,
|
| 887 |
+
"token_sha256_int64le": "c6b4a33d822aa0b404b9c270ff6fa15feff495147a42ecf2e2c75eaab2c706a0",
|
| 888 |
+
"nll": 5005.241058349609,
|
| 889 |
+
"ppl": 11.51862580650079
|
| 890 |
+
},
|
| 891 |
+
{
|
| 892 |
+
"start": 221184,
|
| 893 |
+
"input_tokens": 2048,
|
| 894 |
+
"target_tokens": 2048,
|
| 895 |
+
"token_sha256_int64le": "16a348f3f3a2f91aa52792912ad664b3584734cb7d6a7059db39273ac357d81f",
|
| 896 |
+
"nll": 3859.510673522949,
|
| 897 |
+
"ppl": 6.583237832091985
|
| 898 |
+
},
|
| 899 |
+
{
|
| 900 |
+
"start": 223232,
|
| 901 |
+
"input_tokens": 2048,
|
| 902 |
+
"target_tokens": 2048,
|
| 903 |
+
"token_sha256_int64le": "cce3c564bb6bb43e222444ca912ed084f09a60a2b9ff37b827ce2aa6bbde6621",
|
| 904 |
+
"nll": 3966.062240600586,
|
| 905 |
+
"ppl": 6.93481119477803
|
| 906 |
+
},
|
| 907 |
+
{
|
| 908 |
+
"start": 225280,
|
| 909 |
+
"input_tokens": 2048,
|
| 910 |
+
"target_tokens": 2048,
|
| 911 |
+
"token_sha256_int64le": "6a65cef2c5163831329dd01c22f4b581fa1e3f65a6aa344150b7351bb88bbf2d",
|
| 912 |
+
"nll": 3660.3453483581543,
|
| 913 |
+
"ppl": 5.973171358138797
|
| 914 |
+
},
|
| 915 |
+
{
|
| 916 |
+
"start": 227328,
|
| 917 |
+
"input_tokens": 2048,
|
| 918 |
+
"target_tokens": 2048,
|
| 919 |
+
"token_sha256_int64le": "16a0dee22ed67df3c33d86b8342cfd4d2c3355ca6115d2178fb8a27c73e2961c",
|
| 920 |
+
"nll": 3551.7287216186523,
|
| 921 |
+
"ppl": 5.664635442768009
|
| 922 |
+
},
|
| 923 |
+
{
|
| 924 |
+
"start": 229376,
|
| 925 |
+
"input_tokens": 2048,
|
| 926 |
+
"target_tokens": 2048,
|
| 927 |
+
"token_sha256_int64le": "91de99f25597d4261eed3a7a201ca425d91bcd92ae90e59d8f499f1b6f8513f6",
|
| 928 |
+
"nll": 3634.101303100586,
|
| 929 |
+
"ppl": 5.897116643142605
|
| 930 |
+
},
|
| 931 |
+
{
|
| 932 |
+
"start": 231424,
|
| 933 |
+
"input_tokens": 2048,
|
| 934 |
+
"target_tokens": 2048,
|
| 935 |
+
"token_sha256_int64le": "475eaef6ca5dca8059ced50fb2c9f16729a4881dc494f6b8e58868f2cff583e1",
|
| 936 |
+
"nll": 3649.985809326172,
|
| 937 |
+
"ppl": 5.943033144652248
|
| 938 |
+
},
|
| 939 |
+
{
|
| 940 |
+
"start": 233472,
|
| 941 |
+
"input_tokens": 2048,
|
| 942 |
+
"target_tokens": 2048,
|
| 943 |
+
"token_sha256_int64le": "23eca1cf7560b0090dcb8a656167c8a9841d6ff24637b21b6e86663854e819d9",
|
| 944 |
+
"nll": 4313.483451843262,
|
| 945 |
+
"ppl": 8.21690067984196
|
| 946 |
+
},
|
| 947 |
+
{
|
| 948 |
+
"start": 235520,
|
| 949 |
+
"input_tokens": 2048,
|
| 950 |
+
"target_tokens": 2048,
|
| 951 |
+
"token_sha256_int64le": "98e388fba1b866cdd04fee868ce0ce6ef963302fec97146a792636baabbd7ba8",
|
| 952 |
+
"nll": 4297.187473297119,
|
| 953 |
+
"ppl": 8.151778065028449
|
| 954 |
+
},
|
| 955 |
+
{
|
| 956 |
+
"start": 237568,
|
| 957 |
+
"input_tokens": 2048,
|
| 958 |
+
"target_tokens": 2048,
|
| 959 |
+
"token_sha256_int64le": "789d2a3c1b8f7dee0dba62f3fa7ebfabe30f9c472b505aafee1043632b617593",
|
| 960 |
+
"nll": 4405.434730529785,
|
| 961 |
+
"ppl": 8.594231110065234
|
| 962 |
+
},
|
| 963 |
+
{
|
| 964 |
+
"start": 239616,
|
| 965 |
+
"input_tokens": 2048,
|
| 966 |
+
"target_tokens": 2048,
|
| 967 |
+
"token_sha256_int64le": "e0952a12d1d8d0ad826249adba609c357d311c02052dfd73bcc4118a5a2321e6",
|
| 968 |
+
"nll": 4575.135513305664,
|
| 969 |
+
"ppl": 9.336700155038054
|
| 970 |
+
},
|
| 971 |
+
{
|
| 972 |
+
"start": 241664,
|
| 973 |
+
"input_tokens": 2048,
|
| 974 |
+
"target_tokens": 2048,
|
| 975 |
+
"token_sha256_int64le": "7e4db6466013f514b43fe7187a5506493340098c9d6e9cb19e2de44c1b25aaa5",
|
| 976 |
+
"nll": 3534.9656143188477,
|
| 977 |
+
"ppl": 5.618459010387292
|
| 978 |
+
},
|
| 979 |
+
{
|
| 980 |
+
"start": 243712,
|
| 981 |
+
"input_tokens": 2048,
|
| 982 |
+
"target_tokens": 2048,
|
| 983 |
+
"token_sha256_int64le": "cabbd779acfeab157a23dccccacdc8eab0e4e2462d78450747ad971242481f12",
|
| 984 |
+
"nll": 3865.972499847412,
|
| 985 |
+
"ppl": 6.604041992547408
|
| 986 |
+
},
|
| 987 |
+
{
|
| 988 |
+
"start": 245760,
|
| 989 |
+
"input_tokens": 2048,
|
| 990 |
+
"target_tokens": 2048,
|
| 991 |
+
"token_sha256_int64le": "94cc1da9c6faa265aae847ab2608dc502d9af48b4745cfe3507f980c5353e3a3",
|
| 992 |
+
"nll": 4207.743709564209,
|
| 993 |
+
"ppl": 7.803422021459147
|
| 994 |
+
},
|
| 995 |
+
{
|
| 996 |
+
"start": 247808,
|
| 997 |
+
"input_tokens": 2048,
|
| 998 |
+
"target_tokens": 2048,
|
| 999 |
+
"token_sha256_int64le": "d7e70afddb18ab1e802433e1e1f5fc210cb01f95d54efcfa7aef331e7b289ee2",
|
| 1000 |
+
"nll": 4111.176368713379,
|
| 1001 |
+
"ppl": 7.444014869448214
|
| 1002 |
+
},
|
| 1003 |
+
{
|
| 1004 |
+
"start": 249856,
|
| 1005 |
+
"input_tokens": 2048,
|
| 1006 |
+
"target_tokens": 2048,
|
| 1007 |
+
"token_sha256_int64le": "aa828523335b939e8d0dbfc231fabd2e7e491c08ed1c7a0eef1562c6a4f1c2d4",
|
| 1008 |
+
"nll": 3629.9711685180664,
|
| 1009 |
+
"ppl": 5.885236104555122
|
| 1010 |
+
},
|
| 1011 |
+
{
|
| 1012 |
+
"start": 251904,
|
| 1013 |
+
"input_tokens": 2048,
|
| 1014 |
+
"target_tokens": 2048,
|
| 1015 |
+
"token_sha256_int64le": "160c9f18c8aa5fa6e564a28a5d392bcc036fefdb935ed9f1b1df7bc8d3fb78d4",
|
| 1016 |
+
"nll": 3652.4384384155273,
|
| 1017 |
+
"ppl": 5.950154622874318
|
| 1018 |
+
},
|
| 1019 |
+
{
|
| 1020 |
+
"start": 253952,
|
| 1021 |
+
"input_tokens": 2048,
|
| 1022 |
+
"target_tokens": 2048,
|
| 1023 |
+
"token_sha256_int64le": "b73ede3b386dc168dde9de45a9b65cd026d9bdce7abaf99a17642d6d8f3a88e0",
|
| 1024 |
+
"nll": 3627.9281311035156,
|
| 1025 |
+
"ppl": 5.879368056587513
|
| 1026 |
+
},
|
| 1027 |
+
{
|
| 1028 |
+
"start": 256000,
|
| 1029 |
+
"input_tokens": 2048,
|
| 1030 |
+
"target_tokens": 2048,
|
| 1031 |
+
"token_sha256_int64le": "24d9589ab416fd91e908b5226dbfa7c06409f330a1c7a4bd9a1e021646601822",
|
| 1032 |
+
"nll": 3834.0649490356445,
|
| 1033 |
+
"ppl": 6.501949310700498
|
| 1034 |
+
},
|
| 1035 |
+
{
|
| 1036 |
+
"start": 258048,
|
| 1037 |
+
"input_tokens": 2048,
|
| 1038 |
+
"target_tokens": 2048,
|
| 1039 |
+
"token_sha256_int64le": "9ab8aec5d86c9ee1a0a29b35cb7dde4e8a927f0760cf843a0a1a1cc592f11ddc",
|
| 1040 |
+
"nll": 4115.118362426758,
|
| 1041 |
+
"ppl": 7.458356919685115
|
| 1042 |
+
},
|
| 1043 |
+
{
|
| 1044 |
+
"start": 260096,
|
| 1045 |
+
"input_tokens": 2048,
|
| 1046 |
+
"target_tokens": 2048,
|
| 1047 |
+
"token_sha256_int64le": "5c921b66917fdb0f1fc83be2aedab3c41b8f1283368d67596617f415ca0f04f8",
|
| 1048 |
+
"nll": 3714.1204681396484,
|
| 1049 |
+
"ppl": 6.132088447528145
|
| 1050 |
+
},
|
| 1051 |
+
{
|
| 1052 |
+
"start": 262144,
|
| 1053 |
+
"input_tokens": 2048,
|
| 1054 |
+
"target_tokens": 2048,
|
| 1055 |
+
"token_sha256_int64le": "205de0d3f69598cc0383e171a574a31198e7723fbad128f324710843fe6ff569",
|
| 1056 |
+
"nll": 3916.031120300293,
|
| 1057 |
+
"ppl": 6.767451463344684
|
| 1058 |
+
},
|
| 1059 |
+
{
|
| 1060 |
+
"start": 264192,
|
| 1061 |
+
"input_tokens": 2048,
|
| 1062 |
+
"target_tokens": 2048,
|
| 1063 |
+
"token_sha256_int64le": "384ceca0ff3a4e3408e82053069067cc17dfe752f9b5daece51e04cafab4bb3b",
|
| 1064 |
+
"nll": 3597.432357788086,
|
| 1065 |
+
"ppl": 5.7924698276806845
|
| 1066 |
+
},
|
| 1067 |
+
{
|
| 1068 |
+
"start": 266240,
|
| 1069 |
+
"input_tokens": 2048,
|
| 1070 |
+
"target_tokens": 2048,
|
| 1071 |
+
"token_sha256_int64le": "bce567c974d90531fd5cc7659ca7e07304a70dfae98a8ceecbb42d5663a0c3b7",
|
| 1072 |
+
"nll": 3693.156410217285,
|
| 1073 |
+
"ppl": 6.069638380537006
|
| 1074 |
+
},
|
| 1075 |
+
{
|
| 1076 |
+
"start": 268288,
|
| 1077 |
+
"input_tokens": 2048,
|
| 1078 |
+
"target_tokens": 2048,
|
| 1079 |
+
"token_sha256_int64le": "887e1cf68ef3a7a19c44fc7df1fcb38786a2a904189ad40d3b21a5eb7d09046d",
|
| 1080 |
+
"nll": 3560.2379035949707,
|
| 1081 |
+
"ppl": 5.688220251381439
|
| 1082 |
+
},
|
| 1083 |
+
{
|
| 1084 |
+
"start": 270336,
|
| 1085 |
+
"input_tokens": 2048,
|
| 1086 |
+
"target_tokens": 2048,
|
| 1087 |
+
"token_sha256_int64le": "278594634f7f30199be67eac18d43ca49b972dc8278bae7a857ff8b5d7026140",
|
| 1088 |
+
"nll": 3993.075340270996,
|
| 1089 |
+
"ppl": 7.026887191426562
|
| 1090 |
+
},
|
| 1091 |
+
{
|
| 1092 |
+
"start": 272384,
|
| 1093 |
+
"input_tokens": 2048,
|
| 1094 |
+
"target_tokens": 2048,
|
| 1095 |
+
"token_sha256_int64le": "86c7c332b1436eac3225fa316bdd62402521144b36fff138763f645677f79a93",
|
| 1096 |
+
"nll": 4041.94930267334,
|
| 1097 |
+
"ppl": 7.1965954350412265
|
| 1098 |
+
},
|
| 1099 |
+
{
|
| 1100 |
+
"start": 274432,
|
| 1101 |
+
"input_tokens": 2048,
|
| 1102 |
+
"target_tokens": 2048,
|
| 1103 |
+
"token_sha256_int64le": "4873332acc6a1221452ccf6ab2bc94099dcccd456f5b6ba321958ea8f9c60cef",
|
| 1104 |
+
"nll": 4250.7194900512695,
|
| 1105 |
+
"ppl": 7.968901277340227
|
| 1106 |
+
},
|
| 1107 |
+
{
|
| 1108 |
+
"start": 276480,
|
| 1109 |
+
"input_tokens": 2048,
|
| 1110 |
+
"target_tokens": 2048,
|
| 1111 |
+
"token_sha256_int64le": "017db0f35d2e651d10a7f095493247ef38a8aff669f5e729f854df00f2c3e715",
|
| 1112 |
+
"nll": 4778.883529663086,
|
| 1113 |
+
"ppl": 10.313350525586598
|
| 1114 |
+
},
|
| 1115 |
+
{
|
| 1116 |
+
"start": 278528,
|
| 1117 |
+
"input_tokens": 2048,
|
| 1118 |
+
"target_tokens": 2048,
|
| 1119 |
+
"token_sha256_int64le": "579f6cada791a9b5110c0e1908ed8ae1747c354821a9ea59a1115b467c861a0a",
|
| 1120 |
+
"nll": 3828.751808166504,
|
| 1121 |
+
"ppl": 6.485103119212715
|
| 1122 |
+
},
|
| 1123 |
+
{
|
| 1124 |
+
"start": 280576,
|
| 1125 |
+
"input_tokens": 2048,
|
| 1126 |
+
"target_tokens": 2048,
|
| 1127 |
+
"token_sha256_int64le": "588bc423b576ea870c073d6e32cf95544b2c5e0ba4710892fe8731279da28a38",
|
| 1128 |
+
"nll": 3394.098846435547,
|
| 1129 |
+
"ppl": 5.24499782601533
|
| 1130 |
+
},
|
| 1131 |
+
{
|
| 1132 |
+
"start": 282624,
|
| 1133 |
+
"input_tokens": 2048,
|
| 1134 |
+
"target_tokens": 2048,
|
| 1135 |
+
"token_sha256_int64le": "b43354340e3257bf28d75c85570e060a9f70db695fd34326d61e4d9b50569044",
|
| 1136 |
+
"nll": 2891.138883113861,
|
| 1137 |
+
"ppl": 4.102878939941523
|
| 1138 |
+
},
|
| 1139 |
+
{
|
| 1140 |
+
"start": 284672,
|
| 1141 |
+
"input_tokens": 2048,
|
| 1142 |
+
"target_tokens": 2048,
|
| 1143 |
+
"token_sha256_int64le": "3b33a66548afb24214097359a4c10380e0cefcde0292a0773bbfc78d745b11b5",
|
| 1144 |
+
"nll": 3884.6475734710693,
|
| 1145 |
+
"ppl": 6.6645375940306
|
| 1146 |
+
},
|
| 1147 |
+
{
|
| 1148 |
+
"start": 286720,
|
| 1149 |
+
"input_tokens": 2048,
|
| 1150 |
+
"target_tokens": 2048,
|
| 1151 |
+
"token_sha256_int64le": "eb6ed0c741a4746f386fea624aae9629739463bb40e476d73df622dd264ea6d7",
|
| 1152 |
+
"nll": 4480.775253295898,
|
| 1153 |
+
"ppl": 8.916277524847523
|
| 1154 |
+
},
|
| 1155 |
+
{
|
| 1156 |
+
"start": 288768,
|
| 1157 |
+
"input_tokens": 2048,
|
| 1158 |
+
"target_tokens": 2048,
|
| 1159 |
+
"token_sha256_int64le": "cb791a719554b5899c11832912f980c09146c458b4aef527a3d5d0d406fd0fb2",
|
| 1160 |
+
"nll": 5025.520317077637,
|
| 1161 |
+
"ppl": 11.633249594374545
|
| 1162 |
+
},
|
| 1163 |
+
{
|
| 1164 |
+
"start": 290816,
|
| 1165 |
+
"input_tokens": 2048,
|
| 1166 |
+
"target_tokens": 2048,
|
| 1167 |
+
"token_sha256_int64le": "e7c568baaa1211ddf979a3cc4ebb53c6a45e42464dee6a4c5c7afe10fac4db8e",
|
| 1168 |
+
"nll": 4053.0142822265625,
|
| 1169 |
+
"ppl": 7.23558258500332
|
| 1170 |
+
},
|
| 1171 |
+
{
|
| 1172 |
+
"start": 292864,
|
| 1173 |
+
"input_tokens": 2048,
|
| 1174 |
+
"target_tokens": 2048,
|
| 1175 |
+
"token_sha256_int64le": "bbe3a070e9975e49aca0b68eaf924eea9d3c47fad52a4b0a1748cf5bcd6e3b56",
|
| 1176 |
+
"nll": 4396.634391784668,
|
| 1177 |
+
"ppl": 8.557380582558666
|
| 1178 |
+
},
|
| 1179 |
+
{
|
| 1180 |
+
"start": 294912,
|
| 1181 |
+
"input_tokens": 2048,
|
| 1182 |
+
"target_tokens": 2048,
|
| 1183 |
+
"token_sha256_int64le": "85f551396ba9f11283f03cead2c53f29ce2abedf4051c45bc16df0f34c048088",
|
| 1184 |
+
"nll": 4129.445701599121,
|
| 1185 |
+
"ppl": 7.510716812698255
|
| 1186 |
+
},
|
| 1187 |
+
{
|
| 1188 |
+
"start": 296960,
|
| 1189 |
+
"input_tokens": 2048,
|
| 1190 |
+
"target_tokens": 2048,
|
| 1191 |
+
"token_sha256_int64le": "2accf71a83660262bce4bfa2f3ce41f54f864b36b0aa6bb97447f8fb7522b4fe",
|
| 1192 |
+
"nll": 4474.984817504883,
|
| 1193 |
+
"ppl": 8.891103592163347
|
| 1194 |
+
},
|
| 1195 |
+
{
|
| 1196 |
+
"start": 299008,
|
| 1197 |
+
"input_tokens": 1955,
|
| 1198 |
+
"target_tokens": 1955,
|
| 1199 |
+
"token_sha256_int64le": "4da41d914b86530c1a9d5d118b5b0d9cb2ecf64f1884239fb41e30aef78f8e27",
|
| 1200 |
+
"nll": 3835.993211746216,
|
| 1201 |
+
"ppl": 7.114570507461341
|
| 1202 |
+
}
|
| 1203 |
+
],
|
| 1204 |
+
"elapsed_seconds": 26.991049766540527,
|
| 1205 |
+
"cache_dtype": "SSD scan internal precision",
|
| 1206 |
+
"cache_bytes": null,
|
| 1207 |
+
"logits_chunk_tokens": 64,
|
| 1208 |
+
"score": "next-token cross entropy; no BOS/EOS added; each window starts from zero state"
|
| 1209 |
+
},
|
| 1210 |
+
"base_identity": {
|
| 1211 |
+
"complete": true,
|
| 1212 |
+
"tensors": 507,
|
| 1213 |
+
"parameters": 8236999680,
|
| 1214 |
+
"identity_version_gradients_unchanged": true
|
| 1215 |
+
},
|
| 1216 |
+
"native_restoration": {
|
| 1217 |
+
"complete": true,
|
| 1218 |
+
"modules": 454,
|
| 1219 |
+
"forward_methods_and_hooks_exact": true
|
| 1220 |
+
},
|
| 1221 |
+
"cuda_memory": {
|
| 1222 |
+
"peak_allocated_bytes": 16842274816,
|
| 1223 |
+
"peak_reserved_bytes": 17075011584,
|
| 1224 |
+
"scope": "Resident decoded FP16 weights plus runtime allocations; not encoded model size."
|
| 1225 |
+
}
|
| 1226 |
+
}
|
scripts/audit_recall_native_prefill_wt2_test_v1.py
ADDED
|
@@ -0,0 +1,151 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Independent CPU arithmetic and token coverage audit of frozen WT2 test runs."""
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
import argparse
|
| 5 |
+
import hashlib
|
| 6 |
+
import json
|
| 7 |
+
import math
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
import struct
|
| 10 |
+
import sys
|
| 11 |
+
|
| 12 |
+
sys.dont_write_bytecode = True
|
| 13 |
+
from run_recall_native_prefill_wt2_test_v1 import (ARMS, FORMAT, PACKAGES, PINS, PINS_SHA256,
|
| 14 |
+
PROTOCOL, need, put, sha)
|
| 15 |
+
RUNNER = Path(__file__).with_name('run_recall_native_prefill_wt2_test_v1.py')
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def close(actual, expected, label):
|
| 19 |
+
need(math.isfinite(actual) and math.isclose(actual, expected, rel_tol=1e-12, abs_tol=1e-9), label)
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def audit(directory):
|
| 23 |
+
need(sha(PINS) == PINS_SHA256, 'Frozen test pins changed')
|
| 24 |
+
pins = json.loads(PINS.read_text())
|
| 25 |
+
report = json.loads((directory / 'comparison.json').read_text())
|
| 26 |
+
need(report['format'] == FORMAT and report['complete'], 'Incomplete or unsupported paired report')
|
| 27 |
+
kind = report['model_kind']
|
| 28 |
+
fixed = pins['models'][kind]
|
| 29 |
+
need(report['release_binding'] == fixed and report['pins_sha256'] == sha(PINS) and
|
| 30 |
+
report['protocol_sha256'] == sha(PROTOCOL) and report['runner_sha256'] == sha(RUNNER),
|
| 31 |
+
'Runner/protocol/published checkpoint binding differs')
|
| 32 |
+
need(report['dataset'] == pins['test_dataset'], 'Official test dataset identity differs')
|
| 33 |
+
need(report['checkpoint_frozen_before_test'] is True and report['test_used_for_training_or_selection'] is False,
|
| 34 |
+
'Test used to modify/select the checkpoint')
|
| 35 |
+
backend = report['backend']
|
| 36 |
+
need(backend['packages'] == PACKAGES and backend['cuda_runtime'] == '12.8' and
|
| 37 |
+
backend['num_threads'] == 8 and backend['float32_matmul_precision'] == 'highest' and
|
| 38 |
+
backend['tf32_matmul'] is False, 'Recorded native prefill backend differs')
|
| 39 |
+
need(report['windowing'] == {'targets_per_full_window': 2048, 'logits_chunk_tokens': 64,
|
| 40 |
+
'state_reset_each_window': True, 'final_partial_included': True,
|
| 41 |
+
'state_requantized_each_token': False, 'execution': 'prefill',
|
| 42 |
+
'window_count': pins['window_count'], 'target_tokens': pins['target_tokens']}, 'Window protocol differs')
|
| 43 |
+
token_path = directory / 'tokens.int64le'
|
| 44 |
+
raw = token_path.read_bytes()
|
| 45 |
+
token_receipt = report['tokens_file']
|
| 46 |
+
need(token_receipt == {'file': 'tokens.int64le', 'bytes': len(raw), 'sha256': sha(token_path)} and
|
| 47 |
+
token_receipt['sha256'] == pins['test_dataset']['token_stream_sha256_int64le'] and
|
| 48 |
+
len(raw) == pins['test_dataset']['total_tokens'] * 8, 'Raw saved test tokens differ')
|
| 49 |
+
ids = [v[0] for v in struct.iter_unpack('<q', raw)]
|
| 50 |
+
need(all(0 <= token < 256000 for token in ids), 'Token ID outside original vocabulary')
|
| 51 |
+
expected_windows = []
|
| 52 |
+
for start in range(0, len(ids)-1, 2048):
|
| 53 |
+
end = min(start+2049, len(ids))
|
| 54 |
+
expected_windows.append((start, end-start-1, hashlib.sha256(raw[start*8:end*8]).hexdigest()))
|
| 55 |
+
need(len(expected_windows) == 147 and sum(n for _, n, _ in expected_windows) == 300963,
|
| 56 |
+
'Independent full test target coverage differs')
|
| 57 |
+
ledger_path = directory / 'weight_ledger.json'
|
| 58 |
+
need(report['weight_ledger'] == {'file': 'weight_ledger.json', 'sha256': sha(ledger_path)}, 'Weight ledger changed')
|
| 59 |
+
ledger = json.loads(ledger_path.read_text())
|
| 60 |
+
expected_hashes = ledger['tensor_sha256']
|
| 61 |
+
need(len(expected_hashes) == 507 and ledger['provenance'] == report['weight_provenance'], 'Weight provenance differs')
|
| 62 |
+
if kind == 'w4':
|
| 63 |
+
manifest_path = directory / 'w4_manifest.json'
|
| 64 |
+
need(sha(manifest_path) == fixed['w4_manifest_sha256'], 'Copied published W4 manifest changed')
|
| 65 |
+
manifest = json.loads(manifest_path.read_text())
|
| 66 |
+
need(expected_hashes == {n: v['decoded_sha256'] for n, v in manifest['tensors'].items()} and
|
| 67 |
+
manifest['source_checkpoint_sha256'] == fixed['source_checkpoint_sha256'] and
|
| 68 |
+
manifest['parameter_count'] == 8236999680 and manifest['tensor_count'] == 507 and
|
| 69 |
+
manifest['complete'], 'Expected W4 loaded bytes differ')
|
| 70 |
+
need(report['package_receipt']['manifest_sha256'] == fixed['w4_manifest_sha256'] and
|
| 71 |
+
report['package_receipt']['file_hashes_verified'] and report['package_receipt']['decoded_hashes_verified'],
|
| 72 |
+
'Original strict W4 loader did not verify actual files/decoded weights')
|
| 73 |
+
need(report['package_receipt']['source_checkpoint_sha256'] == fixed['source_checkpoint_sha256'] and
|
| 74 |
+
report['package_receipt']['parameter_count'] == 8236999680, 'Pinned source checkpoint differs')
|
| 75 |
+
for key in ('initial_weight_check', 'final_weight_check'):
|
| 76 |
+
check = report[key]
|
| 77 |
+
need(check['complete'] and check['actual_content_checked'] and check['tensors'] == 507 and
|
| 78 |
+
check['actual_tensor_sha256'] == expected_hashes, 'Actual loaded weight contents differ: ' + key)
|
| 79 |
+
adapter_expected = fixed['adapter']['tensor_sha256']
|
| 80 |
+
def check_adapter(receipt):
|
| 81 |
+
need(receipt == {'complete': True, 'tensor_sha256': adapter_expected,
|
| 82 |
+
'payload_bytes': 2308208, 'parameters': 1154104, 'tensor_count': 224}, 'Actual adapter content differs')
|
| 83 |
+
check_adapter(report['published_adapter_content'])
|
| 84 |
+
probe = report['adapter_removal_probe']
|
| 85 |
+
need(probe['complete'] and probe['matches_unadapted_bitwise'] and probe['tokens'] == 128 and
|
| 86 |
+
probe['hidden_sha256'] == report['unadapted_probe_hidden_sha256'], 'Adapter removal changed native hidden output')
|
| 87 |
+
need(report['final_native_restoration']['complete'] and
|
| 88 |
+
report['final_native_restoration']['forward_methods_and_hooks_exact'], 'Native methods/hooks not restored')
|
| 89 |
+
identity = report['final_base_identity']
|
| 90 |
+
need(identity['complete'] and identity['tensors'] == 507 and identity['parameters'] == 8236999680 and
|
| 91 |
+
identity['identity_version_gradients_unchanged'], 'Base identities changed')
|
| 92 |
+
rows = {}
|
| 93 |
+
for arm in ARMS:
|
| 94 |
+
arm_path = directory / (arm + '.json')
|
| 95 |
+
receipt = report['reports'][arm]
|
| 96 |
+
need(receipt['file'] == arm_path.name and receipt['sha256'] == sha(arm_path), 'Arm report changed')
|
| 97 |
+
row = json.loads(arm_path.read_text())
|
| 98 |
+
adapted = arm == 'resurface'
|
| 99 |
+
need(row['format'] == FORMAT and row['complete'] and row['arm'] == arm and row['model_kind'] == kind and
|
| 100 |
+
row['adapter_loaded'] is adapted and row['dataset'] == pins['test_dataset'] and
|
| 101 |
+
row['adapter_sha256'] == (fixed['adapter']['sha256'] if adapted else None), 'Arm checkpoint identity differs')
|
| 102 |
+
need(row['native_restoration']['complete'] and row['native_restoration']['forward_methods_and_hooks_exact'] and
|
| 103 |
+
row['base_identity']['complete'] and row['base_identity']['identity_version_gradients_unchanged'],
|
| 104 |
+
'Arm did not preserve frozen native model')
|
| 105 |
+
if adapted:
|
| 106 |
+
check_adapter(row['loaded_adapter_content'])
|
| 107 |
+
check = row['frozen_base_check']
|
| 108 |
+
need(check['actual_content_checked'] and check['identity_version_gradients_unchanged'] and
|
| 109 |
+
check['tensors'] == 507 and check['parameters'] == 8236999680, 'Adapted base not checked')
|
| 110 |
+
ppl = row['ppl']
|
| 111 |
+
need(ppl['execution'] == 'prefill' and ppl['cache_dtype'] == 'SSD scan internal precision' and
|
| 112 |
+
ppl['cache_bytes'] is None and ppl['logits_chunk_tokens'] == 64 and
|
| 113 |
+
ppl['target_tokens'] == 300963 and len(ppl['windows']) == 147, 'Arm numerical path/population differs')
|
| 114 |
+
total_nll = 0.0
|
| 115 |
+
for actual, (start, count, token_sha) in zip(ppl['windows'], expected_windows):
|
| 116 |
+
need((actual['start'], actual['input_tokens'], actual['target_tokens'], actual['token_sha256_int64le']) ==
|
| 117 |
+
(start, count, count, token_sha), 'Window token/coverage mismatch')
|
| 118 |
+
need(math.isfinite(actual['nll']) and actual['nll'] >= 0, 'Invalid window NLL')
|
| 119 |
+
close(actual['ppl'], math.exp(actual['nll']/count), 'Window PPL arithmetic mismatch')
|
| 120 |
+
total_nll += actual['nll']
|
| 121 |
+
close(ppl['nll'], total_nll, 'Aggregate NLL differs from exact full window sum')
|
| 122 |
+
close(ppl['ppl'], math.exp(total_nll/300963), 'Aggregate PPL arithmetic differs')
|
| 123 |
+
close(receipt['ppl'], ppl['ppl'], 'Arm receipt PPL differs')
|
| 124 |
+
close(report['ppl'][arm], ppl['ppl'], 'Comparison PPL differs')
|
| 125 |
+
rows[arm] = ppl
|
| 126 |
+
left, right = (rows[arm] for arm in ARMS)
|
| 127 |
+
close(report['ppl_relative_change'], right['ppl']/left['ppl']-1, 'PPL relative change differs')
|
| 128 |
+
need(report['matched_windows_improved'] == sum(r['nll'] < l['nll'] for l, r in zip(left['windows'], right['windows'])),
|
| 129 |
+
'Matched improved-window count differs')
|
| 130 |
+
return {'format': FORMAT + '_CPU_AUDIT', 'complete': True, 'passed': True,
|
| 131 |
+
'comparison_sha256': sha(directory / 'comparison.json'), 'pins_sha256': sha(PINS),
|
| 132 |
+
'protocol_sha256': sha(PROTOCOL), 'runner_sha256': sha(RUNNER), 'auditor_sha256': sha(__file__),
|
| 133 |
+
'model_kind': kind, 'split': 'test', 'ppl_execution': 'prefill',
|
| 134 |
+
'windows': 147, 'targets': 300963, 'ppl': report['ppl'],
|
| 135 |
+
'cuda_initialized': False, 'torch_imported': 'torch' in sys.modules,
|
| 136 |
+
'scope': 'Independent saved-token/hash/NLL arithmetic and frozen-checkpoint evidence audit; no GPU inference replay.'}
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
def main():
|
| 140 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 141 |
+
parser.add_argument('--output-dir', type=Path, required=True)
|
| 142 |
+
args = parser.parse_args()
|
| 143 |
+
path = args.output_dir / 'cpu_audit_v1.json'
|
| 144 |
+
need(not path.exists(), 'Preserve existing CPU audit')
|
| 145 |
+
result = audit(args.output_dir)
|
| 146 |
+
put(path, result)
|
| 147 |
+
print(json.dumps(result, indent=2))
|
| 148 |
+
|
| 149 |
+
|
| 150 |
+
if __name__ == '__main__':
|
| 151 |
+
main()
|
scripts/prepare_recall_wt2_test_inputs_v1.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""CPU helpers: assemble released runtime files or recreate WT2 test tokens."""
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
import argparse
|
| 5 |
+
import hashlib
|
| 6 |
+
import json
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
import shutil
|
| 9 |
+
import sys
|
| 10 |
+
|
| 11 |
+
sys.dont_write_bytecode = True
|
| 12 |
+
from run_recall_native_prefill_wt2_test_v1 import PINS, PINS_SHA256, need, put, sha
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def fixed_pins():
|
| 16 |
+
need(sha(PINS) == PINS_SHA256, 'Frozen test pins changed')
|
| 17 |
+
return json.loads(PINS.read_text())
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def prepare_runtime(args):
|
| 21 |
+
fixed = fixed_pins()['models'][args.model_kind]
|
| 22 |
+
need(not args.out_dir.exists(), 'Fresh runtime output required')
|
| 23 |
+
source = args.download_root.resolve()
|
| 24 |
+
planned = {}
|
| 25 |
+
for relative, expected in fixed['frozen_files'].items():
|
| 26 |
+
# The original W4 Hub bundle has code/ but keeps evidence in reports/.
|
| 27 |
+
code_candidate = source / 'code' / relative
|
| 28 |
+
path = code_candidate if code_candidate.is_file() else source / relative
|
| 29 |
+
need(path.is_file() and sha(path) == expected, 'Published runtime/evidence differs: ' + relative)
|
| 30 |
+
planned[relative] = path
|
| 31 |
+
adapter = source / fixed['adapter_path']
|
| 32 |
+
if not adapter.is_file() and args.model_kind == 'w4':
|
| 33 |
+
adapter = source / 'adapter/adapter_fp16.pt'
|
| 34 |
+
need(adapter.is_file() and sha(adapter) == fixed['adapter']['sha256'], 'Published adapter differs')
|
| 35 |
+
planned[fixed['adapter_path']] = adapter
|
| 36 |
+
args.out_dir.mkdir(parents=True)
|
| 37 |
+
for relative, path in planned.items():
|
| 38 |
+
target = args.out_dir / relative
|
| 39 |
+
target.parent.mkdir(parents=True, exist_ok=True)
|
| 40 |
+
shutil.copyfile(path, target)
|
| 41 |
+
receipt = {'complete': True, 'model_kind': args.model_kind, 'pins_sha256': sha(PINS),
|
| 42 |
+
'source': str(source), 'runtime_root': str(args.out_dir.resolve()),
|
| 43 |
+
'files': {relative: {'bytes': path.stat().st_size, 'sha256': sha(path)}
|
| 44 |
+
for relative, path in planned.items()},
|
| 45 |
+
'weight_files_copied': False, 'cuda_initialized': False, 'torch_imported': 'torch' in sys.modules}
|
| 46 |
+
put(args.out_dir / 'test_runtime_receipt.json', receipt)
|
| 47 |
+
print(json.dumps(receipt, indent=2))
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def recreate_tokens(args):
|
| 51 |
+
pins = fixed_pins()
|
| 52 |
+
release = args.release_root.resolve()
|
| 53 |
+
fixed = pins['models'][args.model_kind]
|
| 54 |
+
for relative in ('mamba2_recall/__init__.py', 'mamba2_recall/runtime.py', 'mamba2_recall/calibration.py'):
|
| 55 |
+
need(sha(release / relative) == fixed['frozen_files'][relative], 'Frozen tokenizer/data source changed')
|
| 56 |
+
need('mamba2_recall' not in sys.modules, 'A different runtime was imported')
|
| 57 |
+
sys.path.insert(0, str(release))
|
| 58 |
+
from mamba2_recall import runtime, calibration
|
| 59 |
+
import torch
|
| 60 |
+
need(not torch.cuda.is_initialized(), 'Token reconstruction must be CPU only')
|
| 61 |
+
tokenizer = runtime.SentencePieceTokenizer(args.tokenizer_source_dir)
|
| 62 |
+
ids, metadata = calibration.load_wikitext_tokens(tokenizer, 'test')
|
| 63 |
+
need(metadata == pins['test_dataset'], 'Pinned official test tokenization differs')
|
| 64 |
+
raw = ids.numpy().astype('<i8', copy=False).tobytes()
|
| 65 |
+
need(hashlib.sha256(raw).hexdigest() == metadata['token_stream_sha256_int64le'], 'Raw token identity differs')
|
| 66 |
+
args.out_dir.mkdir(parents=True, exist_ok=True)
|
| 67 |
+
path = args.out_dir / 'tokens.int64le'
|
| 68 |
+
if path.exists():
|
| 69 |
+
need(sha(path) == metadata['token_stream_sha256_int64le'], 'Existing raw token file differs')
|
| 70 |
+
else:
|
| 71 |
+
path.write_bytes(raw)
|
| 72 |
+
need(not torch.cuda.is_initialized(), 'CPU helper initialized CUDA')
|
| 73 |
+
receipt = {'complete': True, 'model_kind': args.model_kind, 'dataset': metadata,
|
| 74 |
+
'file': str(path.resolve()), 'bytes': len(raw), 'sha256': sha(path),
|
| 75 |
+
'cuda_initialized': False, 'weight_checkpoint_loaded': False}
|
| 76 |
+
print(json.dumps(receipt, indent=2))
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def main():
|
| 80 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 81 |
+
sub = parser.add_subparsers(dest='command', required=True)
|
| 82 |
+
runtime = sub.add_parser('runtime', help='Small frozen files from the original published bundle; no weights copied')
|
| 83 |
+
runtime.add_argument('--model-kind', choices=('fp16', 'w4'), required=True)
|
| 84 |
+
runtime.add_argument('--download-root', type=Path, required=True)
|
| 85 |
+
runtime.add_argument('--out-dir', type=Path, required=True)
|
| 86 |
+
tokens = sub.add_parser('tokens', help='Recreate unpublished raw tokens for the CPU evidence audit')
|
| 87 |
+
tokens.add_argument('--model-kind', choices=('fp16', 'w4'), required=True)
|
| 88 |
+
tokens.add_argument('--release-root', type=Path, required=True)
|
| 89 |
+
tokens.add_argument('--tokenizer-source-dir', type=Path, required=True)
|
| 90 |
+
tokens.add_argument('--out-dir', type=Path, required=True)
|
| 91 |
+
args = parser.parse_args()
|
| 92 |
+
(prepare_runtime if args.command == 'runtime' else recreate_tokens)(args)
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
if __name__ == '__main__':
|
| 96 |
+
main()
|
scripts/run_recall_native_prefill_wt2_test_v1.py
ADDED
|
@@ -0,0 +1,275 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Frozen published FP16/INT4 Recall checkpoints: complete official WT2 test PPL."""
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
import argparse
|
| 5 |
+
import contextlib
|
| 6 |
+
import hashlib
|
| 7 |
+
import importlib.metadata
|
| 8 |
+
import json
|
| 9 |
+
from pathlib import Path
|
| 10 |
+
import sys
|
| 11 |
+
import time
|
| 12 |
+
import traceback
|
| 13 |
+
|
| 14 |
+
sys.dont_write_bytecode = True
|
| 15 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 16 |
+
PROTOCOL = ROOT / 'docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PROTOCOL.md'
|
| 17 |
+
PINS = ROOT / 'docs/RECALL_NATIVE_PREFILL_WT2_TEST_V1_PINS.json'
|
| 18 |
+
PINS_SHA256 = '7e85a47cad0733ac386205dd91438f9df0b64c3c92706529c8109f5e428f40aa'
|
| 19 |
+
FORMAT = 'RECALL_NATIVE_PREFILL_WT2_TEST_V1'
|
| 20 |
+
ARMS = ('without_resurface', 'resurface')
|
| 21 |
+
PACKAGES = {'torch': '2.11.0+cu128', 'mamba-ssm': '2.3.2.post1',
|
| 22 |
+
'numpy': '1.26.4', 'triton': '3.6.0', 'datasets': '4.8.5',
|
| 23 |
+
'sentencepiece': '0.2.1'}
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def sha(path):
|
| 27 |
+
digest = hashlib.sha256()
|
| 28 |
+
with Path(path).open('rb') as stream:
|
| 29 |
+
for block in iter(lambda: stream.read(8 * 1024**2), b''):
|
| 30 |
+
digest.update(block)
|
| 31 |
+
return digest.hexdigest()
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def need(ok, message):
|
| 35 |
+
if not ok:
|
| 36 |
+
raise ValueError(message)
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def put(path, value):
|
| 40 |
+
path = Path(path)
|
| 41 |
+
temporary = path.with_name(path.name + '.partial')
|
| 42 |
+
temporary.write_text(json.dumps(value, indent=2, allow_nan=False) + '\n')
|
| 43 |
+
temporary.replace(path)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
def preflight(kind, release, adapter, w4_dir):
|
| 47 |
+
need(sha(PINS) == PINS_SHA256, 'Frozen test pins changed')
|
| 48 |
+
pins = json.loads(PINS.read_text())
|
| 49 |
+
fixed = pins['models'][kind]
|
| 50 |
+
for relative, expected in fixed['frozen_files'].items():
|
| 51 |
+
need(sha(release / relative) == expected, 'Released file changed: ' + relative)
|
| 52 |
+
need(sha(adapter) == fixed['adapter']['sha256'], 'Published adapter changed')
|
| 53 |
+
need(adapter.stat().st_size == fixed['adapter']['bytes'], 'Adapter size differs')
|
| 54 |
+
train_relative = next(n for n in fixed['frozen_files'] if n.endswith('_train.json'))
|
| 55 |
+
train = json.loads((release / train_relative).read_text())
|
| 56 |
+
need(train['complete'] and train['successful_updates'] == 1536 and
|
| 57 |
+
train['binding'] == fixed['adapter_binding'] and train['adapter'] == fixed['adapter'],
|
| 58 |
+
'Published training checkpoint binding differs')
|
| 59 |
+
if kind == 'w4':
|
| 60 |
+
need(w4_dir is not None, 'W4 requires --w4-dir')
|
| 61 |
+
need(sha(w4_dir / 'manifest.json') == fixed['w4_manifest_sha256'], 'Published W4 manifest differs')
|
| 62 |
+
else:
|
| 63 |
+
need(w4_dir is None, 'FP16 must not supply a W4 package')
|
| 64 |
+
return pins, fixed
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def native_snapshot(model):
|
| 68 |
+
return [(module, module.forward.__func__ if hasattr(module.forward, '__func__') else module.forward,
|
| 69 |
+
{name: tuple(getattr(module, name).items()) for name in
|
| 70 |
+
('_forward_pre_hooks', '_forward_hooks', '_backward_hooks')})
|
| 71 |
+
for module in model.modules()]
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def check_native_snapshot(snapshot):
|
| 75 |
+
for module, forward, hooks in snapshot:
|
| 76 |
+
now = module.forward.__func__ if hasattr(module.forward, '__func__') else module.forward
|
| 77 |
+
need(now is forward, 'Native forward method was replaced')
|
| 78 |
+
for name, entries in hooks.items():
|
| 79 |
+
need(tuple(getattr(module, name).items()) == entries, 'Native hook ownership was not restored')
|
| 80 |
+
return {'complete': True, 'modules': len(snapshot), 'forward_methods_and_hooks_exact': True}
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
def base_identities(model):
|
| 84 |
+
return {n: (id(p), p.data_ptr(), p._version) for n, p in model.named_parameters()}
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def check_base_identity(model, before):
|
| 88 |
+
now = dict(model.named_parameters())
|
| 89 |
+
need(set(now) == set(before), 'Base parameter inventory changed')
|
| 90 |
+
for name, p in now.items():
|
| 91 |
+
need((id(p), p.data_ptr(), p._version) == before[name] and not p.requires_grad and p.grad is None,
|
| 92 |
+
'Base identity/version/gradient changed: ' + name)
|
| 93 |
+
return {'complete': True, 'tensors': len(now), 'parameters': sum(p.numel() for p in now.values()),
|
| 94 |
+
'identity_version_gradients_unchanged': True}
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def source_ledger(runtime, source_dir, torch):
|
| 98 |
+
# Independent expected bytes from verified BF16 source, not the loaded GPU model.
|
| 99 |
+
state = runtime.load_source_state(source_dir)
|
| 100 |
+
ledger = {}
|
| 101 |
+
for name, value in state.items():
|
| 102 |
+
digest = hashlib.sha256()
|
| 103 |
+
if value.ndim == 0:
|
| 104 |
+
digest.update(value.half().contiguous().numpy().tobytes())
|
| 105 |
+
else:
|
| 106 |
+
for first in range(0, value.shape[0], 128):
|
| 107 |
+
digest.update(value[first:first+128].to(dtype=torch.float16).contiguous().numpy().tobytes())
|
| 108 |
+
ledger[name] = digest.hexdigest()
|
| 109 |
+
del state
|
| 110 |
+
return ledger
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
def check_content(model, expected, native):
|
| 114 |
+
actual = {n: native.tensor_hash(p) for n, p in model.named_parameters()}
|
| 115 |
+
need(actual == expected, 'Actual loaded FP16 weights differ from fixed source/package')
|
| 116 |
+
return {'complete': True, 'tensors': len(actual), 'actual_content_checked': True,
|
| 117 |
+
'actual_tensor_sha256': actual}
|
| 118 |
+
|
| 119 |
+
|
| 120 |
+
def adapter_receipt(values, fixed, native):
|
| 121 |
+
hashes = {name: native.tensor_hash(value) for name, value in values.items()}
|
| 122 |
+
payload = sum(value.numel() * value.element_size() for value in values.values())
|
| 123 |
+
need(hashes == fixed['adapter']['tensor_sha256'] and payload == fixed['adapter']['payload_bytes'],
|
| 124 |
+
'Actual adapter tensors differ from published export')
|
| 125 |
+
return {'complete': True, 'tensor_sha256': hashes, 'payload_bytes': payload,
|
| 126 |
+
'parameters': sum(value.numel() for value in values.values()), 'tensor_count': len(values)}
|
| 127 |
+
|
| 128 |
+
|
| 129 |
+
def main():
|
| 130 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 131 |
+
parser.add_argument('--model-kind', choices=('fp16', 'w4'), required=True)
|
| 132 |
+
parser.add_argument('--release-root', type=Path, required=True)
|
| 133 |
+
parser.add_argument('--source-dir', type=Path, required=True)
|
| 134 |
+
parser.add_argument('--w4-dir', type=Path)
|
| 135 |
+
parser.add_argument('--adapter', type=Path)
|
| 136 |
+
parser.add_argument('--out-dir', type=Path)
|
| 137 |
+
parser.add_argument('--preflight-only', action='store_true')
|
| 138 |
+
args = parser.parse_args()
|
| 139 |
+
release = args.release_root.resolve()
|
| 140 |
+
fixed = json.loads(PINS.read_text())['models'][args.model_kind]
|
| 141 |
+
adapter = args.adapter or release / fixed['adapter_path']
|
| 142 |
+
pins, fixed = preflight(args.model_kind, release, adapter, args.w4_dir)
|
| 143 |
+
if args.preflight_only:
|
| 144 |
+
print(json.dumps({'complete': True, 'model_kind': args.model_kind, 'pins_sha256': sha(PINS),
|
| 145 |
+
'verified_files': len(fixed['frozen_files']), 'adapter_sha256': sha(adapter),
|
| 146 |
+
'ppl_execution': 'prefill', 'torch_imported': 'torch' in sys.modules,
|
| 147 |
+
'cuda_initialized': False}, indent=2))
|
| 148 |
+
return
|
| 149 |
+
need(args.out_dir is not None and not args.out_dir.exists(), 'Fresh --out-dir required')
|
| 150 |
+
# Import exactly one published package per process, never the newer research runtime.
|
| 151 |
+
need('mamba2_recall' not in sys.modules, 'A different runtime was already imported')
|
| 152 |
+
sys.path.insert(0, str(release))
|
| 153 |
+
from mamba2_recall import runtime, evaluation, calibration, resurface_native as native
|
| 154 |
+
import torch
|
| 155 |
+
need(all(Path(m.__file__).resolve().parent == release / 'mamba2_recall'
|
| 156 |
+
for m in (runtime, evaluation, calibration, native)), 'Runtime import escaped frozen release')
|
| 157 |
+
packages = {name: importlib.metadata.version(name) for name in PACKAGES}
|
| 158 |
+
need(packages == PACKAGES and torch.version.cuda == '12.8', 'Recorded release backend versions required')
|
| 159 |
+
need(torch.cuda.is_available(), 'CUDA quality reference required')
|
| 160 |
+
torch.set_num_threads(8)
|
| 161 |
+
torch.backends.cuda.matmul.allow_tf32 = False
|
| 162 |
+
torch.set_float32_matmul_precision('highest')
|
| 163 |
+
args.out_dir.mkdir(parents=True)
|
| 164 |
+
started = time.time()
|
| 165 |
+
report = {'format': FORMAT, 'complete': False, 'model_kind': args.model_kind,
|
| 166 |
+
'protocol_sha256': sha(PROTOCOL), 'pins_sha256': sha(PINS), 'runner_sha256': sha(__file__),
|
| 167 |
+
'release_binding': fixed, 'reports': {}, 'checkpoint_frozen_before_test': True,
|
| 168 |
+
'test_used_for_training_or_selection': False,
|
| 169 |
+
'project_test_text_seen_in_earlier_2p7b_experiments': True,
|
| 170 |
+
'scope': 'Additional official WT2 test PPL of fixed published checkpoints; native parallel SSD prefill. '
|
| 171 |
+
'Base pretraining contamination and cross-split text duplicates not audited.',
|
| 172 |
+
'environment': runtime.environment_receipt(),
|
| 173 |
+
'backend': {'packages': packages, 'cuda_runtime': torch.version.cuda, 'num_threads': torch.get_num_threads(),
|
| 174 |
+
'float32_matmul_precision': torch.get_float32_matmul_precision(),
|
| 175 |
+
'tf32_matmul': torch.backends.cuda.matmul.allow_tf32,
|
| 176 |
+
'tf32_cudnn': torch.backends.cudnn.allow_tf32,
|
| 177 |
+
'fp16_reduced_precision_reduction': torch.backends.cuda.matmul.allow_fp16_reduced_precision_reduction,
|
| 178 |
+
'bf16_reduced_precision_reduction': torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction}}
|
| 179 |
+
path = args.out_dir / 'comparison.json'
|
| 180 |
+
put(path, report)
|
| 181 |
+
try:
|
| 182 |
+
tokenizer = runtime.SentencePieceTokenizer(args.source_dir)
|
| 183 |
+
ids, dataset = calibration.load_wikitext_tokens(tokenizer, 'test')
|
| 184 |
+
need(dataset == pins['test_dataset'], 'Pinned official WT2 test identity differs')
|
| 185 |
+
windows = evaluation.ppl_windows(ids, 2048)
|
| 186 |
+
need(len(windows) == pins['window_count'] and len(ids)-1 == pins['target_tokens'] and
|
| 187 |
+
sum(len(w)-1 for _, w in windows) == len(ids)-1, 'Full test targets omitted or repeated')
|
| 188 |
+
raw = ids.numpy().astype('<i8', copy=False).tobytes()
|
| 189 |
+
(args.out_dir / 'tokens.int64le').write_bytes(raw)
|
| 190 |
+
need(hashlib.sha256(raw).hexdigest() == dataset['token_stream_sha256_int64le'], 'Saved tokens differ')
|
| 191 |
+
payload = native.read_fp16(adapter, expected_binding=fixed['adapter_binding'])
|
| 192 |
+
need(payload['gate_mode'] == 'soft', 'Published gate mode changed')
|
| 193 |
+
report['published_adapter_content'] = adapter_receipt(payload['tensors'], fixed, native)
|
| 194 |
+
report.update(dataset=dataset, tokens_file={'file': 'tokens.int64le', 'bytes': len(raw),
|
| 195 |
+
'sha256': sha(args.out_dir / 'tokens.int64le')},
|
| 196 |
+
windowing={'targets_per_full_window': 2048, 'logits_chunk_tokens': 64,
|
| 197 |
+
'state_reset_each_window': True, 'final_partial_included': True,
|
| 198 |
+
'state_requantized_each_token': False, 'execution': 'prefill',
|
| 199 |
+
'window_count': len(windows), 'target_tokens': len(ids)-1})
|
| 200 |
+
if args.model_kind == 'fp16':
|
| 201 |
+
expected = source_ledger(runtime, args.source_dir, torch)
|
| 202 |
+
model = runtime.load_source_model(args.source_dir)
|
| 203 |
+
provenance = {'method': 'Verified pinned BF16 source cast to FP16 in bounded CPU row chunks',
|
| 204 |
+
'source_checkpoint_sha256': fixed['source_checkpoint_sha256']}
|
| 205 |
+
else:
|
| 206 |
+
from mamba2_recall.w4 import load_w4_model
|
| 207 |
+
manifest = json.loads((args.w4_dir / 'manifest.json').read_text())
|
| 208 |
+
expected = {name: entry['decoded_sha256'] for name, entry in manifest['tensors'].items()}
|
| 209 |
+
# Preserve exact manifest bytes for an independently checkable expected ledger.
|
| 210 |
+
(args.out_dir / 'w4_manifest.json').write_bytes((args.w4_dir / 'manifest.json').read_bytes())
|
| 211 |
+
model = load_w4_model(args.w4_dir)
|
| 212 |
+
provenance = {'method': 'Pinned published W4 manifest decoded FP16 hashes; original strict loader',
|
| 213 |
+
'manifest_sha256': fixed['w4_manifest_sha256'], 'manifest_file': 'w4_manifest.json'}
|
| 214 |
+
need(len(expected) == 507 and sum(p.numel() for p in model.parameters()) == 8236999680,
|
| 215 |
+
'Original 507-tensor pure Mamba2-8B inventory differs')
|
| 216 |
+
put(args.out_dir / 'weight_ledger.json', {'provenance': provenance, 'tensor_sha256': expected})
|
| 217 |
+
report.update(weight_ledger={'file': 'weight_ledger.json', 'sha256': sha(args.out_dir / 'weight_ledger.json')},
|
| 218 |
+
weight_provenance=provenance, package_receipt=model._package_receipt,
|
| 219 |
+
initial_weight_check=check_content(model, expected, native))
|
| 220 |
+
snapshot = native_snapshot(model)
|
| 221 |
+
identities = base_identities(model)
|
| 222 |
+
probe = windows[0][1][:128].cuda()[None]
|
| 223 |
+
with torch.inference_mode():
|
| 224 |
+
original_probe = native.tensor_hash(model.backbone(probe))
|
| 225 |
+
report['unadapted_probe_hidden_sha256'] = original_probe
|
| 226 |
+
put(path, report)
|
| 227 |
+
rows = {}
|
| 228 |
+
for arm in ARMS:
|
| 229 |
+
adapted = arm == 'resurface'
|
| 230 |
+
manager = (native.install_fp16(model, adapter, expected_binding=fixed['adapter_binding'],
|
| 231 |
+
expected_base_hashes=expected) if adapted else contextlib.nullcontext())
|
| 232 |
+
torch.cuda.reset_peak_memory_stats()
|
| 233 |
+
with manager as bank:
|
| 234 |
+
row = {'format': FORMAT, 'arm': arm, 'model_kind': args.model_kind, 'complete': False,
|
| 235 |
+
'adapter_loaded': adapted, 'adapter_sha256': fixed['adapter']['sha256'] if adapted else None,
|
| 236 |
+
'dataset': dataset, 'ppl': evaluation.evaluate_ppl(model, windows, execution='prefill', logits_chunk=64)}
|
| 237 |
+
if adapted:
|
| 238 |
+
row['loaded_adapter_content'] = adapter_receipt(bank.masters, fixed, native)
|
| 239 |
+
row['frozen_base_check'] = bank.assert_base_frozen(check_values=True)
|
| 240 |
+
row['base_identity'] = check_base_identity(model, identities)
|
| 241 |
+
row.update(complete=True, native_restoration=check_native_snapshot(snapshot),
|
| 242 |
+
cuda_memory={'peak_allocated_bytes': torch.cuda.max_memory_allocated(),
|
| 243 |
+
'peak_reserved_bytes': torch.cuda.max_memory_reserved(),
|
| 244 |
+
'scope': 'Resident decoded FP16 weights plus runtime allocations; not encoded model size.'})
|
| 245 |
+
arm_path = args.out_dir / (arm + '.json')
|
| 246 |
+
put(arm_path, row)
|
| 247 |
+
rows[arm] = row
|
| 248 |
+
report['reports'][arm] = {'file': arm_path.name, 'sha256': sha(arm_path), 'ppl': row['ppl']['ppl']}
|
| 249 |
+
put(path, report)
|
| 250 |
+
with torch.inference_mode():
|
| 251 |
+
restored_probe = native.tensor_hash(model.backbone(probe))
|
| 252 |
+
need(restored_probe == original_probe, 'Adapter removal failed bitwise native prefill hidden-state replay')
|
| 253 |
+
report['adapter_removal_probe'] = {'complete': True, 'hidden_sha256': restored_probe,
|
| 254 |
+
'matches_unadapted_bitwise': True, 'tokens': 128}
|
| 255 |
+
report['final_native_restoration'] = check_native_snapshot(snapshot)
|
| 256 |
+
report['final_base_identity'] = check_base_identity(model, identities)
|
| 257 |
+
report['final_weight_check'] = check_content(model, expected, native)
|
| 258 |
+
left, right = (rows[a]['ppl'] for a in ARMS)
|
| 259 |
+
need([(r['start'], r['target_tokens'], r['token_sha256_int64le']) for r in left['windows']] ==
|
| 260 |
+
[(r['start'], r['target_tokens'], r['token_sha256_int64le']) for r in right['windows']],
|
| 261 |
+
'Unmatched official test populations')
|
| 262 |
+
report.update(complete=True, ppl={a: rows[a]['ppl']['ppl'] for a in ARMS},
|
| 263 |
+
ppl_relative_change=right['ppl']/left['ppl']-1,
|
| 264 |
+
matched_windows_improved=sum(r['nll'] < l['nll'] for l, r in zip(left['windows'], right['windows'])),
|
| 265 |
+
elapsed_seconds=time.time()-started)
|
| 266 |
+
put(path, report)
|
| 267 |
+
print(json.dumps({k: report[k] for k in ('complete', 'model_kind', 'ppl', 'ppl_relative_change')}, indent=2), flush=True)
|
| 268 |
+
except Exception as error:
|
| 269 |
+
report.update(complete=False, error=str(error), traceback=traceback.format_exc(), elapsed_seconds=time.time()-started)
|
| 270 |
+
put(path, report)
|
| 271 |
+
raise
|
| 272 |
+
|
| 273 |
+
|
| 274 |
+
if __name__ == '__main__':
|
| 275 |
+
main()
|