Release Vela-1.0-Encoder-307M-Modality with frozen task and native engine evidence

New text-model release with immutable task weights, original-parent lineage where retained, independent final evaluation, source-pinned training reproduction, frozen LoRA, and FP32 ONNX GPU qualification through 32K. Model card reports source, class and long-context limitations; existing repositories remain unchanged.

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+onnx/model.onnx.data filter=lfs diff=lfs merge=lfs -text
+lora/tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 vLLM Semantic Router contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

NOTICE.md

@@ -0,0 +1,20 @@
+# Attribution and scope
+
+The new Modality task starts from a fresh adapter and prediction head on the
+MIT-licensed mmBERT encoder, then adapts to the MIT-licensed Vela Base at
+`5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114`. The original Modality task weights are
+only an evaluation baseline; they are not a parent of this new task model.
+This release's task weights are MIT licensed.
+
+Bundled Semantic Router training/evaluation code and original annotation/task
+contract code are Apache-2.0; the full license is in `reproducibility/LICENSE`.
+Global-MMLU and Aya are Apache-2.0. DiffusionDB caption metadata are CC0-1.0;
+images are not included or required. MMLU-Pro is MIT and the original category
+supplement is Apache-2.0; they are used for historical-source screening in the
+academic-retention source construction. Pinned revisions, source licenses and
+file hashes are in `reproducibility/modality-source-files.json`.
+
+Reviewed source labels are assistant-authored task judgments, not expert human
+annotations. Authored output-contract families and background text do not become
+naturally observed routing requests through their association with caption or
+academic datasets. Historical encoder-pretraining exposure remains unknown.

README.md

@@ -0,0 +1,228 @@
+---
+language:
+- en
+- zh
+- es
+- fr
+- de
+- ja
+license: mit
+library_name: transformers
+pipeline_tag: text-classification
+base_model: llm-semantic-router/Vela-1.0-Encoder-307M
+tags:
+- modernbert
+- semantic-router
+- modality-routing
+- text-only
+- long-context
+- vela
+---
+
+# Vela 1.0 Encoder 307M Modality
+
+This **text-only** classifier identifies the requested output type:
+`AR` (text/autoregressive response), `DIFFUSION` (image generation), or `BOTH`
+(explicitly requested text and image outputs). It does not consume images or
+audio, generate those outputs, identify an input file's media type, or choose
+an arbitrary inference engine. These labels describe the router's existing
+output contract, not a universal classification of generative architectures.
+
+The new task improves independent authored output-contract accuracy from
+**34.63% to 92.96%** and long-context stress from **24/72 to 54/72** correct.
+There is a regression on the small natural-request AR slice, **19/19 to 16/19**.
+The authored score is not an estimate of natural production routing accuracy.
+
+## Model and fixed lineage
+
+| Property | Value |
+|---|---|
+| Architecture | Standard `ModernBertForSequenceClassification` |
+| Shared encoder parameters | 306,939,648 |
+| Complete merged task parameters | 307,532,547 |
+| Pooling | Mean, explicitly serialized |
+| Weights / final evaluation | FP32 |
+| Maximum input capacity | 32,768 tokenizer tokens including special tokens |
+| Label IDs | `AR=0`, `DIFFUSION=1`, `BOTH=2` |
+| Final selected candidate | `modality-v6-vela-base`, step 200 |
+| Actual final parent | `llm-semantic-router/Vela-1.0-Encoder-307M` at `5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114` |
+| Merged weight SHA256 | `d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7` |
+
+The task began with a fresh LoRA and task head on the fixed original mmBERT
+encoder, then underwent short/diverse/long curricula. The original Modality
+adapter and merged task weights were not continued. After the original-parent
+candidate was selected, a direct transfer to Vela Base failed development
+retention; a bounded 200-step adaptation passed every predeclared gate while
+preserving the trained task head. The final candidate was frozen before final
+predictions. See [the freeze decision](evaluation/freeze-decision.json) and
+[migration protocol](evaluation/migration-plan.json).
+
+The public runtime mapping is `modality_mapping.json`. The generic mapping and
+HF `id2label` / `label2id` describe the same frozen IDs. Mapping files, pooling,
+tokenizer and weights are locked together in
+[the candidate receipt](evaluation/candidate-lock.json).
+
+## Independent FP32 final results
+
+The baseline is `llm-semantic-router/mmbert32k-modality-router-merged` at
+`9f05f15889d32987d29bbe2e71bff945902ff553`. Both models receive the same full input,
+fixed labels and argmax scoring with FP32 parameters/forward, without truncation.
+No final outcome was used to choose weights, steps, thresholds or a new candidate.
+
+| Final population | Rows / source groups | Old correct | Vela correct | Old macro F1 | Vela macro F1 |
+|---|---:|---:|---:|---:|---:|
+| Authored multilingual output contracts with gallery captions | 1,080 / 57 | 374 (34.63%) | 1,004 (92.96%) | 0.2216 | 0.9291 |
+| Academic text-answer / AR retention | 240 / 218 | 232 (96.67%) | 240 (100%) | 0.3277 | 0.3333 |
+| Aya original natural requests, all AR | 19 / 6 | 19 (100%) | 16 (84.21%) | 0.3333 | 0.3048 |
+| Authored long stress, EN/ZH | 72 / 6 | 24 (33.33%) | 54 (75%) | 0.2667 | 0.7395 |
+
+Macro F1 in the table uses all three contract labels. The academic and Aya slices
+contain only AR, so a perfect result has full-contract macro F1 1/3. They test
+AR retention and cannot establish DIFFUSION/BOTH performance. Aya has 15 French
+rows, two Spanish, one English and one German; the new errors are two French
+and the German request. These six correlated source groups provide limited
+natural coverage, and their regression remains in the report. Source requests
+were individually assistant-reviewed for this task, not independently annotated
+by expert human raters.
+
+The three-way output-contract examples use held-out source groups and authored
+contract families, sometimes attached to DiffusionDB captions. They are controlled
+text instructions rather than naturally observed image-generation traffic.
+Captions and language variants are correlated; row counts do not imply 1,080
+independent scenarios. Historical encoder-pretraining exposure is unknown.
+
+| Authored output-contract language | Rows | Old accuracy | Vela accuracy |
+|---|---:|---:|---:|
+| English | 180 | 33.33% | 97.78% |
+| Chinese | 180 | 32.22% | 95.00% |
+| Spanish | 180 | 35.56% | 97.22% |
+| French | 180 | 33.33% | 91.11% |
+| German | 180 | 38.89% | 93.33% |
+| Japanese | 180 | 34.44% | 83.33% |
+
+Japanese remains the weakest of these authored-language slices. These language
+scores are not natural-request or general multilingual understanding estimates.
+
+| Long stress budget | Rows | Old correct | Vela correct | Vela macro F1 |
+|---|---:|---:|---:|---:|
+| 4,096 | 18 | 6 | 15 | 0.8222 |
+| 8,192 | 18 | 6 | 15 | 0.8222 |
+| 16,384 | 18 | 6 | 13 | 0.7167 |
+| 32,768 | 18 | 6 | 11 | 0.5889 |
+
+These 72 rows reuse six held-out source payload groups across three positions
+and four lengths. Authored repeated background prose is separated from training
+backgrounds, but this is a small stress test, not a natural long-document
+benchmark. English improves to 32/36, Chinese to 22/36. At 32K, seven of 18 still
+fail; maximum position capacity does not guarantee correct task identification.
+
+Complete labels, predictions, source groups, per-class/language/length scores and
+paired regressions are available in
+[the recomputable comparison](evaluation/comparison.json) and
+[the frozen FP32 report](evaluation/final-candidate-fp32.json).
+
+## Data and training
+
+The recipe uses reviewed Aya text requests, academic text-answer examples from
+Global-MMLU, authored output contracts and CC0 DiffusionDB caption metadata.
+No images are downloaded. Source/duplicate groups and output-contract families
+are separated across training, development and final partitions. Train-only
+contract diversification adds new instruction and negative-context patterns.
+Long curricula retain parent groups and combine a marked task with independent
+background text; the 4K subset precedes the full 32K curriculum.
+
+| Stage | Maximum tokens | Steps / selected step | Learning rate |
+|---|---:|---:|---:|
+| Fresh short task | 2,048 | 600 / 100 | 3e-5 |
+| Train-only contract diversification | 2,048 | 600 / 300 | 1e-5 |
+| Diverse 4K curriculum | 4,096 | 600 / 400 | 1e-5 |
+| Diverse full-context curriculum | 32,768 | 400 / 400 | 5e-6 |
+| Vela Base adaptation | 32,768 | 200 / 200 | 2e-6 |
+
+Training uses LoRA rank 32 / alpha 64, dropout 0.05, attention/MLP projections,
+complete task head and classifier, FP32 parameters, BF16 autocast, SDPA and
+activation checkpointing. The 4K/full stages balance length and label; the final
+adaptation keeps those sampling semantics with optional token-budget microbatch
+regrouping to reduce padding. Development source-macro F1 selects the long-stage
+checkpoints. Source sampling and source-weighted metrics are separate concepts.
+The failed direct transfer and earlier context trial are retained as diagnostics;
+training steps are not chosen from final performance.
+
+All 14 frozen data files were rebuilt byte-for-byte. The public initializer
+reproduced the recorded initial adapter hash. The standalone package contains
+source revisions/hashes/licenses, train/dev/final manifests, annotation sidecars,
+training receipts, explicit export and final evaluation commands.
+
+## Use
+
+Use Transformers 4.57.6 and a compatible Torch build. FP32 is the qualified
+reference precision; any additional engine/precision artifact has separate
+qualification. Choose a conservative context budget for your device.
+
+```python
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+model_id = "llm-semantic-router/Vela-1.0-Encoder-307M-Modality"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForSequenceClassification.from_pretrained(
+    model_id, torch_dtype=torch.float32, attn_implementation="sdpa"
+).eval()
+inputs = tokenizer("Create an illustration and explain its composition.",
+                   return_tensors="pt", truncation=False)
+if inputs["input_ids"].shape[1] > 32768:
+    raise ValueError("Input exceeds the model token budget")
+with torch.inference_mode():
+    probabilities = model(**inputs).logits.softmax(-1)[0]
+label_id = int(probabilities.argmax())
+print(model.config.id2label[label_id], float(probabilities[label_id]))
+```
+
+Pin a repository revision for production. Softmax confidence is not calibrated
+risk, and this three-label head has no explicit abstain/unknown class. Negated,
+quoted or indirect image-generation instructions remain useful application-specific
+validation targets. Selecting a backend, generating an image, enforcing thresholds
+and deciding which part of a conversation to classify belong to the router.
+Forward timings in evaluation exclude HTTP, tokenization, queueing and routing;
+they are not end-to-end latency claims.
+
+## Reproduction and license
+
+Follow [the standalone guide](reproducibility/README.md) to reconstruct sources,
+initialize, train, export and score. Frozen text-free labels and predictions
+support independent metric recomputation. Weight hashes remain unchanged after
+final evaluation. The task weights are MIT licensed; copied code is Apache-2.0
+and datasets retain their source licenses. See [NOTICE](NOTICE.md) and the
+pinned source manifest for attribution and scope.
+
+## Qualified native and ONNX variants
+
+The released `onnx/model.onnx` uses FP32 query-block attention and fixes both
+input batch dimensions to **one request**. Batches larger than one are explicitly
+rejected. This matches the Router's single-request classifier interface; it is
+not a claim that all dynamic batching modes passed. The adjacent external weight
+file is required. No CK custom-op library is needed for this variant.
+
+On the recorded AMD ROCm execution provider, native interface comparisons cover
+short inputs and 4K, 8K, 16K and 32K without truncation; inputs above 32,768 tokens
+are rejected. The largest native probability/confidence difference was
+0.00000042. The ONNX-level report separately checks the complete output
+probability vector, unchanged argmax labels, and GPU execution of attention
+inside each loop. See [native AMD evidence](evaluation/engines/vela-modality-blocked-b1-native-gpu-32k-v1.json)
+and [graph/profile evidence](evaluation/engines/vela-modality-blocked-b1-onnx-gpu-v1.json).
+Candle CPU numerical evidence in the same directory covers the stated short
+probe set; long-context HF task-quality tables do not substitute for native
+engine qualification. These are equivalence tests on fixed inputs, not another
+independent accuracy test or a serving-latency benchmark.
+
+The `lora/` variant retains the exact development-selected adapter and pinned
+parent. Use its `load_adapter.py` example to restore task configuration and
+pooling before loading weights. Independent padded bilingual/Unicode CPU
+comparisons against the frozen merged checkpoint are recorded in
+`standalone-verification.json`; the adapter does not silently change lineage.
+See [ONNX reproduction](reproducibility/onnx/README.md).
+
+The [storage provenance](evaluation/engines/blocked-storage-provenance.json) binds
+the released graph and external data file to the qualified graph and the frozen
+FP32 export. All 157 retained initializer tensors match their original dtype,
+shape and bytes; graph rewriting did not alter task weights.

adapter-qualification.json

@@ -0,0 +1,58 @@
+{
+  "task": "Modality",
+  "base_model": "llm-semantic-router/Vela-1.0-Encoder-307M",
+  "base_revision": "5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114",
+  "base_weight_sha256": "ccc274d12ef6f94f54c0c23f0388527b0a925268a1332b3a6e5c8e56868efb00",
+  "adapter_weight_sha256": "52fec9bffae938380605bd1da242fcfded6c75964b89ce112de5206f14f4eb00",
+  "merged_weight_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "dtype": "float32",
+  "provider": "Torch CPU SDPA",
+  "pooling": "mean",
+  "explicit_frozen_task_config": true,
+  "probability_atol": 0.0002,
+  "probability_rtol": 0.0001,
+  "all_argmax_equal_required": true,
+  "quality_benchmark": false,
+  "long_context_qualification": false,
+  "results": [
+    {
+      "batch": 2,
+      "padded_tokens": 14,
+      "attention_mask_counts": [
+        14,
+        10
+      ],
+      "input_sha256": "5b14566354399d634cd9322d69f32fbe1a3400dcc272481d96318e8370184777",
+      "maximum_absolute_logit_difference": 1.3828277587890625e-05,
+      "maximum_absolute_probability_difference": 3.259629011154175e-09,
+      "argmax_differences": 0,
+      "passed": true
+    },
+    {
+      "batch": 2,
+      "padded_tokens": 20,
+      "attention_mask_counts": [
+        20,
+        11
+      ],
+      "input_sha256": "efd7d799cc51fc1a3a5dacdb84ad5267feb972e6498a7824d1d51e424b2082d2",
+      "maximum_absolute_logit_difference": 1.6450881958007812e-05,
+      "maximum_absolute_probability_difference": 2.1141022443771362e-07,
+      "argmax_differences": 0,
+      "passed": true
+    },
+    {
+      "batch": 2,
+      "padded_tokens": 102,
+      "attention_mask_counts": [
+        6,
+        102
+      ],
+      "input_sha256": "3af336fced079e4eeb53e968b3bd26cbb3ed2c996b37d710c6108708ba5f8770",
+      "maximum_absolute_logit_difference": 8.463859558105469e-05,
+      "maximum_absolute_probability_difference": 2.6635825634002686e-06,
+      "argmax_differences": 0,
+      "passed": true
+    }
+  ]
+}

adapter-variant-manifest.json

@@ -0,0 +1,19 @@
+{
+  "task": "Modality",
+  "files": {
+    "adapter-qualification.json": "1c3f3675e975a08f26685ff955263f2c17e6de624115b3a627ea90ca1b5f9c98",
+    "lora/README.md": "46387be05f1fa1bd1fd92c09f615bf8e3bd8d9b499089f785148c1c186f0bd8e",
+    "lora/adapter_config.json": "426d2ca8619487565d658930462e30a0d156641f0800c255f84b3610f44139d5",
+    "lora/adapter_model.safetensors": "52fec9bffae938380605bd1da242fcfded6c75964b89ce112de5206f14f4eb00",
+    "lora/config.json": "4cc261fd20f7acf8eb78c7c81491a57eaff5cbf5dbe73ed738fc87af97e729f2",
+    "lora/label_mapping.json": "63c62f883903b470c24ab5c760f593d0dbe4a01823376ee312dbf77fa9b7686a",
+    "lora/load_adapter.py": "759023bdc0bd563b422068c6b3dd28361a5c2ecdd3f4b8073ef931e040940298",
+    "lora/modality_mapping.json": "40890ec869e786e4ca6510367bd1a6cdc2feedaf8564b345b24b38bbce7e0edd",
+    "lora/special_tokens_map.json": "6e3204c5a4004719c185007a745d1940471a6fb02521cfa0a00306dbb6bc5903",
+    "lora/tokenizer.json": "22fd4a60565f25fee8ebd2866aab07456dd46cbbeb45eaf506529ed8451ee58f",
+    "lora/tokenizer_config.json": "74a259bb1a3811a7e3028adcd07a65765d866d5e66e0e883f0994ccfa67e8455",
+    "lora/verify_adapter.py": "ad9a6a32d8bc488dd56976936c4cccc217939a9c29256a069d4e9430c901b85f",
+    "standalone-verification.json": "1f1004e815211afa6ca714f169e311a54a6b27cc539de304f0d1064ebb6b1421"
+  },
+  "independent_standalone_verification": true
+}

config.json

@@ -0,0 +1,57 @@
+{
+  "architectures": [
+    "ModernBertForSequenceClassification"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 1,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 1,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "id2label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  },
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "label2id": {
+    "AR": 0,
+    "BOTH": 2,
+    "DIFFUSION": 1
+  },
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 160000,
+  "mask_token_id": 4,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 0,
+  "position_embedding_type": "sans_pos",
+  "problem_type": "single_label_classification",
+  "repad_logits_with_grad": false,
+  "sep_token_id": 1,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "transformers_version": "4.57.6",
+  "vocab_size": 256000
+}

evaluation/candidate-config.json

@@ -0,0 +1,57 @@
+{
+  "architectures": [
+    "ModernBertForSequenceClassification"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 1,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 1,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "id2label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  },
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "label2id": {
+    "AR": 0,
+    "BOTH": 2,
+    "DIFFUSION": 1
+  },
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 160000,
+  "mask_token_id": 4,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 0,
+  "position_embedding_type": "sans_pos",
+  "problem_type": "single_label_classification",
+  "repad_logits_with_grad": false,
+  "sep_token_id": 1,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "transformers_version": "4.57.6",
+  "vocab_size": 256000
+}

evaluation/candidate-label_mapping.json

@@ -0,0 +1,22 @@
+{
+  "label2id": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "id2label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  },
+  "label_to_idx": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "idx_to_label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  }
+}

evaluation/candidate-lock.json

@@ -0,0 +1,34 @@
+{
+  "base_model": "llm-semantic-router/Vela-1.0-Encoder-307M",
+  "base_revision": "5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114",
+  "adapter_sha256": "52fec9bffae938380605bd1da242fcfded6c75964b89ce112de5206f14f4eb00",
+  "run_manifest_sha256": "a6e0136fa8a08b0e03dd581518c2cda1d1a3468c5e32d84be0690888b350baf8",
+  "parameter_count": 307532547,
+  "encoder_parameter_count": 306939648,
+  "label2id": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "runtime_task": "modality",
+  "runtime_mapping_files": [
+    "label_mapping.json",
+    "modality_mapping.json"
+  ],
+  "task": "single_label_classification",
+  "classifier_pooling": "mean",
+  "max_position_embeddings": 32768,
+  "position_capacity_is_quality_evidence": false,
+  "weight_dtype": "float32",
+  "test_used_for_selection": false,
+  "merge_probe_max_absolute_difference": 3.230571746826172e-05,
+  "files": {
+    "config.json": "4cc261fd20f7acf8eb78c7c81491a57eaff5cbf5dbe73ed738fc87af97e729f2",
+    "label_mapping.json": "63c62f883903b470c24ab5c760f593d0dbe4a01823376ee312dbf77fa9b7686a",
+    "modality_mapping.json": "40890ec869e786e4ca6510367bd1a6cdc2feedaf8564b345b24b38bbce7e0edd",
+    "model.safetensors": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+    "special_tokens_map.json": "6e3204c5a4004719c185007a745d1940471a6fb02521cfa0a00306dbb6bc5903",
+    "tokenizer.json": "22fd4a60565f25fee8ebd2866aab07456dd46cbbeb45eaf506529ed8451ee58f",
+    "tokenizer_config.json": "74a259bb1a3811a7e3028adcd07a65765d866d5e66e0e883f0994ccfa67e8455"
+  }
+}

evaluation/candidate-modality_mapping.json

@@ -0,0 +1,12 @@
+{
+  "label_to_idx": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "idx_to_label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  }
+}

evaluation/comparison.json

@@ -0,0 +1,3276 @@
+{
+  "selection_used_test": false,
+  "metric": "Argmax classification on FP32 frozen predictions; academic and authored stress are not natural-request estimates",
+  "inputs": {
+    "manifest": "2bf2b4787248172f9df19fdc2203b20e5c83f1967f355922e5e9ba5025064fe0",
+    "baseline": "2fe80dcebad0334c54c8d1bd064346d2e9e702a00fb01fba4b7186cd518d8c4e",
+    "candidate": "2f28bc2e3f7c9a6cd3898efe6210202fc75c4d265158210c1f148635cf8a131c",
+    "contract": "87c8b1584d114e307901a3e1a203d93433cffbea0e62e3810825009a5f280a2a"
+  },
+  "sources": {
+    "academic-text-answer": {
+      "rows": 240,
+      "source_groups": 218,
+      "baseline": {
+        "rows": 240,
+        "accuracy": 0.9666666666666667,
+        "macro_f1": 0.3276836158192091,
+        "macro_f1_present_labels": 0.9830508474576272,
+        "per_label": {
+          "AR": {
+            "support": 240,
+            "tp": 232,
+            "fp": 0,
+            "fn": 8,
+            "precision": 1.0,
+            "recall": 0.9666666666666667,
+            "f1": 0.9830508474576272
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 8,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            232,
+            0,
+            8
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "candidate": {
+        "rows": 240,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 240,
+            "tp": 240,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            240,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "paired_correct_changes": {
+        "improved": 8,
+        "regressed": 0
+      },
+      "by_language": {
+        "de": {
+          "rows": 40,
+          "source_groups": 40,
+          "baseline": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 0
+          }
+        },
+        "en": {
+          "rows": 40,
+          "source_groups": 40,
+          "baseline": {
+            "rows": 40,
+            "accuracy": 0.975,
+            "macro_f1": 0.32911392405063294,
+            "macro_f1_present_labels": 0.9873417721518988,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 39,
+                "fp": 0,
+                "fn": 1,
+                "precision": 1.0,
+                "recall": 0.975,
+                "f1": 0.9873417721518988
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 1,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                39,
+                0,
+                1
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 1,
+            "regressed": 0
+          }
+        },
+        "es": {
+          "rows": 40,
+          "source_groups": 40,
+          "baseline": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 0
+          }
+        },
+        "fr": {
+          "rows": 40,
+          "source_groups": 40,
+          "baseline": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 0
+          }
+        },
+        "ja": {
+          "rows": 40,
+          "source_groups": 40,
+          "baseline": {
+            "rows": 40,
+            "accuracy": 0.9,
+            "macro_f1": 0.3157894736842105,
+            "macro_f1_present_labels": 0.9473684210526315,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 36,
+                "fp": 0,
+                "fn": 4,
+                "precision": 1.0,
+                "recall": 0.9,
+                "f1": 0.9473684210526315
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 4,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                36,
+                0,
+                4
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 4,
+            "regressed": 0
+          }
+        },
+        "zh": {
+          "rows": 40,
+          "source_groups": 40,
+          "baseline": {
+            "rows": 40,
+            "accuracy": 0.925,
+            "macro_f1": 0.3203463203463203,
+            "macro_f1_present_labels": 0.961038961038961,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 37,
+                "fp": 0,
+                "fn": 3,
+                "precision": 1.0,
+                "recall": 0.925,
+                "f1": 0.961038961038961
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 3,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                37,
+                0,
+                3
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 40,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 40,
+                "tp": 40,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                40,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 3,
+            "regressed": 0
+          }
+        }
+      },
+      "by_length": {
+        "short": {
+          "rows": 240,
+          "source_groups": 218,
+          "baseline": {
+            "rows": 240,
+            "accuracy": 0.9666666666666667,
+            "macro_f1": 0.3276836158192091,
+            "macro_f1_present_labels": 0.9830508474576272,
+            "per_label": {
+              "AR": {
+                "support": 240,
+                "tp": 232,
+                "fp": 0,
+                "fn": 8,
+                "precision": 1.0,
+                "recall": 0.9666666666666667,
+                "f1": 0.9830508474576272
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 8,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                232,
+                0,
+                8
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 240,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 240,
+                "tp": 240,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                240,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 8,
+            "regressed": 0
+          }
+        }
+      }
+    },
+    "authored-context-on-frozen-source": {
+      "rows": 72,
+      "source_groups": 6,
+      "baseline": {
+        "rows": 72,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 24,
+            "tp": 12,
+            "fp": 24,
+            "fn": 12,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 24,
+            "tp": 12,
+            "fp": 24,
+            "fn": 12,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 24,
+            "tp": 0,
+            "fp": 0,
+            "fn": 24,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            12,
+            12,
+            0
+          ],
+          [
+            12,
+            12,
+            0
+          ],
+          [
+            12,
+            12,
+            0
+          ]
+        ]
+      },
+      "candidate": {
+        "rows": 72,
+        "accuracy": 0.75,
+        "macro_f1": 0.7394880644233535,
+        "macro_f1_present_labels": 0.7394880644233535,
+        "per_label": {
+          "AR": {
+            "support": 24,
+            "tp": 18,
+            "fp": 3,
+            "fn": 6,
+            "precision": 0.8571428571428571,
+            "recall": 0.75,
+            "f1": 0.8
+          },
+          "DIFFUSION": {
+            "support": 24,
+            "tp": 12,
+            "fp": 2,
+            "fn": 12,
+            "precision": 0.8571428571428571,
+            "recall": 0.5,
+            "f1": 0.631578947368421
+          },
+          "BOTH": {
+            "support": 24,
+            "tp": 24,
+            "fp": 13,
+            "fn": 0,
+            "precision": 0.6486486486486487,
+            "recall": 1.0,
+            "f1": 0.7868852459016393
+          }
+        },
+        "confusion": [
+          [
+            18,
+            2,
+            4
+          ],
+          [
+            3,
+            12,
+            9
+          ],
+          [
+            0,
+            0,
+            24
+          ]
+        ]
+      },
+      "paired_correct_changes": {
+        "improved": 32,
+        "regressed": 2
+      },
+      "by_language": {
+        "en": {
+          "rows": 36,
+          "source_groups": 3,
+          "baseline": {
+            "rows": 36,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.16666666666666666,
+            "macro_f1_present_labels": 0.16666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 12,
+                "tp": 0,
+                "fp": 0,
+                "fn": 12,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "DIFFUSION": {
+                "support": 12,
+                "tp": 12,
+                "fp": 24,
+                "fn": 0,
+                "precision": 0.3333333333333333,
+                "recall": 1.0,
+                "f1": 0.5
+              },
+              "BOTH": {
+                "support": 12,
+                "tp": 0,
+                "fp": 0,
+                "fn": 12,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                0,
+                12,
+                0
+              ],
+              [
+                0,
+                12,
+                0
+              ],
+              [
+                0,
+                12,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 36,
+            "accuracy": 0.8888888888888888,
+            "macro_f1": 0.8857142857142857,
+            "macro_f1_present_labels": 0.8857142857142857,
+            "per_label": {
+              "AR": {
+                "support": 12,
+                "tp": 8,
+                "fp": 0,
+                "fn": 4,
+                "precision": 1.0,
+                "recall": 0.6666666666666666,
+                "f1": 0.8
+              },
+              "DIFFUSION": {
+                "support": 12,
+                "tp": 12,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "BOTH": {
+                "support": 12,
+                "tp": 12,
+                "fp": 4,
+                "fn": 0,
+                "precision": 0.75,
+                "recall": 1.0,
+                "f1": 0.8571428571428571
+              }
+            },
+            "confusion": [
+              [
+                8,
+                0,
+                4
+              ],
+              [
+                0,
+                12,
+                0
+              ],
+              [
+                0,
+                0,
+                12
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 20,
+            "regressed": 0
+          }
+        },
+        "zh": {
+          "rows": 36,
+          "source_groups": 3,
+          "baseline": {
+            "rows": 36,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.16666666666666666,
+            "macro_f1_present_labels": 0.16666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 12,
+                "tp": 12,
+                "fp": 24,
+                "fn": 0,
+                "precision": 0.3333333333333333,
+                "recall": 1.0,
+                "f1": 0.5
+              },
+              "DIFFUSION": {
+                "support": 12,
+                "tp": 0,
+                "fp": 0,
+                "fn": 12,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 12,
+                "tp": 0,
+                "fp": 0,
+                "fn": 12,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                12,
+                0,
+                0
+              ],
+              [
+                12,
+                0,
+                0
+              ],
+              [
+                12,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 36,
+            "accuracy": 0.6111111111111112,
+            "macro_f1": 0.5090909090909091,
+            "macro_f1_present_labels": 0.5090909090909091,
+            "per_label": {
+              "AR": {
+                "support": 12,
+                "tp": 10,
+                "fp": 3,
+                "fn": 2,
+                "precision": 0.7692307692307693,
+                "recall": 0.8333333333333334,
+                "f1": 0.8
+              },
+              "DIFFUSION": {
+                "support": 12,
+                "tp": 0,
+                "fp": 2,
+                "fn": 12,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 12,
+                "tp": 12,
+                "fp": 9,
+                "fn": 0,
+                "precision": 0.5714285714285714,
+                "recall": 1.0,
+                "f1": 0.7272727272727273
+              }
+            },
+            "confusion": [
+              [
+                10,
+                2,
+                0
+              ],
+              [
+                3,
+                0,
+                9
+              ],
+              [
+                0,
+                0,
+                12
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 12,
+            "regressed": 2
+          }
+        }
+      },
+      "by_length": {
+        "16384": {
+          "rows": 18,
+          "source_groups": 6,
+          "baseline": {
+            "rows": 18,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.26666666666666666,
+            "macro_f1_present_labels": 0.26666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 0,
+                "fp": 0,
+                "fn": 6,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 18,
+            "accuracy": 0.7222222222222222,
+            "macro_f1": 0.7166666666666667,
+            "macro_f1_present_labels": 0.7166666666666667,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 4,
+                "fp": 0,
+                "fn": 2,
+                "precision": 1.0,
+                "recall": 0.6666666666666666,
+                "f1": 0.8
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 1,
+                "fn": 3,
+                "precision": 0.75,
+                "recall": 0.5,
+                "f1": 0.6
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 6,
+                "fp": 4,
+                "fn": 0,
+                "precision": 0.6,
+                "recall": 1.0,
+                "f1": 0.75
+              }
+            },
+            "confusion": [
+              [
+                4,
+                1,
+                1
+              ],
+              [
+                0,
+                3,
+                3
+              ],
+              [
+                0,
+                0,
+                6
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 8,
+            "regressed": 1
+          }
+        },
+        "32768": {
+          "rows": 18,
+          "source_groups": 6,
+          "baseline": {
+            "rows": 18,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.26666666666666666,
+            "macro_f1_present_labels": 0.26666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 0,
+                "fp": 0,
+                "fn": 6,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 18,
+            "accuracy": 0.6111111111111112,
+            "macro_f1": 0.5888888888888889,
+            "macro_f1_present_labels": 0.5888888888888889,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 2,
+                "fp": 0,
+                "fn": 4,
+                "precision": 1.0,
+                "recall": 0.3333333333333333,
+                "f1": 0.5
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 1,
+                "fn": 3,
+                "precision": 0.75,
+                "recall": 0.5,
+                "f1": 0.6
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 6,
+                "fp": 6,
+                "fn": 0,
+                "precision": 0.5,
+                "recall": 1.0,
+                "f1": 0.6666666666666666
+              }
+            },
+            "confusion": [
+              [
+                2,
+                1,
+                3
+              ],
+              [
+                0,
+                3,
+                3
+              ],
+              [
+                0,
+                0,
+                6
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 6,
+            "regressed": 1
+          }
+        },
+        "4096": {
+          "rows": 18,
+          "source_groups": 6,
+          "baseline": {
+            "rows": 18,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.26666666666666666,
+            "macro_f1_present_labels": 0.26666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 0,
+                "fp": 0,
+                "fn": 6,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 18,
+            "accuracy": 0.8333333333333334,
+            "macro_f1": 0.8222222222222223,
+            "macro_f1_present_labels": 0.8222222222222223,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 6,
+                "fp": 3,
+                "fn": 0,
+                "precision": 0.6666666666666666,
+                "recall": 1.0,
+                "f1": 0.8
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 0,
+                "fn": 3,
+                "precision": 1.0,
+                "recall": 0.5,
+                "f1": 0.6666666666666666
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 6,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              }
+            },
+            "confusion": [
+              [
+                6,
+                0,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                0,
+                0,
+                6
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 9,
+            "regressed": 0
+          }
+        },
+        "8192": {
+          "rows": 18,
+          "source_groups": 6,
+          "baseline": {
+            "rows": 18,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.26666666666666666,
+            "macro_f1_present_labels": 0.26666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 6,
+                "fn": 3,
+                "precision": 0.3333333333333333,
+                "recall": 0.5,
+                "f1": 0.4
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 0,
+                "fp": 0,
+                "fn": 6,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ],
+              [
+                3,
+                3,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 18,
+            "accuracy": 0.8333333333333334,
+            "macro_f1": 0.8222222222222223,
+            "macro_f1_present_labels": 0.8222222222222223,
+            "per_label": {
+              "AR": {
+                "support": 6,
+                "tp": 6,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 6,
+                "tp": 3,
+                "fp": 0,
+                "fn": 3,
+                "precision": 1.0,
+                "recall": 0.5,
+                "f1": 0.6666666666666666
+              },
+              "BOTH": {
+                "support": 6,
+                "tp": 6,
+                "fp": 3,
+                "fn": 0,
+                "precision": 0.6666666666666666,
+                "recall": 1.0,
+                "f1": 0.8
+              }
+            },
+            "confusion": [
+              [
+                6,
+                0,
+                0
+              ],
+              [
+                0,
+                3,
+                3
+              ],
+              [
+                0,
+                0,
+                6
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 9,
+            "regressed": 0
+          }
+        }
+      }
+    },
+    "authored-output-contract-with-gallery-caption": {
+      "rows": 1080,
+      "source_groups": 57,
+      "baseline": {
+        "rows": 1080,
+        "accuracy": 0.34629629629629627,
+        "macro_f1": 0.2216396874882791,
+        "macro_f1_present_labels": 0.2216396874882791,
+        "per_label": {
+          "AR": {
+            "support": 360,
+            "tp": 31,
+            "fp": 46,
+            "fn": 329,
+            "precision": 0.4025974025974026,
+            "recall": 0.08611111111111111,
+            "f1": 0.14187643020594964
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 339,
+            "fp": 653,
+            "fn": 21,
+            "precision": 0.34173387096774194,
+            "recall": 0.9416666666666667,
+            "f1": 0.5014792899408284
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 4,
+            "fp": 7,
+            "fn": 356,
+            "precision": 0.36363636363636365,
+            "recall": 0.011111111111111112,
+            "f1": 0.0215633423180593
+          }
+        },
+        "confusion": [
+          [
+            31,
+            323,
+            6
+          ],
+          [
+            20,
+            339,
+            1
+          ],
+          [
+            26,
+            330,
+            4
+          ]
+        ]
+      },
+      "candidate": {
+        "rows": 1080,
+        "accuracy": 0.9296296296296296,
+        "macro_f1": 0.9290630636363364,
+        "macro_f1_present_labels": 0.9290630636363364,
+        "per_label": {
+          "AR": {
+            "support": 360,
+            "tp": 358,
+            "fp": 70,
+            "fn": 2,
+            "precision": 0.8364485981308412,
+            "recall": 0.9944444444444445,
+            "f1": 0.9086294416243654
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 290,
+            "fp": 6,
+            "fn": 70,
+            "precision": 0.9797297297297297,
+            "recall": 0.8055555555555556,
+            "f1": 0.8841463414634146
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 356,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.9888888888888889,
+            "f1": 0.994413407821229
+          }
+        },
+        "confusion": [
+          [
+            358,
+            2,
+            0
+          ],
+          [
+            70,
+            290,
+            0
+          ],
+          [
+            0,
+            4,
+            356
+          ]
+        ]
+      },
+      "paired_correct_changes": {
+        "improved": 692,
+        "regressed": 62
+      },
+      "by_language": {
+        "de": {
+          "rows": 180,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 180,
+            "accuracy": 0.3888888888888889,
+            "macro_f1": 0.2765939238339752,
+            "macro_f1_present_labels": 0.2765939238339752,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 13,
+                "fp": 22,
+                "fn": 47,
+                "precision": 0.37142857142857144,
+                "recall": 0.21666666666666667,
+                "f1": 0.2736842105263158
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 57,
+                "fp": 88,
+                "fn": 3,
+                "precision": 0.3931034482758621,
+                "recall": 0.95,
+                "f1": 0.5560975609756098
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 0,
+                "fp": 0,
+                "fn": 60,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                13,
+                47,
+                0
+              ],
+              [
+                3,
+                57,
+                0
+              ],
+              [
+                19,
+                41,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 180,
+            "accuracy": 0.9333333333333333,
+            "macro_f1": 0.9326599326599326,
+            "macro_f1_present_labels": 0.9326599326599326,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 60,
+                "fp": 12,
+                "fn": 0,
+                "precision": 0.8333333333333334,
+                "recall": 1.0,
+                "f1": 0.9090909090909091
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 48,
+                "fp": 0,
+                "fn": 12,
+                "precision": 1.0,
+                "recall": 0.8,
+                "f1": 0.8888888888888888
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 60,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              }
+            },
+            "confusion": [
+              [
+                60,
+                0,
+                0
+              ],
+              [
+                12,
+                48,
+                0
+              ],
+              [
+                0,
+                0,
+                60
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 109,
+            "regressed": 11
+          }
+        },
+        "en": {
+          "rows": 180,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 180,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.16666666666666666,
+            "macro_f1_present_labels": 0.16666666666666666,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 0,
+                "fp": 0,
+                "fn": 60,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 60,
+                "fp": 120,
+                "fn": 0,
+                "precision": 0.3333333333333333,
+                "recall": 1.0,
+                "f1": 0.5
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 0,
+                "fp": 0,
+                "fn": 60,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                0,
+                60,
+                0
+              ],
+              [
+                0,
+                60,
+                0
+              ],
+              [
+                0,
+                60,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 180,
+            "accuracy": 0.9777777777777777,
+            "macro_f1": 0.9777716032231175,
+            "macro_f1_present_labels": 0.9777716032231175,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 59,
+                "fp": 3,
+                "fn": 1,
+                "precision": 0.9516129032258065,
+                "recall": 0.9833333333333333,
+                "f1": 0.9672131147540983
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 57,
+                "fp": 1,
+                "fn": 3,
+                "precision": 0.9827586206896551,
+                "recall": 0.95,
+                "f1": 0.9661016949152542
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 60,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              }
+            },
+            "confusion": [
+              [
+                59,
+                1,
+                0
+              ],
+              [
+                3,
+                57,
+                0
+              ],
+              [
+                0,
+                0,
+                60
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 119,
+            "regressed": 3
+          }
+        },
+        "es": {
+          "rows": 180,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 180,
+            "accuracy": 0.35555555555555557,
+            "macro_f1": 0.22549019607843138,
+            "macro_f1_present_labels": 0.22549019607843138,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 6,
+                "fp": 2,
+                "fn": 54,
+                "precision": 0.75,
+                "recall": 0.1,
+                "f1": 0.17647058823529413
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 58,
+                "fp": 114,
+                "fn": 2,
+                "precision": 0.3372093023255814,
+                "recall": 0.9666666666666667,
+                "f1": 0.5
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 0,
+                "fp": 0,
+                "fn": 60,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                6,
+                54,
+                0
+              ],
+              [
+                2,
+                58,
+                0
+              ],
+              [
+                0,
+                60,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 180,
+            "accuracy": 0.9722222222222222,
+            "macro_f1": 0.972214405895479,
+            "macro_f1_present_labels": 0.972214405895479,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 60,
+                "fp": 2,
+                "fn": 0,
+                "precision": 0.967741935483871,
+                "recall": 1.0,
+                "f1": 0.9836065573770492
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 58,
+                "fp": 3,
+                "fn": 2,
+                "precision": 0.9508196721311475,
+                "recall": 0.9666666666666667,
+                "f1": 0.9586776859504132
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 57,
+                "fp": 0,
+                "fn": 3,
+                "precision": 1.0,
+                "recall": 0.95,
+                "f1": 0.9743589743589743
+              }
+            },
+            "confusion": [
+              [
+                60,
+                0,
+                0
+              ],
+              [
+                2,
+                58,
+                0
+              ],
+              [
+                0,
+                3,
+                57
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 113,
+            "regressed": 2
+          }
+        },
+        "fr": {
+          "rows": 180,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 180,
+            "accuracy": 0.3333333333333333,
+            "macro_f1": 0.17819217819217817,
+            "macro_f1_present_labels": 0.17819217819217817,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 1,
+                "fp": 5,
+                "fn": 59,
+                "precision": 0.16666666666666666,
+                "recall": 0.016666666666666666,
+                "f1": 0.030303030303030304
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 59,
+                "fp": 115,
+                "fn": 1,
+                "precision": 0.3390804597701149,
+                "recall": 0.9833333333333333,
+                "f1": 0.5042735042735043
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 0,
+                "fp": 0,
+                "fn": 60,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                1,
+                59,
+                0
+              ],
+              [
+                1,
+                59,
+                0
+              ],
+              [
+                4,
+                56,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 180,
+            "accuracy": 0.9111111111111111,
+            "macro_f1": 0.9098473771059786,
+            "macro_f1_present_labels": 0.9098473771059786,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 60,
+                "fp": 15,
+                "fn": 0,
+                "precision": 0.8,
+                "recall": 1.0,
+                "f1": 0.8888888888888888
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 45,
+                "fp": 1,
+                "fn": 15,
+                "precision": 0.9782608695652174,
+                "recall": 0.75,
+                "f1": 0.8490566037735849
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 59,
+                "fp": 0,
+                "fn": 1,
+                "precision": 1.0,
+                "recall": 0.9833333333333333,
+                "f1": 0.9915966386554622
+              }
+            },
+            "confusion": [
+              [
+                60,
+                0,
+                0
+              ],
+              [
+                15,
+                45,
+                0
+              ],
+              [
+                0,
+                1,
+                59
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 119,
+            "regressed": 15
+          }
+        },
+        "ja": {
+          "rows": 180,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 180,
+            "accuracy": 0.34444444444444444,
+            "macro_f1": 0.26517856622425845,
+            "macro_f1_present_labels": 0.26517856622425845,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 10,
+                "fp": 13,
+                "fn": 50,
+                "precision": 0.43478260869565216,
+                "recall": 0.16666666666666666,
+                "f1": 0.24096385542168675
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 49,
+                "fp": 103,
+                "fn": 11,
+                "precision": 0.3223684210526316,
+                "recall": 0.8166666666666667,
+                "f1": 0.46226415094339623
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 3,
+                "fp": 2,
+                "fn": 57,
+                "precision": 0.6,
+                "recall": 0.05,
+                "f1": 0.09230769230769231
+              }
+            },
+            "confusion": [
+              [
+                10,
+                48,
+                2
+              ],
+              [
+                11,
+                49,
+                0
+              ],
+              [
+                2,
+                55,
+                3
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 180,
+            "accuracy": 0.8333333333333334,
+            "macro_f1": 0.8222222222222223,
+            "macro_f1_present_labels": 0.8222222222222223,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 60,
+                "fp": 30,
+                "fn": 0,
+                "precision": 0.6666666666666666,
+                "recall": 1.0,
+                "f1": 0.8
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 30,
+                "fp": 0,
+                "fn": 30,
+                "precision": 1.0,
+                "recall": 0.5,
+                "f1": 0.6666666666666666
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 60,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              }
+            },
+            "confusion": [
+              [
+                60,
+                0,
+                0
+              ],
+              [
+                30,
+                30,
+                0
+              ],
+              [
+                0,
+                0,
+                60
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 111,
+            "regressed": 23
+          }
+        },
+        "zh": {
+          "rows": 180,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 180,
+            "accuracy": 0.32222222222222224,
+            "macro_f1": 0.18338507683485847,
+            "macro_f1_present_labels": 0.18338507683485847,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 1,
+                "fp": 4,
+                "fn": 59,
+                "precision": 0.2,
+                "recall": 0.016666666666666666,
+                "f1": 0.03076923076923077
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 56,
+                "fp": 113,
+                "fn": 4,
+                "precision": 0.33136094674556216,
+                "recall": 0.9333333333333333,
+                "f1": 0.4890829694323144
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 1,
+                "fp": 5,
+                "fn": 59,
+                "precision": 0.16666666666666666,
+                "recall": 0.016666666666666666,
+                "f1": 0.030303030303030304
+              }
+            },
+            "confusion": [
+              [
+                1,
+                55,
+                4
+              ],
+              [
+                3,
+                56,
+                1
+              ],
+              [
+                1,
+                58,
+                1
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 180,
+            "accuracy": 0.95,
+            "macro_f1": 0.9498292801895337,
+            "macro_f1_present_labels": 0.9498292801895337,
+            "per_label": {
+              "AR": {
+                "support": 60,
+                "tp": 59,
+                "fp": 8,
+                "fn": 1,
+                "precision": 0.8805970149253731,
+                "recall": 0.9833333333333333,
+                "f1": 0.9291338582677166
+              },
+              "DIFFUSION": {
+                "support": 60,
+                "tp": 52,
+                "fp": 1,
+                "fn": 8,
+                "precision": 0.9811320754716981,
+                "recall": 0.8666666666666667,
+                "f1": 0.9203539823008849
+              },
+              "BOTH": {
+                "support": 60,
+                "tp": 60,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              }
+            },
+            "confusion": [
+              [
+                59,
+                1,
+                0
+              ],
+              [
+                8,
+                52,
+                0
+              ],
+              [
+                0,
+                0,
+                60
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 121,
+            "regressed": 8
+          }
+        }
+      },
+      "by_length": {
+        "short": {
+          "rows": 1080,
+          "source_groups": 57,
+          "baseline": {
+            "rows": 1080,
+            "accuracy": 0.34629629629629627,
+            "macro_f1": 0.2216396874882791,
+            "macro_f1_present_labels": 0.2216396874882791,
+            "per_label": {
+              "AR": {
+                "support": 360,
+                "tp": 31,
+                "fp": 46,
+                "fn": 329,
+                "precision": 0.4025974025974026,
+                "recall": 0.08611111111111111,
+                "f1": 0.14187643020594964
+              },
+              "DIFFUSION": {
+                "support": 360,
+                "tp": 339,
+                "fp": 653,
+                "fn": 21,
+                "precision": 0.34173387096774194,
+                "recall": 0.9416666666666667,
+                "f1": 0.5014792899408284
+              },
+              "BOTH": {
+                "support": 360,
+                "tp": 4,
+                "fp": 7,
+                "fn": 356,
+                "precision": 0.36363636363636365,
+                "recall": 0.011111111111111112,
+                "f1": 0.0215633423180593
+              }
+            },
+            "confusion": [
+              [
+                31,
+                323,
+                6
+              ],
+              [
+                20,
+                339,
+                1
+              ],
+              [
+                26,
+                330,
+                4
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 1080,
+            "accuracy": 0.9296296296296296,
+            "macro_f1": 0.9290630636363364,
+            "macro_f1_present_labels": 0.9290630636363364,
+            "per_label": {
+              "AR": {
+                "support": 360,
+                "tp": 358,
+                "fp": 70,
+                "fn": 2,
+                "precision": 0.8364485981308412,
+                "recall": 0.9944444444444445,
+                "f1": 0.9086294416243654
+              },
+              "DIFFUSION": {
+                "support": 360,
+                "tp": 290,
+                "fp": 6,
+                "fn": 70,
+                "precision": 0.9797297297297297,
+                "recall": 0.8055555555555556,
+                "f1": 0.8841463414634146
+              },
+              "BOTH": {
+                "support": 360,
+                "tp": 356,
+                "fp": 0,
+                "fn": 4,
+                "precision": 1.0,
+                "recall": 0.9888888888888889,
+                "f1": 0.994413407821229
+              }
+            },
+            "confusion": [
+              [
+                358,
+                2,
+                0
+              ],
+              [
+                70,
+                290,
+                0
+              ],
+              [
+                0,
+                4,
+                356
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 692,
+            "regressed": 62
+          }
+        }
+      }
+    },
+    "aya-original-human": {
+      "rows": 19,
+      "source_groups": 6,
+      "baseline": {
+        "rows": 19,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 19,
+            "tp": 19,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            19,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "candidate": {
+        "rows": 19,
+        "accuracy": 0.8421052631578947,
+        "macro_f1": 0.30476190476190473,
+        "macro_f1_present_labels": 0.9142857142857143,
+        "per_label": {
+          "AR": {
+            "support": 19,
+            "tp": 16,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.8421052631578947,
+            "f1": 0.9142857142857143
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 1,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 2,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            16,
+            1,
+            2
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "paired_correct_changes": {
+        "improved": 0,
+        "regressed": 3
+      },
+      "by_language": {
+        "de": {
+          "rows": 1,
+          "source_groups": 1,
+          "baseline": {
+            "rows": 1,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 1,
+                "tp": 1,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                1,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 1,
+            "accuracy": 0.0,
+            "macro_f1": 0.0,
+            "macro_f1_present_labels": 0.0,
+            "per_label": {
+              "AR": {
+                "support": 1,
+                "tp": 0,
+                "fp": 0,
+                "fn": 1,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 1,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                0,
+                0,
+                1
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 1
+          }
+        },
+        "en": {
+          "rows": 1,
+          "source_groups": 1,
+          "baseline": {
+            "rows": 1,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 1,
+                "tp": 1,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                1,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 1,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 1,
+                "tp": 1,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                1,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 0
+          }
+        },
+        "es": {
+          "rows": 2,
+          "source_groups": 2,
+          "baseline": {
+            "rows": 2,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 2,
+                "tp": 2,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                2,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 2,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 2,
+                "tp": 2,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                2,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 0
+          }
+        },
+        "fr": {
+          "rows": 15,
+          "source_groups": 2,
+          "baseline": {
+            "rows": 15,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 15,
+                "tp": 15,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                15,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 15,
+            "accuracy": 0.8666666666666667,
+            "macro_f1": 0.30952380952380953,
+            "macro_f1_present_labels": 0.9285714285714286,
+            "per_label": {
+              "AR": {
+                "support": 15,
+                "tp": 13,
+                "fp": 0,
+                "fn": 2,
+                "precision": 1.0,
+                "recall": 0.8666666666666667,
+                "f1": 0.9285714285714286
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 1,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 1,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                13,
+                1,
+                1
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 2
+          }
+        }
+      },
+      "by_length": {
+        "short": {
+          "rows": 19,
+          "source_groups": 6,
+          "baseline": {
+            "rows": 19,
+            "accuracy": 1.0,
+            "macro_f1": 0.3333333333333333,
+            "macro_f1_present_labels": 1.0,
+            "per_label": {
+              "AR": {
+                "support": 19,
+                "tp": 19,
+                "fp": 0,
+                "fn": 0,
+                "precision": 1.0,
+                "recall": 1.0,
+                "f1": 1.0
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 0,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                19,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "candidate": {
+            "rows": 19,
+            "accuracy": 0.8421052631578947,
+            "macro_f1": 0.30476190476190473,
+            "macro_f1_present_labels": 0.9142857142857143,
+            "per_label": {
+              "AR": {
+                "support": 19,
+                "tp": 16,
+                "fp": 0,
+                "fn": 3,
+                "precision": 1.0,
+                "recall": 0.8421052631578947,
+                "f1": 0.9142857142857143
+              },
+              "DIFFUSION": {
+                "support": 0,
+                "tp": 0,
+                "fp": 1,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              },
+              "BOTH": {
+                "support": 0,
+                "tp": 0,
+                "fp": 2,
+                "fn": 0,
+                "precision": 0.0,
+                "recall": 0.0,
+                "f1": 0.0
+              }
+            },
+            "confusion": [
+              [
+                16,
+                1,
+                2
+              ],
+              [
+                0,
+                0,
+                0
+              ],
+              [
+                0,
+                0,
+                0
+              ]
+            ]
+          },
+          "paired_correct_changes": {
+            "improved": 0,
+            "regressed": 3
+          }
+        }
+      }
+    }
+  }
+}

evaluation/engines/blocked-storage-provenance.json

@@ -0,0 +1,3850 @@
+{
+  "schema_version": 1,
+  "task": "Modality",
+  "model": "Vela-1.0-Encoder-307M-Modality",
+  "passed": true,
+  "method": "CPU streaming SHA256 of original and final external initializer byte ranges, with exact dtype and shape equality; inline initializers compared by logical numeric bytes. Original files checked against frozen export receipt. Qualified final graph checked against historical GPU receipt. No inference rerun.",
+  "scope": "This receipt binds current external data to the originally exported frozen model weights and the already qualified graph. It adds storage provenance; it does not change or rerun accuracy or numerical engine qualification.",
+  "native_weights_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "original_export_receipt_sha256": "50f867db7e2155acf677b797468f8f0513a4cdaa7a7986e51296d5bdfb6525aa",
+  "original_files": {
+    "model.onnx": "fa09cedb1da477357a74bda382942f0ce0e341a44cde20f46d48bfc0e94af836",
+    "model.onnx.data": "c4c8565db328f68fda33b64550010813880a32b26cb1988e509acc979303bab8"
+  },
+  "qualified_profile_sha256": "7264fa901a81c6078cd8cabb825d38b18a39746b19eff3ecc692813a256ec1c6",
+  "final_files": {
+    "model.onnx": "74a08c52ce47f522071fd71acee080c0a3a74ad12e3d5fb19ae1dfd244f1059c",
+    "model.onnx.data": "f0372b44dd9a9ae0d93f0ee0211ec27644dd2ad94351c8022e94c953bd14c399"
+  },
+  "original_initializer_count": 161,
+  "final_initializer_count": 157,
+  "matched_external_initializers": 137,
+  "matched_inline_initializers": 20,
+  "matched_tensor_bytes": 1230130444,
+  "all_initializers_equal": true,
+  "initializers": [
+    {
+      "name": "model.model.embeddings.tok_embeddings.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          256000,
+          768
+        ],
+        "bytes": 786432000,
+        "sha256": "8d7917c0766aca15106eb571153b4c0da024d499d4e8d57413957db59edfefd4"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          256000,
+          768
+        ],
+        "bytes": 786432000,
+        "sha256": "8d7917c0766aca15106eb571153b4c0da024d499d4e8d57413957db59edfefd4"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.embeddings.norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "c48411993b1c6bfad0f7c0994958526e9d41a4c439369fb9ccabf1b069992801"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "c48411993b1c6bfad0f7c0994958526e9d41a4c439369fb9ccabf1b069992801"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.0.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "ec89f8bd25efe9bee80b2b311322394efe73b8b343ddc8e862b94385a1158294"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "ec89f8bd25efe9bee80b2b311322394efe73b8b343ddc8e862b94385a1158294"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.1.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "4f165660f05290a797d077ed63f44be146c7edc7a2e20a16ccdedbcfc5fa1a8b"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "4f165660f05290a797d077ed63f44be146c7edc7a2e20a16ccdedbcfc5fa1a8b"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.1.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e2b7b7d3d1ce0de28043c771d0df95dbe69985d6063fc517a7df2e03a8fb5e3a"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e2b7b7d3d1ce0de28043c771d0df95dbe69985d6063fc517a7df2e03a8fb5e3a"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.2.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "b923f7ec46d4e08ae4e1da80e8378ed3d66cf82e4d5773f1567bb113d9d86913"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "b923f7ec46d4e08ae4e1da80e8378ed3d66cf82e4d5773f1567bb113d9d86913"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.2.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "88d7dddd88cad01cbe1c51c3eadb58e9798d854cf09d5892ec51d3c486955e54"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "88d7dddd88cad01cbe1c51c3eadb58e9798d854cf09d5892ec51d3c486955e54"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.3.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "23675a4ac01236e2c4b0e9b2c8821be1140500619962b3c2e48b809ec142ad2e"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "23675a4ac01236e2c4b0e9b2c8821be1140500619962b3c2e48b809ec142ad2e"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.3.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "f2179bdf7f369fcd8c79493e2da8f5cb069cff6f5c8d7da0613b2a33c512791f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "f2179bdf7f369fcd8c79493e2da8f5cb069cff6f5c8d7da0613b2a33c512791f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.4.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "eb8b281258b9a7788a5c5a903de6a088b032f5994a6f236dab9d0d945f06bdc5"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "eb8b281258b9a7788a5c5a903de6a088b032f5994a6f236dab9d0d945f06bdc5"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.4.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "a9e7de99faf4a3e66871e0510128390e03ceebfc3da5170bbe6b5e3f17ab7754"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "a9e7de99faf4a3e66871e0510128390e03ceebfc3da5170bbe6b5e3f17ab7754"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.5.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "6165f78c54ce4cf69bc769f70bba15c697ad3d6589ddc7ff7fb6d62a84e44271"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "6165f78c54ce4cf69bc769f70bba15c697ad3d6589ddc7ff7fb6d62a84e44271"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.5.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5b2447b0cbf5acd5ba44f63763232bb2e71e3eebbaffc952fb831024d8608fda"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5b2447b0cbf5acd5ba44f63763232bb2e71e3eebbaffc952fb831024d8608fda"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.6.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "f853dd2ca0d74178d689dd9981d30649af944e84e2a21dbacd06fb314cec6e86"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "f853dd2ca0d74178d689dd9981d30649af944e84e2a21dbacd06fb314cec6e86"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.6.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "049b02dd2dab0ee620f3bccf6a0f29c3b0dbc75cf5d1cc9822c7c0db867cfa3e"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "049b02dd2dab0ee620f3bccf6a0f29c3b0dbc75cf5d1cc9822c7c0db867cfa3e"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.7.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "299fe6ec9c2146dc027b62c7fd10cb656de62f5ea751754b1377585ec9ad79bb"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "299fe6ec9c2146dc027b62c7fd10cb656de62f5ea751754b1377585ec9ad79bb"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.7.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "85913a398c2060a9542b950c743930d96d1135a3aa2a07c5ac5320f127bba7b1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "85913a398c2060a9542b950c743930d96d1135a3aa2a07c5ac5320f127bba7b1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.8.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e5ade1a4f393518288d512258a4dadd7b67410305937d685df26721bb6f83671"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e5ade1a4f393518288d512258a4dadd7b67410305937d685df26721bb6f83671"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.8.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "72c4d8fdfbc03a07e1b17886ac6c7b58848dddb49e4f27966fdcb6b2b92114e1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "72c4d8fdfbc03a07e1b17886ac6c7b58848dddb49e4f27966fdcb6b2b92114e1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.9.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "01083bbdc4d6d22d3ac502f52a9004cb2c2792c10d3ebb4cad5a78fa2c7ccd86"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "01083bbdc4d6d22d3ac502f52a9004cb2c2792c10d3ebb4cad5a78fa2c7ccd86"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.9.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "6128518865106554d16cc194ed9b3db56a036eb07335f7ad2e13f07431a56ab2"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "6128518865106554d16cc194ed9b3db56a036eb07335f7ad2e13f07431a56ab2"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.10.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "0a0118ef2b658d932b1311c6dee628b15d9a158ffe58c805178b13c0f888cd12"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "0a0118ef2b658d932b1311c6dee628b15d9a158ffe58c805178b13c0f888cd12"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.10.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "74f241bb0dae5238ac7cf47ba501c8e0331ae68cfad85c218f361a0d776e8726"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "74f241bb0dae5238ac7cf47ba501c8e0331ae68cfad85c218f361a0d776e8726"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.11.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "70ef5e9993bd7164cfaf4ea451f23c642b74a5ea23d4b0a572bd85a4114ed6e9"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "70ef5e9993bd7164cfaf4ea451f23c642b74a5ea23d4b0a572bd85a4114ed6e9"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.11.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "ecfd4c6b51090624056d99d88c304520c0bdf4def4ea5d2901c07060cec73dd1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "ecfd4c6b51090624056d99d88c304520c0bdf4def4ea5d2901c07060cec73dd1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.12.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e3d47393ffffd762884b97d853be0d8899f0869d426e86556ba07437729ffdb5"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e3d47393ffffd762884b97d853be0d8899f0869d426e86556ba07437729ffdb5"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.12.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "218e71aec18c8b78c5103c9dbf959912a237fad41abd71e22978c6c758216f84"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "218e71aec18c8b78c5103c9dbf959912a237fad41abd71e22978c6c758216f84"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.13.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "76ac261daa6058a20f83af3abf067366a989b9018e1755233290009f0ed579b8"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "76ac261daa6058a20f83af3abf067366a989b9018e1755233290009f0ed579b8"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.13.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "67953b22d5373d3825ad2c10307df9dba38a9c2a68ea89e0cea0255a390233d8"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "67953b22d5373d3825ad2c10307df9dba38a9c2a68ea89e0cea0255a390233d8"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.14.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "bef605095795dfe6d75632489cd9d7bf3c378e639cb29a840d60dd3391018c8f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "bef605095795dfe6d75632489cd9d7bf3c378e639cb29a840d60dd3391018c8f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.14.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "0de4044b70d8c3b7e9d21dd1a27b5e341846cdf22a16e426654cb5cb3ae83207"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "0de4044b70d8c3b7e9d21dd1a27b5e341846cdf22a16e426654cb5cb3ae83207"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.15.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "61121db4134fef26eff12388a48b29bbcf5d661626312f7ee9306fcc3bddbecd"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "61121db4134fef26eff12388a48b29bbcf5d661626312f7ee9306fcc3bddbecd"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.15.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e4557d75724bda343f92103ad32f1489b07748e90372aadf079b0381ae3358eb"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "e4557d75724bda343f92103ad32f1489b07748e90372aadf079b0381ae3358eb"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.16.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "42703d3fc95410028eeefecc489358196e04f6a9638adeab4b1414207082dd11"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "42703d3fc95410028eeefecc489358196e04f6a9638adeab4b1414207082dd11"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.16.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5769e4a44c4b44633b189835fec47e54af362ee6b6a822d413d5ea04b77054eb"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5769e4a44c4b44633b189835fec47e54af362ee6b6a822d413d5ea04b77054eb"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.17.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "bdb5ce1ff0abc1e9eaf6de8abb739ca494ed1903ea7bb45b3c4ef8d16a4cfa27"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "bdb5ce1ff0abc1e9eaf6de8abb739ca494ed1903ea7bb45b3c4ef8d16a4cfa27"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.17.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5821d7cd3931480d2e6c9c635df7eb3ebda3759c3ad215f2ce5a07ad47703566"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5821d7cd3931480d2e6c9c635df7eb3ebda3759c3ad215f2ce5a07ad47703566"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.18.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "da985d47d37e58ed52797efd01955c2946cc90512959ae100510424ab6590f2c"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "da985d47d37e58ed52797efd01955c2946cc90512959ae100510424ab6590f2c"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.18.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5823e60a0c9c797c545d0681928b2efc6daccbdb3f5e2271bb385458a9d6c47a"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5823e60a0c9c797c545d0681928b2efc6daccbdb3f5e2271bb385458a9d6c47a"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.19.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "d34e3a181c091e7d51514bd7546ce2aa0a707ff2a23e8179fabe6f13922e0233"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "d34e3a181c091e7d51514bd7546ce2aa0a707ff2a23e8179fabe6f13922e0233"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.19.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "922ba23b773990b3ecb180b92c47d03b8fa8f2568c08a121c44b6cf5e7cc9325"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "922ba23b773990b3ecb180b92c47d03b8fa8f2568c08a121c44b6cf5e7cc9325"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.20.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "cf6c5ac9804e99c73381a8322a05624d0a22f3040eb4693d0cf9244d9d77f7b0"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "cf6c5ac9804e99c73381a8322a05624d0a22f3040eb4693d0cf9244d9d77f7b0"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.20.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5e77e8f62d46c7d9e09f3563a10bde72181f6b4539c61aedb7ce6212350eb5a7"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "5e77e8f62d46c7d9e09f3563a10bde72181f6b4539c61aedb7ce6212350eb5a7"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.21.attn_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "b15986db9b65b65b9ce0967384ed1282ef590042a16d67f74767db55c0e5458d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "b15986db9b65b65b9ce0967384ed1282ef590042a16d67f74767db55c0e5458d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.layers.21.mlp_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "ca3cbd3aebdc154c44c64a11a368a7b0755dccd31fc6777880db59e6596fafca"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "ca3cbd3aebdc154c44c64a11a368a7b0755dccd31fc6777880db59e6596fafca"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.model.final_norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "02803ade53a2d5463eaf5352e7d798a45ce935444dae0a50d9eb97db9d0a77d1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "02803ade53a2d5463eaf5352e7d798a45ce935444dae0a50d9eb97db9d0a77d1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.head.dense.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "77d0ed479cad468518e7d5599c9d9e5c39e4a46a40af81e5cef39ce51061820f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "77d0ed479cad468518e7d5599c9d9e5c39e4a46a40af81e5cef39ce51061820f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.head.norm.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "c0ff8da291b1f26ee838e7a83ccd3fdf275587e1fccb5c2e45e906266c2bfc7d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768
+        ],
+        "bytes": 3072,
+        "sha256": "c0ff8da291b1f26ee838e7a83ccd3fdf275587e1fccb5c2e45e906266c2bfc7d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.classifier.weight",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          3,
+          768
+        ],
+        "bytes": 9216,
+        "sha256": "3d66e71783c13bfe619ee3e3b8c0dd828d94c1ddf417ee7640b4efb2ff0a213d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          3,
+          768
+        ],
+        "bytes": 9216,
+        "sha256": "3d66e71783c13bfe619ee3e3b8c0dd828d94c1ddf417ee7640b4efb2ff0a213d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "model.classifier.bias",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          3
+        ],
+        "bytes": 12,
+        "sha256": "f879d7ffe60d77c365e80b914d47bbc79ad78ecefc9637cd48782d20fa166e11"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          3
+        ],
+        "bytes": 12,
+        "sha256": "f879d7ffe60d77c365e80b914d47bbc79ad78ecefc9637cd48782d20fa166e11"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_4",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [],
+        "bytes": 8,
+        "sha256": "af5570f5a1810b7af78caf4bc70a660f0df51e42baf91d4de5b2328de0e83dfc"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [],
+        "bytes": 8,
+        "sha256": "af5570f5a1810b7af78caf4bc70a660f0df51e42baf91d4de5b2328de0e83dfc"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_6",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [],
+        "bytes": 8,
+        "sha256": "7c9fa136d4413fa6173637e883b6998d32e1d675f88cddff9dcbcf331820f4b8"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [],
+        "bytes": 8,
+        "sha256": "7c9fa136d4413fa6173637e883b6998d32e1d675f88cddff9dcbcf331820f4b8"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "detach",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "e00e5eb9444182f352323374ef4e08ebcb784725fdd4fd612d7730540b3e0c8c"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "e00e5eb9444182f352323374ef4e08ebcb784725fdd4fd612d7730540b3e0c8c"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_24",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "1765f39d5e2ee8b61268a34d74420f2e90396e02ec52875c8cd92409be6ceb67"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "1765f39d5e2ee8b61268a34d74420f2e90396e02ec52875c8cd92409be6ceb67"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "expand_1",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1,
+          32,
+          1
+        ],
+        "bytes": 128,
+        "sha256": "adb41194debe8e6c185d47754f68cc2dd59960a03cb4ccbc71acd14b860c4604"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1,
+          32,
+          1
+        ],
+        "bytes": 128,
+        "sha256": "adb41194debe8e6c185d47754f68cc2dd59960a03cb4ccbc71acd14b860c4604"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_52",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "af5570f5a1810b7af78caf4bc70a660f0df51e42baf91d4de5b2328de0e83dfc"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "af5570f5a1810b7af78caf4bc70a660f0df51e42baf91d4de5b2328de0e83dfc"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_56",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "9d4ac218fb54041e3a70a8e14db1ea1af9f570f4842b4702ef9323f1ad8f0ec4"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "9d4ac218fb54041e3a70a8e14db1ea1af9f570f4842b4702ef9323f1ad8f0ec4"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_60",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "35be322d094f9d154a8aba4733b8497f180353bd7ae7b0a15f90b586b549f28b"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "35be322d094f9d154a8aba4733b8497f180353bd7ae7b0a15f90b586b549f28b"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_68",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "6a69a6cc7473a16302890cd2a9e93e347281f6ea0e1bb784e589753bed0b3324"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "6a69a6cc7473a16302890cd2a9e93e347281f6ea0e1bb784e589753bed0b3324"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_116",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1
+        ],
+        "bytes": 4,
+        "sha256": "948945eecf4a060f48a79e9fadcd501373212c7f5be4fa5ac1ac216f2469c043"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1
+        ],
+        "bytes": 4,
+        "sha256": "948945eecf4a060f48a79e9fadcd501373212c7f5be4fa5ac1ac216f2469c043"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_128",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "e3e58e351bc840ce171c8c549f2b157733fda22bf529bc08d8d4984af1f77b8d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "e3e58e351bc840ce171c8c549f2b157733fda22bf529bc08d8d4984af1f77b8d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_131",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "8ecb7ab058dc9d0b126a1b4054e56c209308b9b09e6385b2d9ee0666e8d82f71"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "8ecb7ab058dc9d0b126a1b4054e56c209308b9b09e6385b2d9ee0666e8d82f71"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_140",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "dc259a27849660a9b2c360a89ccf44cbfe24286b93ee606d2fbcbd3781f98a59"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "dc259a27849660a9b2c360a89ccf44cbfe24286b93ee606d2fbcbd3781f98a59"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_143",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "81a16b7c72612a32c499c95377c6c1599731ea54467ad4e9fee1b9743df6f46e"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "81a16b7c72612a32c499c95377c6c1599731ea54467ad4e9fee1b9743df6f46e"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_241",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "e5d4f7957244470c49a9567cfa1dd8d4eb9718b9f891426a7056df984871fc4b"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "e5d4f7957244470c49a9567cfa1dd8d4eb9718b9f891426a7056df984871fc4b"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_244",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "71562fb1c132331d9d4262d60755d737f3c28710d2ff44290d49102c1c412d45"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "71562fb1c132331d9d4262d60755d737f3c28710d2ff44290d49102c1c412d45"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_252",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "097334de817d798b383b662c52a6b2bdbaefd8c8f02d202276372d4512f3a594"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "097334de817d798b383b662c52a6b2bdbaefd8c8f02d202276372d4512f3a594"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_255",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "4f9ff48c5539f46a3917db2568d8b731ec1a11ae6cfe84238cb18f8649d6c75b"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "4f9ff48c5539f46a3917db2568d8b731ec1a11ae6cfe84238cb18f8649d6c75b"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_353",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "5614412bcca96639e057df96b421b43150036aa25aa7f62f226743e18a481fd3"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "5614412bcca96639e057df96b421b43150036aa25aa7f62f226743e18a481fd3"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_356",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d58020ffc4ea7d2499c8ab93042b8e3defa50b0de7ea5bdb096dff451be33d9d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d58020ffc4ea7d2499c8ab93042b8e3defa50b0de7ea5bdb096dff451be33d9d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_364",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "a7071101955b62b47bc4a477d5b5796230fce630fe445c58b138bb4ce4d889a1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "a7071101955b62b47bc4a477d5b5796230fce630fe445c58b138bb4ce4d889a1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_367",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "644bf166e116b96699148c6d99ac3089b0732b446cc8cb69fc87b4b0c3543b4f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "644bf166e116b96699148c6d99ac3089b0732b446cc8cb69fc87b4b0c3543b4f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_465",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "d06cb6edbf50417b452e9594d2ef0658340aebdc184b478870f84ad70d1c9e76"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "d06cb6edbf50417b452e9594d2ef0658340aebdc184b478870f84ad70d1c9e76"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_468",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "14f61658f5afaca9e9d2222520a0e81eeb36fd153802b752ebe8772f94eab363"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "14f61658f5afaca9e9d2222520a0e81eeb36fd153802b752ebe8772f94eab363"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_476",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "5521aacc4e2b85379a567b2f9fcc07a6da375b47b242cb0b5d9437a388819e1c"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "5521aacc4e2b85379a567b2f9fcc07a6da375b47b242cb0b5d9437a388819e1c"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_479",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "61a9d8c04373c45fd7157c9487562cacd90fe2fa66209c2ab45dad56f16df8eb"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "61a9d8c04373c45fd7157c9487562cacd90fe2fa66209c2ab45dad56f16df8eb"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_577",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "728e2e8d0f572afd8e4c1d80ba5710635543d488fd7b70c252877210ada22e57"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "728e2e8d0f572afd8e4c1d80ba5710635543d488fd7b70c252877210ada22e57"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_580",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "a6a56a931fb452092bd752c560630871837180777b279cbff1918221edec9ac9"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "a6a56a931fb452092bd752c560630871837180777b279cbff1918221edec9ac9"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_588",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "bb766c89059b5d32c1e8192e148a44b1c579dc76c2cc2ff022e222ad557889b3"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "bb766c89059b5d32c1e8192e148a44b1c579dc76c2cc2ff022e222ad557889b3"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_591",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "83d6d72e2e2fdf9fee484bc29a65e997d39eebdc38f47e054edc3e282978487c"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "83d6d72e2e2fdf9fee484bc29a65e997d39eebdc38f47e054edc3e282978487c"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_689",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "4b5bfdc5053f79d8ab987f80f3b8bb2b9121df7b0f56d1637efed33a5715ef85"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "4b5bfdc5053f79d8ab987f80f3b8bb2b9121df7b0f56d1637efed33a5715ef85"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_692",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "1c634eb39e05e45421e9355bcabeb61308155628f8f9404b47ea66d22d615ef1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "1c634eb39e05e45421e9355bcabeb61308155628f8f9404b47ea66d22d615ef1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_700",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "aa2e0479db6b03016645764695e0cb1f74be90dfd627d9d2713682a789f37ff6"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "aa2e0479db6b03016645764695e0cb1f74be90dfd627d9d2713682a789f37ff6"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_703",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "f519788ceab3b8ad6d48a7b84a91bdd4296ac1e3e4cea0884690f996b8676442"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "f519788ceab3b8ad6d48a7b84a91bdd4296ac1e3e4cea0884690f996b8676442"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_801",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ef6592e754a943c60f47eed2a8cdad5eb0dbf6ffd991a240942cff5fee038840"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ef6592e754a943c60f47eed2a8cdad5eb0dbf6ffd991a240942cff5fee038840"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_804",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "f313208def1bd0fd0de73c1d6c548448f312f3b78c01e160c96d492feee0dfd9"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "f313208def1bd0fd0de73c1d6c548448f312f3b78c01e160c96d492feee0dfd9"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_812",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "d6e032c1a457e3b8a3a57b14d326baf0b7578b88d8940dba9792a9b0e10b1e85"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "d6e032c1a457e3b8a3a57b14d326baf0b7578b88d8940dba9792a9b0e10b1e85"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_815",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "bbe1cd2dfa98a452bfc201211dbe561188f367cccef08a2b776775d14fd0b126"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "bbe1cd2dfa98a452bfc201211dbe561188f367cccef08a2b776775d14fd0b126"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_913",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ffd8dda8ff197a68f15755f30f30bd17233bd5a10ad62d03d19b4306aab28eb9"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ffd8dda8ff197a68f15755f30f30bd17233bd5a10ad62d03d19b4306aab28eb9"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_916",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "37b0046c876d835567e3a5317a38c0ae7b3c6d45f926f24c6be3df0a73e36d9e"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "37b0046c876d835567e3a5317a38c0ae7b3c6d45f926f24c6be3df0a73e36d9e"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_924",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "e44fd4eaf5c3e4a278be5e6d55ee9b7cc981ec1b6f82a4a5120678683e77842b"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "e44fd4eaf5c3e4a278be5e6d55ee9b7cc981ec1b6f82a4a5120678683e77842b"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_927",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "4b33d0562f09df57a1e41e14b5ce574cd3e6b759f17ddcfaa27bde870848442c"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "4b33d0562f09df57a1e41e14b5ce574cd3e6b759f17ddcfaa27bde870848442c"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1025",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "98032d09775f8e6edb065644f1cfefb3fe54d5d12e90f9610ce3b5895d556274"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "98032d09775f8e6edb065644f1cfefb3fe54d5d12e90f9610ce3b5895d556274"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1028",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "30e7396c81a728cdbad90bc00a0888f83128635f102c8fa9f26602c3a4773fc4"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "30e7396c81a728cdbad90bc00a0888f83128635f102c8fa9f26602c3a4773fc4"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1036",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "c15ae30b6f93add6a6044942d5be7b4fd979cb8840b863f38fa30fca1db0e692"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "c15ae30b6f93add6a6044942d5be7b4fd979cb8840b863f38fa30fca1db0e692"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1039",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "a94a018ace24ea3720d3c1174e69a94dc65604ddb7d3cd88f11f8dc130517720"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "a94a018ace24ea3720d3c1174e69a94dc65604ddb7d3cd88f11f8dc130517720"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1137",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "36a23c2ca310da3c9c4db524eca4f29a52ad1df11be416c6ea26ae9b3a71f2c5"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "36a23c2ca310da3c9c4db524eca4f29a52ad1df11be416c6ea26ae9b3a71f2c5"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1140",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "a392f719876e9dc7e4b72714912fe4252193385974a7e2d0cecb5264ee229a19"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "a392f719876e9dc7e4b72714912fe4252193385974a7e2d0cecb5264ee229a19"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1148",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "8e196a46834478ffa55709c06ac9578a065de6a72a1d2c99d6f183ba22fefa02"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "8e196a46834478ffa55709c06ac9578a065de6a72a1d2c99d6f183ba22fefa02"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1151",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "5432a4590e811498cf009b0c1a5e10290293edc63ffcce75a48e7945dd4cb522"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "5432a4590e811498cf009b0c1a5e10290293edc63ffcce75a48e7945dd4cb522"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1249",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "a7ea00b9c61e9e4d5c270e8ecc61dfc98dd38d14c509ac3f40cf4b7c95ee71c4"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "a7ea00b9c61e9e4d5c270e8ecc61dfc98dd38d14c509ac3f40cf4b7c95ee71c4"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1252",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "80cbf37a7cca157b98dcaf4530adb2bc81102d77b3013df388184ec7a05800cd"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "80cbf37a7cca157b98dcaf4530adb2bc81102d77b3013df388184ec7a05800cd"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1260",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "78d077323bb28eaaad50b97790617dd3003091be44efd6c12505f321f470993f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "78d077323bb28eaaad50b97790617dd3003091be44efd6c12505f321f470993f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1263",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "ebee9cd09721be07e63f909754467e0d2e8496c85728cd2042f8ef67d8c41c7d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "ebee9cd09721be07e63f909754467e0d2e8496c85728cd2042f8ef67d8c41c7d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1361",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ce7fa36ccc306ae8d65ec6b1409fddc662442208f7440b3642f12151fc4e4a80"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ce7fa36ccc306ae8d65ec6b1409fddc662442208f7440b3642f12151fc4e4a80"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1364",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "7846a8cb9e40106940a0d828ccb9b9115728f4a383ce508f0e46ad798f117a7f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "7846a8cb9e40106940a0d828ccb9b9115728f4a383ce508f0e46ad798f117a7f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1372",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "7989c676498349efd3aeb496dbcc0c4b253724bd72d2cf343109b0eb24140e22"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "7989c676498349efd3aeb496dbcc0c4b253724bd72d2cf343109b0eb24140e22"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1375",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "c71b276fa63e7e00adeaf14b0a59c0e04305b774b497ed21d87faee4131268db"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "c71b276fa63e7e00adeaf14b0a59c0e04305b774b497ed21d87faee4131268db"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1473",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ee3cb96d693f39268f52e016ca6a5186902876f0c52c1b27c53f0c40cacdf74d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "ee3cb96d693f39268f52e016ca6a5186902876f0c52c1b27c53f0c40cacdf74d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1476",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "bc1ef53ca3bc9f8dcac319da237ca65c8ae2f254577468017c46a01a39b565ac"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "bc1ef53ca3bc9f8dcac319da237ca65c8ae2f254577468017c46a01a39b565ac"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1484",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "c834c09b25e38f5b2832212a02933012bbd1d94c2de03633b79c4e35a9995a54"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "c834c09b25e38f5b2832212a02933012bbd1d94c2de03633b79c4e35a9995a54"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1487",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "be1184abcf55828ca61de038a993890158404f3f0b99327b096bdb767f0487c1"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "be1184abcf55828ca61de038a993890158404f3f0b99327b096bdb767f0487c1"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1585",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "236fa7b232e5a184540bcfa388a2a10e321d3fd8d38c57c029ff6b4df545539c"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "236fa7b232e5a184540bcfa388a2a10e321d3fd8d38c57c029ff6b4df545539c"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1588",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "2c9d6a92b8835f9b96bdc7163603d0f4163dd75f40a8267260b036defb53df24"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "2c9d6a92b8835f9b96bdc7163603d0f4163dd75f40a8267260b036defb53df24"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1596",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "b9f4f69481af0c60a3b211f17185559db75c88df9ae2f02df2c5d5167974509d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "b9f4f69481af0c60a3b211f17185559db75c88df9ae2f02df2c5d5167974509d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1599",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d7cfd772f3ab5cf07cd9dc919ca81a514d3cc8ca8c4087feb95f398cf64c6cf9"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d7cfd772f3ab5cf07cd9dc919ca81a514d3cc8ca8c4087feb95f398cf64c6cf9"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1697",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "f20d3dc40e93f093834765294b75a8d15660bfc1959d531818a616291b7b8442"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "f20d3dc40e93f093834765294b75a8d15660bfc1959d531818a616291b7b8442"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1700",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d25d70fc6901e433fe5492524791467f79fcbf6a11deef299197cf375a17efd5"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d25d70fc6901e433fe5492524791467f79fcbf6a11deef299197cf375a17efd5"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1708",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "ba354e24a72a8a23e7f58361ccf510caaf0d9b7d9923e39580a3a08ba7d80da3"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "ba354e24a72a8a23e7f58361ccf510caaf0d9b7d9923e39580a3a08ba7d80da3"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1711",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "fed35fdf6c4c5ceff51caee65a57fc3b5e904bdf0b0ccd2b44a8d1eeeadd2b9d"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "fed35fdf6c4c5ceff51caee65a57fc3b5e904bdf0b0ccd2b44a8d1eeeadd2b9d"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1809",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "01e078ed1dd0e0d3dacca2f2ed141f53e56b5351b36f464ee46df54359add709"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "01e078ed1dd0e0d3dacca2f2ed141f53e56b5351b36f464ee46df54359add709"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1812",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d33c7aca0b242cc8c20b582fc15e2be4677ecb06ffd13a2bfd2ca8f2b326d3f7"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "d33c7aca0b242cc8c20b582fc15e2be4677ecb06ffd13a2bfd2ca8f2b326d3f7"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1820",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "96e833cda7bd5e4e07d8b2f683bd320fbab937b28d5ef9c40de3e6bf0552bed4"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "96e833cda7bd5e4e07d8b2f683bd320fbab937b28d5ef9c40de3e6bf0552bed4"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1823",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "0de0b167335f6b43428187fd245a46af30715e07e9cb9967a5e79285801589ee"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "0de0b167335f6b43428187fd245a46af30715e07e9cb9967a5e79285801589ee"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1921",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "c60c45ecf572a04b2b8d1f46f6a8cc43e06b9408df491af96f0cf417dfb9250b"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "c60c45ecf572a04b2b8d1f46f6a8cc43e06b9408df491af96f0cf417dfb9250b"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1924",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "fc62dbdc9941f453697c34f03c58b5aa456bbc4d089d017d33cde4160ccc1227"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "fc62dbdc9941f453697c34f03c58b5aa456bbc4d089d017d33cde4160ccc1227"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1932",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "36c0124c31b25a28c8513e6fb6e76da4dcd5363608488132dfb6dcd9b32d1089"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "36c0124c31b25a28c8513e6fb6e76da4dcd5363608488132dfb6dcd9b32d1089"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_1935",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "ada22463330a4c1f4320e35658c172b33991adfeea51364312463422e4e222f5"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "ada22463330a4c1f4320e35658c172b33991adfeea51364312463422e4e222f5"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2033",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "70cc5eb6789f65048feabbef791578ba8999378486ac1810a009fdb32011f0ef"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "70cc5eb6789f65048feabbef791578ba8999378486ac1810a009fdb32011f0ef"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2036",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "008b89f21bd44be6c8811207df9a06e4a1ca4f31a315a9ea205e2306e311d259"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "008b89f21bd44be6c8811207df9a06e4a1ca4f31a315a9ea205e2306e311d259"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2044",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "cce331508f995bb7d29459b9aa1e0d7cfc3d4b33a1c9dff94babb528b8067e65"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "cce331508f995bb7d29459b9aa1e0d7cfc3d4b33a1c9dff94babb528b8067e65"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2047",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "fc31ba338408e8730eb0b26c63b95e6d1be5e5aeed8b76222e8e446bcb1910a9"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "fc31ba338408e8730eb0b26c63b95e6d1be5e5aeed8b76222e8e446bcb1910a9"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2145",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "edf5bb47f0e59e734dc7f987681f3bee65d5ef565b83e16b0141ae469bde4e19"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "edf5bb47f0e59e734dc7f987681f3bee65d5ef565b83e16b0141ae469bde4e19"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2148",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "82bc5667cd87d00e1ef2a50bc426c81b349d9101875e71cc9ddbc71a2c6c6b8b"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "82bc5667cd87d00e1ef2a50bc426c81b349d9101875e71cc9ddbc71a2c6c6b8b"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2156",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "da38d2976197d7cc0b4254db950bf05872154a7348ac342be82a4cf5e73f7ffc"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "da38d2976197d7cc0b4254db950bf05872154a7348ac342be82a4cf5e73f7ffc"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2159",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "f8bfd5a29a791953706db4054a6d919afad6c60ebcdb895073e7a07b9114a3e8"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "f8bfd5a29a791953706db4054a6d919afad6c60ebcdb895073e7a07b9114a3e8"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2257",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "2a3b85b4d57775aa86a0f5c002bbd8618578da4d936fd3f2f9a8c45fe9e8bbef"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "2a3b85b4d57775aa86a0f5c002bbd8618578da4d936fd3f2f9a8c45fe9e8bbef"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2260",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "eb7e563deb048799ecf197c085e056c7119fb29be385b8ddfc738403e629c182"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "eb7e563deb048799ecf197c085e056c7119fb29be385b8ddfc738403e629c182"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2268",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "371251338421ded1360e2d94b3f7889cad6532cacf1ca857050c319ff98e2615"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "371251338421ded1360e2d94b3f7889cad6532cacf1ca857050c319ff98e2615"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2271",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "34bc0a55945385aebcdb22c14caa718ce61ba0abad31962b5d4fce4cad7849f4"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "34bc0a55945385aebcdb22c14caa718ce61ba0abad31962b5d4fce4cad7849f4"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2369",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "47304e7e777c0e77c355ecd16b24c0f902653c4eb7327d699b2b96c760d2a1b4"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "47304e7e777c0e77c355ecd16b24c0f902653c4eb7327d699b2b96c760d2a1b4"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2372",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "21a15f49ade90cf9991f8eb57e6cc21ad04bbb1fbe030cae4a4b5453a9f9b3b6"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "21a15f49ade90cf9991f8eb57e6cc21ad04bbb1fbe030cae4a4b5453a9f9b3b6"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2380",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "76379add6ddbaee76a60f5af868e43c58b79fda94b08321923f7fe3c980db2e2"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "76379add6ddbaee76a60f5af868e43c58b79fda94b08321923f7fe3c980db2e2"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2383",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "edd7b1b1fcb1313c4553cefc7c3c0fd41bfc088c90b91b8c0120635344069bff"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "edd7b1b1fcb1313c4553cefc7c3c0fd41bfc088c90b91b8c0120635344069bff"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2481",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "1206c826a833c8bc5065c67c951ffded36e78b5ab4542d5b361e56ee57aa429f"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          768
+        ],
+        "bytes": 2359296,
+        "sha256": "1206c826a833c8bc5065c67c951ffded36e78b5ab4542d5b361e56ee57aa429f"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2484",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "475c5ef0d11462f712f17503b9ffd115ab59c506eb7f2830a0e8f548d952a679"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          768,
+          2304
+        ],
+        "bytes": 7077888,
+        "sha256": "475c5ef0d11462f712f17503b9ffd115ab59c506eb7f2830a0e8f548d952a679"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_2492",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "7798fc32a103d4939230fb708376ace81655ac476b8c748d59eede47113e2e98"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [
+          1152,
+          768
+        ],
+        "bytes": 3538944,
+        "sha256": "7798fc32a103d4939230fb708376ace81655ac476b8c748d59eede47113e2e98"
+      },
+      "external": true,
+      "passed": true
+    },
+    {
+      "name": "val_8",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "7c9fa136d4413fa6173637e883b6998d32e1d675f88cddff9dcbcf331820f4b8"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "7c9fa136d4413fa6173637e883b6998d32e1d675f88cddff9dcbcf331820f4b8"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_9",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "d86e8112f3c4c4442126f8e9f44f16867da487f29052bf91b810457db34209a4"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "d86e8112f3c4c4442126f8e9f44f16867da487f29052bf91b810457db34209a4"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_27",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "12a3ae445661ce5dee78d0650d33362dec29c4f82af05e7e57fb595bbbacf0ca"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "12a3ae445661ce5dee78d0650d33362dec29c4f82af05e7e57fb595bbbacf0ca"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_29",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "220dde27afee4d537cac96d85d6546f825153b90e828931b74e103807541bc42"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "220dde27afee4d537cac96d85d6546f825153b90e828931b74e103807541bc42"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_30",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "a06f129fc52abf6085679d7cd71dc41ec7580c7f5f73efef6d02dde22bb00994"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "a06f129fc52abf6085679d7cd71dc41ec7580c7f5f73efef6d02dde22bb00994"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_106",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "81b37f7f8b78762feb067131152a700548754f937a0668c1f327d2ee1a2d2801"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "81b37f7f8b78762feb067131152a700548754f937a0668c1f327d2ee1a2d2801"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_108",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "e6ad6c9a3a3b7658c35bacf6553fcb8ffe34387534a648fe18f875b8f7a86ddb"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "e6ad6c9a3a3b7658c35bacf6553fcb8ffe34387534a648fe18f875b8f7a86ddb"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_126",
+      "original": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "8285d24b1ca2820ef8beda7b35738ebb281870d192a06328d284dc3177f4fe31"
+      },
+      "final": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          1
+        ],
+        "bytes": 8,
+        "sha256": "8285d24b1ca2820ef8beda7b35738ebb281870d192a06328d284dc3177f4fe31"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_133",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "9c16095ed50248c0e1658be07f5bafe5dc11382843a99c6fb3a87c141808ceeb"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "9c16095ed50248c0e1658be07f5bafe5dc11382843a99c6fb3a87c141808ceeb"
+      },
+      "external": false,
+      "passed": true
+    },
+    {
+      "name": "val_138",
+      "original": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "d99e58435243d9fef9c88273b8d553b4fba4d0baf8009d29eae74fa99e0d9f57"
+      },
+      "final": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "d99e58435243d9fef9c88273b8d553b4fba4d0baf8009d29eae74fa99e0d9f57"
+      },
+      "external": false,
+      "passed": true
+    }
+  ],
+  "removed_original_initializers": [
+    {
+      "name": "val_17",
+      "identity": {
+        "dtype": "FLOAT",
+        "dtype_id": 1,
+        "shape": [],
+        "bytes": 4,
+        "sha256": "abc4c4e7eee2ce04c6be04358adf5b3d0958255a80f712a8b6d4f4fdda588276"
+      }
+    },
+    {
+      "name": "val_20",
+      "identity": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [],
+        "bytes": 8,
+        "sha256": "a06f129fc52abf6085679d7cd71dc41ec7580c7f5f73efef6d02dde22bb00994"
+      }
+    },
+    {
+      "name": "val_2506",
+      "identity": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          2
+        ],
+        "bytes": 16,
+        "sha256": "0c730b69905c5ef7a4ca5269f72365400bde2dd2c04eaf9bbb3d1c4a265a0131"
+      }
+    },
+    {
+      "name": "val_2507",
+      "identity": {
+        "dtype": "INT64",
+        "dtype_id": 7,
+        "shape": [
+          2
+        ],
+        "bytes": 16,
+        "sha256": "9d34149fbd1fe777eb238799054c8cbfbce372255f219f8740838def9bfd02db"
+      }
+    }
+  ]
+}

evaluation/engines/vela-modality-blocked-b1-native-gpu-32k-v1.json

@@ -0,0 +1,95 @@
+{
+  "engine": "onnx",
+  "gpu": true,
+  "reference_sha256": "2942ef90b1dd97f90e97793e663d4a8f993ef66922b43c818261121fd5f68cd4",
+  "weights_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "parity": [
+    {
+      "tokens": 9,
+      "max_abs_score_error": 4.172325134277344e-07,
+      "argmax_same": true,
+      "native_ms": 392.0625450555235,
+      "input_sha256": "28a5f5d83deb4fffe1ba2c2ddd3e7d8cb5d09c2aa96080454ae3dade0c99406a"
+    },
+    {
+      "tokens": 10,
+      "max_abs_score_error": 2.3283064365386963e-08,
+      "argmax_same": true,
+      "native_ms": 15.951868961565197,
+      "input_sha256": "c2f6fc42badff8e41964d9d96ab2ec7ca95386bb1f9de6498e9e07e3ee40a743"
+    },
+    {
+      "tokens": 11,
+      "max_abs_score_error": 2.7284841053187847e-11,
+      "argmax_same": true,
+      "native_ms": 15.637201024219394,
+      "input_sha256": "d1151ac839a0385c1658a5be98b53a796c7e040c22eaeb8310e11a118132b10a"
+    },
+    {
+      "tokens": 10,
+      "max_abs_score_error": 3.055333763768431e-13,
+      "argmax_same": true,
+      "native_ms": 14.953606063500047,
+      "input_sha256": "df7c0764375e479f20c7b237daa80a3526c0d74afe23d8dbc5d678b2f368b211"
+    },
+    {
+      "tokens": 63,
+      "max_abs_score_error": 7.130438461899757e-10,
+      "argmax_same": true,
+      "native_ms": 15.710555016994476,
+      "input_sha256": "ae2ff5297ca5473f882a421ea7055e249c6096ce24e40a929ade9dc1b0760862"
+    },
+    {
+      "tokens": 64,
+      "max_abs_score_error": 1.6007106751203537e-10,
+      "argmax_same": true,
+      "native_ms": 15.440835035406053,
+      "input_sha256": "2ee0d8dc7879369d6e6827e3be54f03e794dfcee1670afec37c4957e983f0b87"
+    },
+    {
+      "tokens": 65,
+      "max_abs_score_error": 2.954038791358471e-09,
+      "argmax_same": true,
+      "native_ms": 15.359019045718014,
+      "input_sha256": "d0e67fca708f0264df1513f7b0b1508692a6c42eb7bf35a662441e8bf7119d5c"
+    },
+    {
+      "tokens": 512,
+      "max_abs_score_error": 3.956301952712238e-11,
+      "argmax_same": true,
+      "native_ms": 25.833902996964753,
+      "input_sha256": "b1e6aa41d7c4c3d327fff16c873fdb165552ed315254e0c7756175bfe0e1984b"
+    },
+    {
+      "tokens": 4096,
+      "max_abs_score_error": 1.1920928955078125e-07,
+      "argmax_same": true,
+      "native_ms": 196.1183260427788,
+      "input_sha256": "38f565a661a12dc4af985b73b73b86f4e0731f74334a9fde7d9b7e5f5b0d9394"
+    },
+    {
+      "tokens": 8192,
+      "max_abs_score_error": 5.122274160385132e-09,
+      "argmax_same": true,
+      "native_ms": 779.2861249763519,
+      "input_sha256": "ec217f8d64df4984535686f985036008794c15282a464fe0cba1092825b2c788"
+    },
+    {
+      "tokens": 16384,
+      "max_abs_score_error": 1.0986695997416973e-09,
+      "argmax_same": true,
+      "native_ms": 2589.4851229386404,
+      "input_sha256": "2aa1c56673d987a494f1600f6cd36baf4f4e6b7bccf1d2892d0df9e50a3c5f69"
+    },
+    {
+      "tokens": 32768,
+      "max_abs_score_error": 1.2945383787155151e-07,
+      "argmax_same": true,
+      "native_ms": 9555.754297995009,
+      "input_sha256": "fb5e1bdb22c71316ad3c2a458e5a7dcf1453babbf643d2c0e927af53f1c08c54"
+    }
+  ],
+  "overflow_rejected": true,
+  "reopen": true,
+  "label_mismatch_rejected": true
+}

evaluation/engines/vela-modality-blocked-b1-onnx-gpu-v1.json

@@ -0,0 +1,143 @@
+{
+  "qualified": true,
+  "weights_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "reference_sha256": "2942ef90b1dd97f90e97793e663d4a8f993ef66922b43c818261121fd5f68cd4",
+  "graph_sha256": "74a08c52ce47f522071fd71acee080c0a3a74ad12e3d5fb19ae1dfd244f1059c",
+  "precision": "FP32 weights and computation, exact query-block attention",
+  "probability_atol": 0.0002,
+  "probability_rtol": 0.0001,
+  "exact_BIO_required": false,
+  "exact_argmax_required": true,
+  "task": "sequence-classification",
+  "runtime": "1.22.1",
+  "rows": [
+    {
+      "case": "b1s2",
+      "valid_tokens": [
+        2
+      ],
+      "max_logit_error": 5.4836273193359375e-06,
+      "max_probability_error": 3.259629011154175e-09,
+      "argmax_flips": 0,
+      "first_inference_ms": 416.892733075656,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s63",
+      "valid_tokens": [
+        63
+      ],
+      "max_logit_error": 2.1696090698242188e-05,
+      "max_probability_error": 4.6629367034256575e-14,
+      "argmax_flips": 0,
+      "first_inference_ms": 32.77761000208557,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s64",
+      "valid_tokens": [
+        64
+      ],
+      "max_logit_error": 1.0967254638671875e-05,
+      "max_probability_error": 2.0872192862952943e-13,
+      "argmax_flips": 0,
+      "first_inference_ms": 33.219670061953366,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s65",
+      "valid_tokens": [
+        65
+      ],
+      "max_logit_error": 1.3589859008789062e-05,
+      "max_probability_error": 3.623767952376511e-13,
+      "argmax_flips": 0,
+      "first_inference_ms": 34.18030601460487,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s127",
+      "valid_tokens": [
+        127
+      ],
+      "max_logit_error": 2.193450927734375e-05,
+      "max_probability_error": 4.6838977141305804e-11,
+      "argmax_flips": 0,
+      "first_inference_ms": 36.99377097655088,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s128",
+      "valid_tokens": [
+        128
+      ],
+      "max_logit_error": 2.288818359375e-05,
+      "max_probability_error": 2.8421709430404007e-11,
+      "argmax_flips": 0,
+      "first_inference_ms": 34.10984796937555,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s129",
+      "valid_tokens": [
+        129
+      ],
+      "max_logit_error": 2.288818359375e-05,
+      "max_probability_error": 2.6716406864579767e-11,
+      "argmax_flips": 0,
+      "first_inference_ms": 34.23338197171688,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s512",
+      "valid_tokens": [
+        512
+      ],
+      "max_logit_error": 5.900859832763672e-06,
+      "max_probability_error": 5.005858838558197e-09,
+      "argmax_flips": 0,
+      "first_inference_ms": 51.06296797748655,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s4096",
+      "valid_tokens": [
+        4096
+      ],
+      "max_logit_error": 2.6702880859375e-05,
+      "max_probability_error": 7.748603820800781e-07,
+      "argmax_flips": 0,
+      "first_inference_ms": 221.17824701126665,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s32768",
+      "valid_tokens": [
+        32768
+      ],
+      "max_logit_error": 2.0325183868408203e-05,
+      "max_probability_error": 1.1920928955078125e-06,
+      "argmax_flips": 0,
+      "first_inference_ms": 9738.925967016257,
+      "probability_gate": true
+    }
+  ],
+  "diagnostic": false,
+  "static_batch_size": 1,
+  "scope": "numerical qualification, not independent quality or isolated latency benchmark",
+  "providers": [
+    "ROCMExecutionProvider",
+    "CPUExecutionProvider"
+  ],
+  "unqualified_batch_rejected": true,
+  "profile_sha256": "705d6f134ccb875723535ca48d7458f00b53ebd8f7c04452a2c21d7436ec4637",
+  "profile_providers": {
+    "ROCMExecutionProvider": 98742,
+    "CPUExecutionProvider": 12150
+  },
+  "attention_execution": {
+    "ROCMExecutionProvider:MatMul": 6732,
+    "ROCMExecutionProvider:Softmax": 3366
+  },
+  "attention_on_gpu": true
+}

evaluation/engines/vela-modality-candle-cpu-v1.json

@@ -0,0 +1,67 @@
+{
+  "engine": "candle",
+  "gpu": false,
+  "reference_sha256": "2942ef90b1dd97f90e97793e663d4a8f993ef66922b43c818261121fd5f68cd4",
+  "weights_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "parity": [
+    {
+      "tokens": 9,
+      "max_abs_score_error": 1.329183578491211e-05,
+      "argmax_same": true,
+      "native_ms": 169.12583902012557,
+      "input_sha256": "28a5f5d83deb4fffe1ba2c2ddd3e7d8cb5d09c2aa96080454ae3dade0c99406a"
+    },
+    {
+      "tokens": 10,
+      "max_abs_score_error": 3.911089152097702e-06,
+      "argmax_same": true,
+      "native_ms": 117.81790107488632,
+      "input_sha256": "c2f6fc42badff8e41964d9d96ab2ec7ca95386bb1f9de6498e9e07e3ee40a743"
+    },
+    {
+      "tokens": 11,
+      "max_abs_score_error": 1.5072146197780967e-08,
+      "argmax_same": true,
+      "native_ms": 119.22943498939276,
+      "input_sha256": "d1151ac839a0385c1658a5be98b53a796c7e040c22eaeb8310e11a118132b10a"
+    },
+    {
+      "tokens": 10,
+      "max_abs_score_error": 4.339995030022692e-11,
+      "argmax_same": true,
+      "native_ms": 119.12891990505159,
+      "input_sha256": "df7c0764375e479f20c7b237daa80a3526c0d74afe23d8dbc5d678b2f368b211"
+    },
+    {
+      "tokens": 63,
+      "max_abs_score_error": 1.1920928955078125e-07,
+      "argmax_same": true,
+      "native_ms": 195.58919803239405,
+      "input_sha256": "ae2ff5297ca5473f882a421ea7055e249c6096ce24e40a929ade9dc1b0760862"
+    },
+    {
+      "tokens": 64,
+      "max_abs_score_error": 1.1920928955078125e-07,
+      "argmax_same": true,
+      "native_ms": 203.6561779677868,
+      "input_sha256": "2ee0d8dc7879369d6e6827e3be54f03e794dfcee1670afec37c4957e983f0b87"
+    },
+    {
+      "tokens": 65,
+      "max_abs_score_error": 1.2147938832640648e-07,
+      "argmax_same": true,
+      "native_ms": 198.97981407120824,
+      "input_sha256": "d0e67fca708f0264df1513f7b0b1508692a6c42eb7bf35a662441e8bf7119d5c"
+    },
+    {
+      "tokens": 512,
+      "max_abs_score_error": 1.90857463167049e-09,
+      "argmax_same": true,
+      "native_ms": 1900.0832010060549,
+      "input_sha256": "b1e6aa41d7c4c3d327fff16c873fdb165552ed315254e0c7756175bfe0e1984b"
+    }
+  ],
+  "overflow_rejected": true,
+  "reopen": true,
+  "label_mismatch_rejected": true
+}

evaluation/engines/vela-modality-standalone-engine-reproduction-v1.json

@@ -0,0 +1,55 @@
+{
+  "qualified": true,
+  "weights_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "reference_sha256": "bc7fa82ad8bacdb45bf605e5249abc2dca61f081c4d21533c61ff0f535f6e6fa",
+  "graph_sha256": "74a08c52ce47f522071fd71acee080c0a3a74ad12e3d5fb19ae1dfd244f1059c",
+  "precision": "FP32 weights and computation, exact query-block attention",
+  "probability_atol": 0.0002,
+  "probability_rtol": 0.0001,
+  "exact_BIO_required": false,
+  "exact_argmax_required": true,
+  "task": "sequence-classification",
+  "runtime": "1.22.1",
+  "rows": [
+    {
+      "case": "b1s2",
+      "valid_tokens": [
+        2
+      ],
+      "max_logit_error": 1.1444091796875e-05,
+      "max_probability_error": 6.100162863731384e-08,
+      "argmax_flips": 0,
+      "first_inference_ms": 408.44920300878584,
+      "probability_gate": true
+    },
+    {
+      "case": "b1s63",
+      "valid_tokens": [
+        63
+      ],
+      "max_logit_error": 4.9591064453125e-05,
+      "max_probability_error": 2.0383694732117874e-13,
+      "argmax_flips": 0,
+      "first_inference_ms": 34.35661701951176,
+      "probability_gate": true
+    }
+  ],
+  "diagnostic": false,
+  "static_batch_size": 1,
+  "scope": "numerical qualification, not independent quality or isolated latency benchmark",
+  "providers": [
+    "ROCMExecutionProvider",
+    "CPUExecutionProvider"
+  ],
+  "unqualified_batch_rejected": true,
+  "profile_sha256": "4d06390d82e383c8dedd92af2c8141dce06b10235fd6480ee7175477e727e246",
+  "profile_providers": {
+    "ROCMExecutionProvider": 4190,
+    "CPUExecutionProvider": 2430
+  },
+  "attention_execution": {
+    "ROCMExecutionProvider:MatMul": 88,
+    "ROCMExecutionProvider:Softmax": 44
+  },
+  "attention_on_gpu": true
+}

evaluation/final-baseline-fp32.json

@@ -0,0 +1,1445 @@
+{
+  "rows": 1411,
+  "accuracy": 0.4599574769666903,
+  "macro_f1": 0.3669769557178275,
+  "macro_f1_present_labels": 0.3669769557178275,
+  "per_label": {
+    "AR": {
+      "support": 643,
+      "tp": 294,
+      "fp": 70,
+      "fn": 349,
+      "precision": 0.8076923076923077,
+      "recall": 0.4572317262830482,
+      "f1": 0.5839126117179742
+    },
+    "DIFFUSION": {
+      "support": 384,
+      "tp": 351,
+      "fp": 677,
+      "fn": 33,
+      "precision": 0.3414396887159533,
+      "recall": 0.9140625,
+      "f1": 0.4971671388101983
+    },
+    "BOTH": {
+      "support": 384,
+      "tp": 4,
+      "fp": 15,
+      "fn": 380,
+      "precision": 0.21052631578947367,
+      "recall": 0.010416666666666666,
+      "f1": 0.019851116625310174
+    }
+  },
+  "confusion": [
+    [
+      294,
+      335,
+      14
+    ],
+    [
+      32,
+      351,
+      1
+    ],
+    [
+      38,
+      342,
+      4
+    ]
+  ],
+  "breakdowns": {
+    "source": {
+      "academic-text-answer": {
+        "rows": 240,
+        "accuracy": 0.9666666666666667,
+        "macro_f1": 0.3276836158192091,
+        "macro_f1_present_labels": 0.9830508474576272,
+        "per_label": {
+          "AR": {
+            "support": 240,
+            "tp": 232,
+            "fp": 0,
+            "fn": 8,
+            "precision": 1.0,
+            "recall": 0.9666666666666667,
+            "f1": 0.9830508474576272
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 8,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            232,
+            0,
+            8
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "authored-context-on-frozen-source": {
+        "rows": 72,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 24,
+            "tp": 12,
+            "fp": 24,
+            "fn": 12,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 24,
+            "tp": 12,
+            "fp": 24,
+            "fn": 12,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 24,
+            "tp": 0,
+            "fp": 0,
+            "fn": 24,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            12,
+            12,
+            0
+          ],
+          [
+            12,
+            12,
+            0
+          ],
+          [
+            12,
+            12,
+            0
+          ]
+        ]
+      },
+      "authored-output-contract-with-gallery-caption": {
+        "rows": 1080,
+        "accuracy": 0.34629629629629627,
+        "macro_f1": 0.2216396874882791,
+        "macro_f1_present_labels": 0.2216396874882791,
+        "per_label": {
+          "AR": {
+            "support": 360,
+            "tp": 31,
+            "fp": 46,
+            "fn": 329,
+            "precision": 0.4025974025974026,
+            "recall": 0.08611111111111111,
+            "f1": 0.14187643020594964
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 339,
+            "fp": 653,
+            "fn": 21,
+            "precision": 0.34173387096774194,
+            "recall": 0.9416666666666667,
+            "f1": 0.5014792899408284
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 4,
+            "fp": 7,
+            "fn": 356,
+            "precision": 0.36363636363636365,
+            "recall": 0.011111111111111112,
+            "f1": 0.0215633423180593
+          }
+        },
+        "confusion": [
+          [
+            31,
+            323,
+            6
+          ],
+          [
+            20,
+            339,
+            1
+          ],
+          [
+            26,
+            330,
+            4
+          ]
+        ]
+      },
+      "aya-original-human": {
+        "rows": 19,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 19,
+            "tp": 19,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            19,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      }
+    },
+    "language": {
+      "de": {
+        "rows": 220,
+        "accuracy": 0.5,
+        "macro_f1": 0.38727061556329856,
+        "macro_f1_present_labels": 0.38727061556329856,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 53,
+            "fp": 22,
+            "fn": 47,
+            "precision": 0.7066666666666667,
+            "recall": 0.53,
+            "f1": 0.6057142857142858
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 57,
+            "fp": 88,
+            "fn": 3,
+            "precision": 0.3931034482758621,
+            "recall": 0.95,
+            "f1": 0.5560975609756098
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 0,
+            "fp": 0,
+            "fn": 60,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            53,
+            47,
+            0
+          ],
+          [
+            3,
+            57,
+            0
+          ],
+          [
+            19,
+            41,
+            0
+          ]
+        ]
+      },
+      "deu": {
+        "rows": 1,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 1,
+            "tp": 1,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            1,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "en": {
+        "rows": 256,
+        "accuracy": 0.43359375,
+        "macro_f1": 0.33885209713024284,
+        "macro_f1_present_labels": 0.33885209713024284,
+        "per_label": {
+          "AR": {
+            "support": 112,
+            "tp": 39,
+            "fp": 0,
+            "fn": 73,
+            "precision": 1.0,
+            "recall": 0.3482142857142857,
+            "f1": 0.5165562913907285
+          },
+          "DIFFUSION": {
+            "support": 72,
+            "tp": 72,
+            "fp": 144,
+            "fn": 0,
+            "precision": 0.3333333333333333,
+            "recall": 1.0,
+            "f1": 0.5
+          },
+          "BOTH": {
+            "support": 72,
+            "tp": 0,
+            "fp": 1,
+            "fn": 72,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            39,
+            72,
+            1
+          ],
+          [
+            0,
+            72,
+            0
+          ],
+          [
+            0,
+            72,
+            0
+          ]
+        ]
+      },
+      "eng": {
+        "rows": 1,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 1,
+            "tp": 1,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            1,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "es": {
+        "rows": 220,
+        "accuracy": 0.4727272727272727,
+        "macro_f1": 0.37387387387387383,
+        "macro_f1_present_labels": 0.37387387387387383,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 46,
+            "fp": 2,
+            "fn": 54,
+            "precision": 0.9583333333333334,
+            "recall": 0.46,
+            "f1": 0.6216216216216216
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 58,
+            "fp": 114,
+            "fn": 2,
+            "precision": 0.3372093023255814,
+            "recall": 0.9666666666666667,
+            "f1": 0.5
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 0,
+            "fp": 0,
+            "fn": 60,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            46,
+            54,
+            0
+          ],
+          [
+            2,
+            58,
+            0
+          ],
+          [
+            0,
+            60,
+            0
+          ]
+        ]
+      },
+      "fr": {
+        "rows": 220,
+        "accuracy": 0.45454545454545453,
+        "macro_f1": 0.3553057799633142,
+        "macro_f1_present_labels": 0.3553057799633142,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 41,
+            "fp": 5,
+            "fn": 59,
+            "precision": 0.8913043478260869,
+            "recall": 0.41,
+            "f1": 0.5616438356164384
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 59,
+            "fp": 115,
+            "fn": 1,
+            "precision": 0.3390804597701149,
+            "recall": 0.9833333333333333,
+            "f1": 0.5042735042735043
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 0,
+            "fp": 0,
+            "fn": 60,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            41,
+            59,
+            0
+          ],
+          [
+            1,
+            59,
+            0
+          ],
+          [
+            4,
+            56,
+            0
+          ]
+        ]
+      },
+      "fra": {
+        "rows": 15,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 15,
+            "tp": 15,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            15,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "ja": {
+        "rows": 220,
+        "accuracy": 0.44545454545454544,
+        "macro_f1": 0.3759456749612615,
+        "macro_f1_present_labels": 0.3759456749612615,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 46,
+            "fp": 13,
+            "fn": 54,
+            "precision": 0.7796610169491526,
+            "recall": 0.46,
+            "f1": 0.5786163522012578
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 49,
+            "fp": 103,
+            "fn": 11,
+            "precision": 0.3223684210526316,
+            "recall": 0.8166666666666667,
+            "f1": 0.46226415094339623
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 3,
+            "fp": 6,
+            "fn": 57,
+            "precision": 0.3333333333333333,
+            "recall": 0.05,
+            "f1": 0.08695652173913043
+          }
+        },
+        "confusion": [
+          [
+            46,
+            48,
+            6
+          ],
+          [
+            11,
+            49,
+            0
+          ],
+          [
+            2,
+            55,
+            3
+          ]
+        ]
+      },
+      "spa": {
+        "rows": 2,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 2,
+            "tp": 2,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            2,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "zh": {
+        "rows": 256,
+        "accuracy": 0.41796875,
+        "macro_f1": 0.3385791459849357,
+        "macro_f1_present_labels": 0.3385791459849357,
+        "per_label": {
+          "AR": {
+            "support": 112,
+            "tp": 50,
+            "fp": 28,
+            "fn": 62,
+            "precision": 0.6410256410256411,
+            "recall": 0.44642857142857145,
+            "f1": 0.5263157894736842
+          },
+          "DIFFUSION": {
+            "support": 72,
+            "tp": 56,
+            "fp": 113,
+            "fn": 16,
+            "precision": 0.33136094674556216,
+            "recall": 0.7777777777777778,
+            "f1": 0.46473029045643155
+          },
+          "BOTH": {
+            "support": 72,
+            "tp": 1,
+            "fp": 8,
+            "fn": 71,
+            "precision": 0.1111111111111111,
+            "recall": 0.013888888888888888,
+            "f1": 0.024691358024691357
+          }
+        },
+        "confusion": [
+          [
+            50,
+            55,
+            7
+          ],
+          [
+            15,
+            56,
+            1
+          ],
+          [
+            13,
+            58,
+            1
+          ]
+        ]
+      }
+    },
+    "length_bucket": {
+      "16384": {
+        "rows": 18,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 0,
+            "fp": 0,
+            "fn": 6,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ]
+        ]
+      },
+      "32768": {
+        "rows": 18,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 0,
+            "fp": 0,
+            "fn": 6,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ]
+        ]
+      },
+      "4096": {
+        "rows": 18,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 0,
+            "fp": 0,
+            "fn": 6,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ]
+        ]
+      },
+      "8192": {
+        "rows": 18,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 6,
+            "fn": 3,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 0,
+            "fp": 0,
+            "fn": 6,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ]
+        ]
+      },
+      "short": {
+        "rows": 1339,
+        "accuracy": 0.46676624346527257,
+        "macro_f1": 0.37271747042740394,
+        "macro_f1_present_labels": 0.37271747042740394,
+        "per_label": {
+          "AR": {
+            "support": 619,
+            "tp": 282,
+            "fp": 46,
+            "fn": 337,
+            "precision": 0.8597560975609756,
+            "recall": 0.4555735056542811,
+            "f1": 0.5955649419218585
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 339,
+            "fp": 653,
+            "fn": 21,
+            "precision": 0.34173387096774194,
+            "recall": 0.9416666666666667,
+            "f1": 0.5014792899408284
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 4,
+            "fp": 15,
+            "fn": 356,
+            "precision": 0.21052631578947367,
+            "recall": 0.011111111111111112,
+            "f1": 0.021108179419525065
+          }
+        },
+        "confusion": [
+          [
+            282,
+            323,
+            14
+          ],
+          [
+            20,
+            339,
+            1
+          ],
+          [
+            26,
+            330,
+            4
+          ]
+        ]
+      }
+    },
+    "position": {
+      "head": {
+        "rows": 24,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 8,
+            "fn": 4,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 4,
+            "fp": 8,
+            "fn": 4,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 0,
+            "fp": 0,
+            "fn": 8,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            4,
+            4,
+            0
+          ]
+        ]
+      },
+      "middle": {
+        "rows": 24,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 8,
+            "fn": 4,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 4,
+            "fp": 8,
+            "fn": 4,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 0,
+            "fp": 0,
+            "fn": 8,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            4,
+            4,
+            0
+          ]
+        ]
+      },
+      "tail": {
+        "rows": 24,
+        "accuracy": 0.3333333333333333,
+        "macro_f1": 0.26666666666666666,
+        "macro_f1_present_labels": 0.26666666666666666,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 8,
+            "fn": 4,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 4,
+            "fp": 8,
+            "fn": 4,
+            "precision": 0.3333333333333333,
+            "recall": 0.5,
+            "f1": 0.4
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 0,
+            "fp": 0,
+            "fn": 8,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            4,
+            4,
+            0
+          ]
+        ]
+      },
+      "unspecified": {
+        "rows": 1339,
+        "accuracy": 0.46676624346527257,
+        "macro_f1": 0.37271747042740394,
+        "macro_f1_present_labels": 0.37271747042740394,
+        "per_label": {
+          "AR": {
+            "support": 619,
+            "tp": 282,
+            "fp": 46,
+            "fn": 337,
+            "precision": 0.8597560975609756,
+            "recall": 0.4555735056542811,
+            "f1": 0.5955649419218585
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 339,
+            "fp": 653,
+            "fn": 21,
+            "precision": 0.34173387096774194,
+            "recall": 0.9416666666666667,
+            "f1": 0.5014792899408284
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 4,
+            "fp": 15,
+            "fn": 356,
+            "precision": 0.21052631578947367,
+            "recall": 0.011111111111111112,
+            "f1": 0.021108179419525065
+          }
+        },
+        "confusion": [
+          [
+            282,
+            323,
+            14
+          ],
+          [
+            20,
+            339,
+            1
+          ],
+          [
+            26,
+            330,
+            4
+          ]
+        ]
+      }
+    }
+  },
+  "actual_tokens": {
+    "min": 10,
+    "max": 32768
+  },
+  "forward_batches_ms": [
+    6446.7283230042085,
+    10.122793028131127,
+    9.915609029121697,
+    10.60864597093314,
+    9.46409406606108,
+    9.408487007021904,
+    9.18875599745661,
+    9.207460912875831,
+    9.627168998122215,
+    9.453007951378822,
+    9.44347190670669,
+    9.709913050755858,
+    9.246921981684864,
+    9.530481998808682,
+    9.563550935126841,
+    9.204318979755044,
+    12.11533194873482,
+    10.337620973587036,
+    10.059229098260403,
+    9.48656804393977,
+    9.727524011395872,
+    9.305126033723354,
+    9.306956082582474,
+    9.370358078740537,
+    9.624226950109005,
+    9.914231020957232,
+    9.723542025312781,
+    9.962645010091364,
+    8.913283003494143,
+    8.884695009328425,
+    9.641692042350769,
+    9.30216396227479,
+    9.33746702503413,
+    9.300264995545149,
+    9.47726599406451,
+    9.318343014456332,
+    9.37076995614916,
+    9.122074930928648,
+    9.680890012532473,
+    9.092447930015624,
+    9.602330974303186,
+    9.138197056017816,
+    9.636719012632966,
+    9.29853692650795,
+    9.532617987133563,
+    9.35582397505641,
+    9.25972405821085,
+    9.057466988451779,
+    9.71064600162208,
+    9.222558932378888,
+    9.376573958434165,
+    9.44302196148783,
+    9.570201043970883,
+    9.22214100137353,
+    9.490184020251036,
+    9.412553044967353,
+    9.052847046405077,
+    8.90732102561742,
+    9.635501075536013,
+    9.422325994819403,
+    9.483458008617163,
+    9.357245056889951,
+    9.323348058387637,
+    9.352453052997589,
+    9.479686967097223,
+    9.523665066808462,
+    10.155343101359904,
+    11.546792928129435,
+    10.459608980454504,
+    10.044816066510975,
+    9.434658102691174,
+    9.269515983760357,
+    9.521526051685214,
+    9.515123907476664,
+    9.610714972950518,
+    11.145189986564219,
+    10.389451985247433,
+    9.879508055746555,
+    12.027611024677753,
+    14.165389002300799,
+    18.497899058274925,
+    13.323834049515426,
+    25.66497807856649,
+    10.81712101586163,
+    9.532027994282544,
+    9.62286302819848,
+    9.576256037689745,
+    9.407049044966698,
+    9.413274936378002,
+    9.253080934286118,
+    9.664084063842893,
+    9.84158890787512,
+    9.16440796572715,
+    9.031309047713876,
+    92.17983298003674,
+    94.1256070509553,
+    88.64477602764964,
+    86.3022570265457,
+    89.82986700721085,
+    92.94207894708961,
+    93.198585906066,
+    85.87321802042425,
+    88.31606793683022,
+    85.75395005755126,
+    83.37531797587872,
+    85.72892798110843,
+    88.96502095740288,
+    86.0117319971323,
+    86.33086597546935,
+    86.55514393467456,
+    85.48676606733352,
+    85.1146939676255,
+    277.01961307320744,
+    284.71940592862666,
+    283.83382700849324,
+    291.9321529334411,
+    340.95058403909206,
+    273.0076120933518,
+    272.6765110855922,
+    283.3383050747216,
+    269.2251419648528,
+    283.6749879643321,
+    283.6119490675628,
+    278.4766390686855,
+    277.30921108741313,
+    290.81435909029096,
+    280.8271209942177,
+    273.62499199807644,
+    273.96527596283704,
+    268.319983035326,
+    1035.654591047205,
+    1062.3674780363217,
+    1062.9829469835386,
+    1018.123218906112,
+    1085.1643879432231,
+    1091.3887589704245,
+    1005.2773610223085,
+    1050.7057759677991,
+    1105.7434329995885,
+    1062.8094230778515,
+    1055.2198939258233,
+    1032.5770160416141,
+    1053.3472939860076,
+    1077.022343990393,
+    1085.838780971244,
+    1035.0606159772724,
+    1080.3039139136672,
+    1039.5317010115832,
+    3934.3111380003393,
+    3868.6304059810936,
+    4303.185440949164,
+    4105.504778912291,
+    4010.3370539145544,
+    3995.786834973842,
+    4333.715492975898,
+    4184.374890988693,
+    4037.6546720508486,
+    4065.9116219030693,
+    4104.565493064001,
+    4185.04842906259,
+    4191.750784986652,
+    4087.02986210119,
+    4069.8690769495443,
+    4116.871495964006,
+    4112.496968009509,
+    4220.928212045692
+  ],
+  "precision": {
+    "parameters": "float32",
+    "forward_autocast": null,
+    "device_type": "cuda"
+  },
+  "data_files": [
+    {
+      "file": "test.jsonl",
+      "sha256": "20700706f7fc8876bf1b1235689d16c5ce4512fba35c70bc043abeb436b0cc53"
+    },
+    {
+      "file": "test.jsonl",
+      "sha256": "fa8ec4fc2a6398ce0ef078bb9f4998e0bf120922afc4a77ff1fed4b2c17e1d4c"
+    }
+  ]
+}

evaluation/final-candidate-fp32.json

@@ -0,0 +1,1445 @@
+{
+  "rows": 1411,
+  "accuracy": 0.9312544294826365,
+  "macro_f1": 0.9274533214132176,
+  "macro_f1_present_labels": 0.9274533214132176,
+  "per_label": {
+    "AR": {
+      "support": 643,
+      "tp": 632,
+      "fp": 73,
+      "fn": 11,
+      "precision": 0.8964539007092198,
+      "recall": 0.9828926905132193,
+      "f1": 0.9376854599406528
+    },
+    "DIFFUSION": {
+      "support": 384,
+      "tp": 302,
+      "fp": 9,
+      "fn": 82,
+      "precision": 0.9710610932475884,
+      "recall": 0.7864583333333334,
+      "f1": 0.8690647482014389
+    },
+    "BOTH": {
+      "support": 384,
+      "tp": 380,
+      "fp": 15,
+      "fn": 4,
+      "precision": 0.9620253164556962,
+      "recall": 0.9895833333333334,
+      "f1": 0.975609756097561
+    }
+  },
+  "confusion": [
+    [
+      632,
+      5,
+      6
+    ],
+    [
+      73,
+      302,
+      9
+    ],
+    [
+      0,
+      4,
+      380
+    ]
+  ],
+  "breakdowns": {
+    "source": {
+      "academic-text-answer": {
+        "rows": 240,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 240,
+            "tp": 240,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            240,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "authored-context-on-frozen-source": {
+        "rows": 72,
+        "accuracy": 0.75,
+        "macro_f1": 0.7394880644233535,
+        "macro_f1_present_labels": 0.7394880644233535,
+        "per_label": {
+          "AR": {
+            "support": 24,
+            "tp": 18,
+            "fp": 3,
+            "fn": 6,
+            "precision": 0.8571428571428571,
+            "recall": 0.75,
+            "f1": 0.8
+          },
+          "DIFFUSION": {
+            "support": 24,
+            "tp": 12,
+            "fp": 2,
+            "fn": 12,
+            "precision": 0.8571428571428571,
+            "recall": 0.5,
+            "f1": 0.631578947368421
+          },
+          "BOTH": {
+            "support": 24,
+            "tp": 24,
+            "fp": 13,
+            "fn": 0,
+            "precision": 0.6486486486486487,
+            "recall": 1.0,
+            "f1": 0.7868852459016393
+          }
+        },
+        "confusion": [
+          [
+            18,
+            2,
+            4
+          ],
+          [
+            3,
+            12,
+            9
+          ],
+          [
+            0,
+            0,
+            24
+          ]
+        ]
+      },
+      "authored-output-contract-with-gallery-caption": {
+        "rows": 1080,
+        "accuracy": 0.9296296296296296,
+        "macro_f1": 0.9290630636363364,
+        "macro_f1_present_labels": 0.9290630636363364,
+        "per_label": {
+          "AR": {
+            "support": 360,
+            "tp": 358,
+            "fp": 70,
+            "fn": 2,
+            "precision": 0.8364485981308412,
+            "recall": 0.9944444444444445,
+            "f1": 0.9086294416243654
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 290,
+            "fp": 6,
+            "fn": 70,
+            "precision": 0.9797297297297297,
+            "recall": 0.8055555555555556,
+            "f1": 0.8841463414634146
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 356,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.9888888888888889,
+            "f1": 0.994413407821229
+          }
+        },
+        "confusion": [
+          [
+            358,
+            2,
+            0
+          ],
+          [
+            70,
+            290,
+            0
+          ],
+          [
+            0,
+            4,
+            356
+          ]
+        ]
+      },
+      "aya-original-human": {
+        "rows": 19,
+        "accuracy": 0.8421052631578947,
+        "macro_f1": 0.30476190476190473,
+        "macro_f1_present_labels": 0.9142857142857143,
+        "per_label": {
+          "AR": {
+            "support": 19,
+            "tp": 16,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.8421052631578947,
+            "f1": 0.9142857142857143
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 1,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 2,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            16,
+            1,
+            2
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      }
+    },
+    "language": {
+      "de": {
+        "rows": 220,
+        "accuracy": 0.9454545454545454,
+        "macro_f1": 0.944095038434661,
+        "macro_f1_present_labels": 0.944095038434661,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 100,
+            "fp": 12,
+            "fn": 0,
+            "precision": 0.8928571428571429,
+            "recall": 1.0,
+            "f1": 0.9433962264150944
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 48,
+            "fp": 0,
+            "fn": 12,
+            "precision": 1.0,
+            "recall": 0.8,
+            "f1": 0.8888888888888888
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 60,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            100,
+            0,
+            0
+          ],
+          [
+            12,
+            48,
+            0
+          ],
+          [
+            0,
+            0,
+            60
+          ]
+        ]
+      },
+      "deu": {
+        "rows": 1,
+        "accuracy": 0.0,
+        "macro_f1": 0.0,
+        "macro_f1_present_labels": 0.0,
+        "per_label": {
+          "AR": {
+            "support": 1,
+            "tp": 0,
+            "fp": 0,
+            "fn": 1,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 1,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            0,
+            0,
+            1
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "en": {
+        "rows": 256,
+        "accuracy": 0.96875,
+        "macro_f1": 0.9695893076174767,
+        "macro_f1_present_labels": 0.9695893076174767,
+        "per_label": {
+          "AR": {
+            "support": 112,
+            "tp": 107,
+            "fp": 3,
+            "fn": 5,
+            "precision": 0.9727272727272728,
+            "recall": 0.9553571428571429,
+            "f1": 0.963963963963964
+          },
+          "DIFFUSION": {
+            "support": 72,
+            "tp": 69,
+            "fp": 1,
+            "fn": 3,
+            "precision": 0.9857142857142858,
+            "recall": 0.9583333333333334,
+            "f1": 0.971830985915493
+          },
+          "BOTH": {
+            "support": 72,
+            "tp": 72,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.9473684210526315,
+            "recall": 1.0,
+            "f1": 0.972972972972973
+          }
+        },
+        "confusion": [
+          [
+            107,
+            1,
+            4
+          ],
+          [
+            3,
+            69,
+            0
+          ],
+          [
+            0,
+            0,
+            72
+          ]
+        ]
+      },
+      "eng": {
+        "rows": 1,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 1,
+            "tp": 1,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            1,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "es": {
+        "rows": 220,
+        "accuracy": 0.9772727272727273,
+        "macro_f1": 0.9743785567367925,
+        "macro_f1_present_labels": 0.9743785567367925,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 100,
+            "fp": 2,
+            "fn": 0,
+            "precision": 0.9803921568627451,
+            "recall": 1.0,
+            "f1": 0.9900990099009901
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 58,
+            "fp": 3,
+            "fn": 2,
+            "precision": 0.9508196721311475,
+            "recall": 0.9666666666666667,
+            "f1": 0.9586776859504132
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 57,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.95,
+            "f1": 0.9743589743589743
+          }
+        },
+        "confusion": [
+          [
+            100,
+            0,
+            0
+          ],
+          [
+            2,
+            58,
+            0
+          ],
+          [
+            0,
+            3,
+            57
+          ]
+        ]
+      },
+      "fr": {
+        "rows": 220,
+        "accuracy": 0.9272727272727272,
+        "macro_f1": 0.9236286001895273,
+        "macro_f1_present_labels": 0.9236286001895273,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 100,
+            "fp": 15,
+            "fn": 0,
+            "precision": 0.8695652173913043,
+            "recall": 1.0,
+            "f1": 0.9302325581395349
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 45,
+            "fp": 1,
+            "fn": 15,
+            "precision": 0.9782608695652174,
+            "recall": 0.75,
+            "f1": 0.8490566037735849
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 59,
+            "fp": 0,
+            "fn": 1,
+            "precision": 1.0,
+            "recall": 0.9833333333333333,
+            "f1": 0.9915966386554622
+          }
+        },
+        "confusion": [
+          [
+            100,
+            0,
+            0
+          ],
+          [
+            15,
+            45,
+            0
+          ],
+          [
+            0,
+            1,
+            59
+          ]
+        ]
+      },
+      "fra": {
+        "rows": 15,
+        "accuracy": 0.8666666666666667,
+        "macro_f1": 0.30952380952380953,
+        "macro_f1_present_labels": 0.9285714285714286,
+        "per_label": {
+          "AR": {
+            "support": 15,
+            "tp": 13,
+            "fp": 0,
+            "fn": 2,
+            "precision": 1.0,
+            "recall": 0.8666666666666667,
+            "f1": 0.9285714285714286
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 1,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 1,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            13,
+            1,
+            1
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "ja": {
+        "rows": 220,
+        "accuracy": 0.8636363636363636,
+        "macro_f1": 0.8454106280193235,
+        "macro_f1_present_labels": 0.8454106280193235,
+        "per_label": {
+          "AR": {
+            "support": 100,
+            "tp": 100,
+            "fp": 30,
+            "fn": 0,
+            "precision": 0.7692307692307693,
+            "recall": 1.0,
+            "f1": 0.8695652173913043
+          },
+          "DIFFUSION": {
+            "support": 60,
+            "tp": 30,
+            "fp": 0,
+            "fn": 30,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "BOTH": {
+            "support": 60,
+            "tp": 60,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            100,
+            0,
+            0
+          ],
+          [
+            30,
+            30,
+            0
+          ],
+          [
+            0,
+            0,
+            60
+          ]
+        ]
+      },
+      "spa": {
+        "rows": 2,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 2,
+            "tp": 2,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            2,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "zh": {
+        "rows": 256,
+        "accuracy": 0.91015625,
+        "macro_f1": 0.8999097602657681,
+        "macro_f1_present_labels": 0.8999097602657681,
+        "per_label": {
+          "AR": {
+            "support": 112,
+            "tp": 109,
+            "fp": 11,
+            "fn": 3,
+            "precision": 0.9083333333333333,
+            "recall": 0.9732142857142857,
+            "f1": 0.9396551724137931
+          },
+          "DIFFUSION": {
+            "support": 72,
+            "tp": 52,
+            "fp": 3,
+            "fn": 20,
+            "precision": 0.9454545454545454,
+            "recall": 0.7222222222222222,
+            "f1": 0.8188976377952756
+          },
+          "BOTH": {
+            "support": 72,
+            "tp": 72,
+            "fp": 9,
+            "fn": 0,
+            "precision": 0.8888888888888888,
+            "recall": 1.0,
+            "f1": 0.9411764705882353
+          }
+        },
+        "confusion": [
+          [
+            109,
+            3,
+            0
+          ],
+          [
+            11,
+            52,
+            9
+          ],
+          [
+            0,
+            0,
+            72
+          ]
+        ]
+      }
+    },
+    "length_bucket": {
+      "16384": {
+        "rows": 18,
+        "accuracy": 0.7222222222222222,
+        "macro_f1": 0.7166666666666667,
+        "macro_f1_present_labels": 0.7166666666666667,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 4,
+            "fp": 0,
+            "fn": 2,
+            "precision": 1.0,
+            "recall": 0.6666666666666666,
+            "f1": 0.8
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 1,
+            "fn": 3,
+            "precision": 0.75,
+            "recall": 0.5,
+            "f1": 0.6
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6,
+            "recall": 1.0,
+            "f1": 0.75
+          }
+        },
+        "confusion": [
+          [
+            4,
+            1,
+            1
+          ],
+          [
+            0,
+            3,
+            3
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "32768": {
+        "rows": 18,
+        "accuracy": 0.6111111111111112,
+        "macro_f1": 0.5888888888888889,
+        "macro_f1_present_labels": 0.5888888888888889,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 2,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.3333333333333333,
+            "f1": 0.5
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 1,
+            "fn": 3,
+            "precision": 0.75,
+            "recall": 0.5,
+            "f1": 0.6
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 6,
+            "fn": 0,
+            "precision": 0.5,
+            "recall": 1.0,
+            "f1": 0.6666666666666666
+          }
+        },
+        "confusion": [
+          [
+            2,
+            1,
+            3
+          ],
+          [
+            0,
+            3,
+            3
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "4096": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            6,
+            0,
+            0
+          ],
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "8192": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          }
+        },
+        "confusion": [
+          [
+            6,
+            0,
+            0
+          ],
+          [
+            0,
+            3,
+            3
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "short": {
+        "rows": 1339,
+        "accuracy": 0.9410007468259896,
+        "macro_f1": 0.938961528246535,
+        "macro_f1_present_labels": 0.938961528246535,
+        "per_label": {
+          "AR": {
+            "support": 619,
+            "tp": 614,
+            "fp": 70,
+            "fn": 5,
+            "precision": 0.8976608187134503,
+            "recall": 0.9919224555735057,
+            "f1": 0.9424405218726016
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 290,
+            "fp": 7,
+            "fn": 70,
+            "precision": 0.9764309764309764,
+            "recall": 0.8055555555555556,
+            "f1": 0.882800608828006
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 356,
+            "fp": 2,
+            "fn": 4,
+            "precision": 0.994413407821229,
+            "recall": 0.9888888888888889,
+            "f1": 0.9916434540389972
+          }
+        },
+        "confusion": [
+          [
+            614,
+            3,
+            2
+          ],
+          [
+            70,
+            290,
+            0
+          ],
+          [
+            0,
+            4,
+            356
+          ]
+        ]
+      }
+    },
+    "position": {
+      "head": {
+        "rows": 24,
+        "accuracy": 0.7916666666666666,
+        "macro_f1": 0.7805555555555556,
+        "macro_f1_present_labels": 0.7805555555555556,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 7,
+            "fp": 1,
+            "fn": 1,
+            "precision": 0.875,
+            "recall": 0.875,
+            "f1": 0.875
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          }
+        },
+        "confusion": [
+          [
+            7,
+            0,
+            1
+          ],
+          [
+            1,
+            4,
+            3
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "middle": {
+        "rows": 24,
+        "accuracy": 0.75,
+        "macro_f1": 0.7384615384615385,
+        "macro_f1_present_labels": 0.7384615384615385,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 6,
+            "fp": 1,
+            "fn": 2,
+            "precision": 0.8571428571428571,
+            "recall": 0.75,
+            "f1": 0.8
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 4,
+            "fp": 1,
+            "fn": 4,
+            "precision": 0.8,
+            "recall": 0.5,
+            "f1": 0.6153846153846154
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          }
+        },
+        "confusion": [
+          [
+            6,
+            1,
+            1
+          ],
+          [
+            1,
+            4,
+            3
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "tail": {
+        "rows": 24,
+        "accuracy": 0.7083333333333334,
+        "macro_f1": 0.6971916971916973,
+        "macro_f1_present_labels": 0.6971916971916973,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 5,
+            "fp": 1,
+            "fn": 3,
+            "precision": 0.8333333333333334,
+            "recall": 0.625,
+            "f1": 0.7142857142857143
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 4,
+            "fp": 1,
+            "fn": 4,
+            "precision": 0.8,
+            "recall": 0.5,
+            "f1": 0.6153846153846154
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 5,
+            "fn": 0,
+            "precision": 0.6153846153846154,
+            "recall": 1.0,
+            "f1": 0.7619047619047619
+          }
+        },
+        "confusion": [
+          [
+            5,
+            1,
+            2
+          ],
+          [
+            1,
+            4,
+            3
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "unspecified": {
+        "rows": 1339,
+        "accuracy": 0.9410007468259896,
+        "macro_f1": 0.938961528246535,
+        "macro_f1_present_labels": 0.938961528246535,
+        "per_label": {
+          "AR": {
+            "support": 619,
+            "tp": 614,
+            "fp": 70,
+            "fn": 5,
+            "precision": 0.8976608187134503,
+            "recall": 0.9919224555735057,
+            "f1": 0.9424405218726016
+          },
+          "DIFFUSION": {
+            "support": 360,
+            "tp": 290,
+            "fp": 7,
+            "fn": 70,
+            "precision": 0.9764309764309764,
+            "recall": 0.8055555555555556,
+            "f1": 0.882800608828006
+          },
+          "BOTH": {
+            "support": 360,
+            "tp": 356,
+            "fp": 2,
+            "fn": 4,
+            "precision": 0.994413407821229,
+            "recall": 0.9888888888888889,
+            "f1": 0.9916434540389972
+          }
+        },
+        "confusion": [
+          [
+            614,
+            3,
+            2
+          ],
+          [
+            70,
+            290,
+            0
+          ],
+          [
+            0,
+            4,
+            356
+          ]
+        ]
+      }
+    }
+  },
+  "actual_tokens": {
+    "min": 10,
+    "max": 32768
+  },
+  "forward_batches_ms": [
+    6867.30867694132,
+    9.979294030927122,
+    10.217698058113456,
+    9.577172924764454,
+    8.845490054227412,
+    9.165168972685933,
+    9.64318704791367,
+    9.60456405300647,
+    8.76278302166611,
+    8.968289010226727,
+    9.47466993238777,
+    9.677233058027923,
+    9.375215042382479,
+    9.823185973800719,
+    9.662057971581817,
+    9.179563028737903,
+    11.730095953680575,
+    9.293120936490595,
+    8.677210076712072,
+    9.391285013407469,
+    9.813950979150832,
+    9.575207019224763,
+    9.551984025165439,
+    8.885096991434693,
+    9.53869300428778,
+    8.742582984268665,
+    8.80620803218335,
+    8.974284981377423,
+    8.370739058591425,
+    8.279451983980834,
+    8.752770954743028,
+    8.849306963384151,
+    8.838624926283956,
+    8.66746692918241,
+    8.844563039019704,
+    8.776937960647047,
+    8.734277915209532,
+    8.659832063131034,
+    8.778530056588352,
+    8.63358296919614,
+    8.674151031300426,
+    8.512609987519681,
+    9.786258917301893,
+    8.91635799780488,
+    8.802980999462306,
+    8.703672094270587,
+    8.956133970059454,
+    9.44127095863223,
+    9.805834968574345,
+    8.607528056018054,
+    8.948493050411344,
+    8.621277054771781,
+    8.823783951811492,
+    8.64119699690491,
+    8.930942974984646,
+    8.808633079752326,
+    8.573618018999696,
+    8.275469997897744,
+    8.803748991340399,
+    8.867671946063638,
+    8.73371004126966,
+    9.083968005143106,
+    9.118168032728136,
+    8.670299896039069,
+    10.530916973948479,
+    9.835724020376801,
+    9.535206016153097,
+    10.862831026315689,
+    9.53808892518282,
+    9.13307594601065,
+    8.736607967875898,
+    8.652991964481771,
+    8.785621030256152,
+    8.867122000083327,
+    8.924660040065646,
+    8.9677240466699,
+    8.791130967438221,
+    9.663502918556333,
+    10.761761921457946,
+    10.900439927354455,
+    13.626113068312407,
+    13.141665025614202,
+    26.179153006523848,
+    10.871967067942023,
+    10.509088053368032,
+    10.136986966244876,
+    10.065859998576343,
+    10.550233069807291,
+    9.864723077043891,
+    10.230808984488249,
+    10.376129997894168,
+    10.429278947412968,
+    10.588767006993294,
+    10.126109002158046,
+    95.11675895191729,
+    94.12748797331005,
+    93.1606040103361,
+    94.69386609271169,
+    95.08034691680223,
+    95.58988595381379,
+    90.89502296410501,
+    92.33838389627635,
+    94.54926906619221,
+    90.11891495902091,
+    91.88718197401613,
+    91.07365796808153,
+    92.71830692887306,
+    90.55000694934279,
+    90.10055009275675,
+    91.15621098317206,
+    89.35096103232354,
+    92.38172706682235,
+    297.9037309996784,
+    295.3836730448529,
+    301.1517010163516,
+    310.11260591913015,
+    285.07210698444396,
+    281.12302406225353,
+    293.4185180347413,
+    301.85998196247965,
+    317.9404450347647,
+    308.69928805623204,
+    284.06618908047676,
+    290.1087219361216,
+    289.306262973696,
+    306.11998005770147,
+    309.847544063814,
+    316.40311097726226,
+    296.2943450547755,
+    291.6298599448055,
+    1101.8928970443085,
+    1187.7785919932649,
+    1115.6065230024979,
+    1103.6816139239818,
+    1052.2410059347749,
+    1127.9304079944268,
+    1140.2252310654148,
+    1127.1864819573238,
+    1144.78641806636,
+    1179.7228540526703,
+    1068.4462479548529,
+    1104.9326949287206,
+    1157.1569900261238,
+    1121.0174020379782,
+    1086.3883190322667,
+    1128.0008970061317,
+    1118.4036789927632,
+    1113.7385519687086,
+    4351.286676013842,
+    4134.929118910804,
+    4212.978955009021,
+    4117.138245957904,
+    4257.8679820289835,
+    4264.182258048095,
+    4146.985196042806,
+    4124.883470009081,
+    4204.956557019614,
+    4176.235282910056,
+    4096.2329669855535,
+    4119.872158044018,
+    4040.9102799603716,
+    4103.718529921025,
+    4144.207939971238,
+    4094.621406053193,
+    4071.070461999625,
+    4106.690975022502
+  ],
+  "precision": {
+    "parameters": "float32",
+    "forward_autocast": null,
+    "device_type": "cuda"
+  },
+  "data_files": [
+    {
+      "file": "test.jsonl",
+      "sha256": "20700706f7fc8876bf1b1235689d16c5ce4512fba35c70bc043abeb436b0cc53"
+    },
+    {
+      "file": "test.jsonl",
+      "sha256": "fa8ec4fc2a6398ce0ef078bb9f4998e0bf120922afc4a77ff1fed4b2c17e1d4c"
+    }
+  ]
+}

evaluation/final-evaluation-plan.json

@@ -0,0 +1,13 @@
+{
+  "candidate_lock_sha256": "34c6015eb91e0fd500371f9bad66cf28600533acb0c6197022eff3b3eeaa3b5f",
+  "candidate_frozen_before_evaluation": true,
+  "test_selection": false,
+  "precision": "FP32 parameters and FP32 forward",
+  "baseline_revision": "9f05f15889d32987d29bbe2e71bff945902ff553",
+  "baseline_pooling": "mean",
+  "candidate_pooling": "mean",
+  "timing_is_service_latency": false,
+  "natural_labels": "Aya requests individually assistant-reviewed; academic text-answer and original output-contract templates are not naturally observed three-way routing requests",
+  "historical_pretraining_or_old_task_exposure": "Unknown for original weights; split isolation established for this new repair training only",
+  "long_context_source": "Authored stress with frozen held-out payload/source groups and backgrounds; repeated prose is not a natural long-document benchmark"
+}

evaluation/freeze-decision.json

@@ -0,0 +1,33 @@
+{
+  "decision": "accept Vela Base step200 after all predeclared short and long development retention gates pass",
+  "base_model": "llm-semantic-router/Vela-1.0-Encoder-307M",
+  "base_revision": "5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114",
+  "selected_step": 200,
+  "classifier_pooling": "mean",
+  "vela_adaptation_steps": 200,
+  "vela_selected_step": 200,
+  "vela_retention_checks": {
+    "short": true,
+    "long": true,
+    "4096": true,
+    "8192": true,
+    "16384": true,
+    "32768": true,
+    "natural_aya": true,
+    "academic-text-answer": true,
+    "authored-output-contract-with-gallery-caption": true
+  },
+  "retention_gate_not_relaxed": true,
+  "test_used": false,
+  "test_predictions_before_freeze": false,
+  "adapter_sha256": "52fec9bffae938380605bd1da242fcfded6c75964b89ce112de5206f14f4eb00",
+  "final_data_sha256": {
+    "corpora/modality-v2/test.jsonl": "20700706f7fc8876bf1b1235689d16c5ce4512fba35c70bc043abeb436b0cc53",
+    "corpora/modality-v2-context-v2/test.jsonl": "fa8ec4fc2a6398ce0ef078bb9f4998e0bf120922afc4a77ff1fed4b2c17e1d4c"
+  },
+  "final_rows": {
+    "corpora/modality-v2/test.jsonl": 1339,
+    "corpora/modality-v2-context-v2/test.jsonl": 72
+  },
+  "original_baseline_pooling": "preserve its published mean configuration; no pooling override"
+}

evaluation/migration-adaptation-plan.json

@@ -0,0 +1,11 @@
+{
+  "purpose": "bounded Vela Base adaptation before final freeze",
+  "steps": 200,
+  "learning_rate": 2e-06,
+  "same_frozen_partitions": true,
+  "test_used": false,
+  "selection": "source-macro-f1",
+  "migration_gate": "unchanged base-migration/modality-v5/plan.json",
+  "task_head_preserved": true,
+  "sampling_semantics": "same global4 samples, explicit token-budget microbatch regrouping; dropout trajectory can differ"
+}

evaluation/migration-decision.json

@@ -0,0 +1,15 @@
+{
+  "checks": {
+    "short": false,
+    "long": true,
+    "4096": true,
+    "8192": true,
+    "16384": true,
+    "32768": true,
+    "natural_aya": true,
+    "academic-text-answer": true,
+    "authored-output-contract-with-gallery-caption": false
+  },
+  "accepted": false,
+  "test_used": false
+}

evaluation/migration-dev-old.json

@@ -0,0 +1,1410 @@
+{
+  "rows": 1001,
+  "accuracy": 0.8551448551448552,
+  "macro_f1": 0.8591739356008521,
+  "macro_f1_present_labels": 0.8591739356008521,
+  "per_label": {
+    "AR": {
+      "support": 473,
+      "tp": 328,
+      "fp": 0,
+      "fn": 145,
+      "precision": 1.0,
+      "recall": 0.693446088794926,
+      "f1": 0.818976279650437
+    },
+    "DIFFUSION": {
+      "support": 264,
+      "tp": 264,
+      "fp": 74,
+      "fn": 0,
+      "precision": 0.7810650887573964,
+      "recall": 1.0,
+      "f1": 0.8770764119601329
+    },
+    "BOTH": {
+      "support": 264,
+      "tp": 264,
+      "fp": 71,
+      "fn": 0,
+      "precision": 0.7880597014925373,
+      "recall": 1.0,
+      "f1": 0.8814691151919867
+    }
+  },
+  "confusion": [
+    [
+      328,
+      74,
+      71
+    ],
+    [
+      0,
+      264,
+      0
+    ],
+    [
+      0,
+      0,
+      264
+    ]
+  ],
+  "breakdowns": {
+    "source": {
+      "academic-text-answer": {
+        "rows": 180,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 180,
+            "tp": 180,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            180,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "authored-context-on-frozen-source": {
+        "rows": 72,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 24,
+            "tp": 12,
+            "fp": 0,
+            "fn": 12,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 24,
+            "tp": 24,
+            "fp": 12,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 24,
+            "tp": 24,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            12,
+            12,
+            0
+          ],
+          [
+            0,
+            24,
+            0
+          ],
+          [
+            0,
+            0,
+            24
+          ]
+        ]
+      },
+      "authored-output-contract-with-gallery-caption": {
+        "rows": 720,
+        "accuracy": 0.8152777777777778,
+        "macro_f1": 0.791155643058494,
+        "macro_f1_present_labels": 0.791155643058494,
+        "per_label": {
+          "AR": {
+            "support": 240,
+            "tp": 107,
+            "fp": 0,
+            "fn": 133,
+            "precision": 1.0,
+            "recall": 0.44583333333333336,
+            "f1": 0.6167146974063401
+          },
+          "DIFFUSION": {
+            "support": 240,
+            "tp": 240,
+            "fp": 62,
+            "fn": 0,
+            "precision": 0.7947019867549668,
+            "recall": 1.0,
+            "f1": 0.8856088560885609
+          },
+          "BOTH": {
+            "support": 240,
+            "tp": 240,
+            "fp": 71,
+            "fn": 0,
+            "precision": 0.7717041800643086,
+            "recall": 1.0,
+            "f1": 0.8711433756805808
+          }
+        },
+        "confusion": [
+          [
+            107,
+            62,
+            71
+          ],
+          [
+            0,
+            240,
+            0
+          ],
+          [
+            0,
+            0,
+            240
+          ]
+        ]
+      },
+      "aya-original-human": {
+        "rows": 29,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 29,
+            "tp": 29,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            29,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      }
+    },
+    "language": {
+      "de": {
+        "rows": 150,
+        "accuracy": 1.0,
+        "macro_f1": 1.0,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 70,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            70,
+            0,
+            0
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "deu": {
+        "rows": 2,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 2,
+            "tp": 2,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            2,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "en": {
+        "rows": 186,
+        "accuracy": 0.7849462365591398,
+        "macro_f1": 0.7998805256869773,
+        "macro_f1_present_labels": 0.7998805256869773,
+        "per_label": {
+          "AR": {
+            "support": 82,
+            "tp": 42,
+            "fp": 0,
+            "fn": 40,
+            "precision": 1.0,
+            "recall": 0.5121951219512195,
+            "f1": 0.6774193548387096
+          },
+          "DIFFUSION": {
+            "support": 52,
+            "tp": 52,
+            "fp": 40,
+            "fn": 0,
+            "precision": 0.5652173913043478,
+            "recall": 1.0,
+            "f1": 0.7222222222222222
+          },
+          "BOTH": {
+            "support": 52,
+            "tp": 52,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            42,
+            40,
+            0
+          ],
+          [
+            0,
+            52,
+            0
+          ],
+          [
+            0,
+            0,
+            52
+          ]
+        ]
+      },
+      "eng": {
+        "rows": 5,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 5,
+            "tp": 5,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            5,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "es": {
+        "rows": 150,
+        "accuracy": 0.7333333333333333,
+        "macro_f1": 0.7411255411255411,
+        "macro_f1_present_labels": 0.7411255411255411,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 30,
+            "fp": 0,
+            "fn": 40,
+            "precision": 1.0,
+            "recall": 0.42857142857142855,
+            "f1": 0.6
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 8,
+            "fn": 0,
+            "precision": 0.8333333333333334,
+            "recall": 1.0,
+            "f1": 0.9090909090909091
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 32,
+            "fn": 0,
+            "precision": 0.5555555555555556,
+            "recall": 1.0,
+            "f1": 0.7142857142857143
+          }
+        },
+        "confusion": [
+          [
+            30,
+            8,
+            32
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "fr": {
+        "rows": 150,
+        "accuracy": 0.8,
+        "macro_f1": 0.8181818181818182,
+        "macro_f1_present_labels": 0.8181818181818182,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 40,
+            "fp": 0,
+            "fn": 30,
+            "precision": 1.0,
+            "recall": 0.5714285714285714,
+            "f1": 0.7272727272727273
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 30,
+            "fn": 0,
+            "precision": 0.5714285714285714,
+            "recall": 1.0,
+            "f1": 0.7272727272727273
+          }
+        },
+        "confusion": [
+          [
+            40,
+            0,
+            30
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "ja": {
+        "rows": 150,
+        "accuracy": 0.94,
+        "macro_f1": 0.943391371472682,
+        "macro_f1_present_labels": 0.943391371472682,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 61,
+            "fp": 0,
+            "fn": 9,
+            "precision": 1.0,
+            "recall": 0.8714285714285714,
+            "f1": 0.9312977099236641
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 9,
+            "fn": 0,
+            "precision": 0.8163265306122449,
+            "recall": 1.0,
+            "f1": 0.898876404494382
+          }
+        },
+        "confusion": [
+          [
+            61,
+            0,
+            9
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "spa": {
+        "rows": 19,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 19,
+            "tp": 19,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            19,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "zh": {
+        "rows": 186,
+        "accuracy": 0.8602150537634409,
+        "macro_f1": 0.8705314009661835,
+        "macro_f1_present_labels": 0.8705314009661835,
+        "per_label": {
+          "AR": {
+            "support": 82,
+            "tp": 56,
+            "fp": 0,
+            "fn": 26,
+            "precision": 1.0,
+            "recall": 0.6829268292682927,
+            "f1": 0.8115942028985508
+          },
+          "DIFFUSION": {
+            "support": 52,
+            "tp": 52,
+            "fp": 26,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 52,
+            "tp": 52,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            56,
+            26,
+            0
+          ],
+          [
+            0,
+            52,
+            0
+          ],
+          [
+            0,
+            0,
+            52
+          ]
+        ]
+      },
+      "zho": {
+        "rows": 3,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 3,
+            "tp": 3,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      }
+    },
+    "length_bucket": {
+      "16384": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "32768": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "4096": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "8192": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "short": {
+        "rows": 929,
+        "accuracy": 0.8568353067814855,
+        "macro_f1": 0.8609653408729382,
+        "macro_f1_present_labels": 0.8609653408729382,
+        "per_label": {
+          "AR": {
+            "support": 449,
+            "tp": 316,
+            "fp": 0,
+            "fn": 133,
+            "precision": 1.0,
+            "recall": 0.7037861915367484,
+            "f1": 0.8261437908496732
+          },
+          "DIFFUSION": {
+            "support": 240,
+            "tp": 240,
+            "fp": 62,
+            "fn": 0,
+            "precision": 0.7947019867549668,
+            "recall": 1.0,
+            "f1": 0.8856088560885609
+          },
+          "BOTH": {
+            "support": 240,
+            "tp": 240,
+            "fp": 71,
+            "fn": 0,
+            "precision": 0.7717041800643086,
+            "recall": 1.0,
+            "f1": 0.8711433756805808
+          }
+        },
+        "confusion": [
+          [
+            316,
+            62,
+            71
+          ],
+          [
+            0,
+            240,
+            0
+          ],
+          [
+            0,
+            0,
+            240
+          ]
+        ]
+      }
+    },
+    "position": {
+      "head": {
+        "rows": 24,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            0,
+            8,
+            0
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "middle": {
+        "rows": 24,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            0,
+            8,
+            0
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "tail": {
+        "rows": 24,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            0,
+            8,
+            0
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "unspecified": {
+        "rows": 929,
+        "accuracy": 0.8568353067814855,
+        "macro_f1": 0.8609653408729382,
+        "macro_f1_present_labels": 0.8609653408729382,
+        "per_label": {
+          "AR": {
+            "support": 449,
+            "tp": 316,
+            "fp": 0,
+            "fn": 133,
+            "precision": 1.0,
+            "recall": 0.7037861915367484,
+            "f1": 0.8261437908496732
+          },
+          "DIFFUSION": {
+            "support": 240,
+            "tp": 240,
+            "fp": 62,
+            "fn": 0,
+            "precision": 0.7947019867549668,
+            "recall": 1.0,
+            "f1": 0.8856088560885609
+          },
+          "BOTH": {
+            "support": 240,
+            "tp": 240,
+            "fp": 71,
+            "fn": 0,
+            "precision": 0.7717041800643086,
+            "recall": 1.0,
+            "f1": 0.8711433756805808
+          }
+        },
+        "confusion": [
+          [
+            316,
+            62,
+            71
+          ],
+          [
+            0,
+            240,
+            0
+          ],
+          [
+            0,
+            0,
+            240
+          ]
+        ]
+      }
+    }
+  },
+  "actual_tokens": {
+    "min": 9,
+    "max": 32768
+  },
+  "forward_batches_ms": [
+    7039.483752916567,
+    21.584214991889894,
+    20.40059503633529,
+    19.5585381006822,
+    19.531563972122967,
+    19.734615925699472,
+    19.469924038276076,
+    19.26641701720655,
+    18.840476986952126,
+    19.38274095300585,
+    19.678111071698368,
+    19.068480003625154,
+    25.945398956537247,
+    19.180268049240112,
+    19.252500031143427,
+    19.425202044658363,
+    19.266484072431922,
+    19.290908006951213,
+    19.509149016812444,
+    19.28546593990177,
+    20.57167107705027,
+    19.616086035966873,
+    20.440079038962722,
+    29.335040017031133,
+    24.305583094246686,
+    58.74593602493405,
+    19.150480045937,
+    19.049357972107828,
+    19.0588979749009,
+    19.150931970216334,
+    19.140038988552988,
+    19.038981990888715,
+    19.242074922658503,
+    19.93995893280953,
+    19.138233037665486,
+    19.876311998814344,
+    18.998789018951356,
+    19.22773791011423,
+    19.245053990744054,
+    19.192507956176996,
+    19.094596966169775,
+    19.125562976114452,
+    19.157727947458625,
+    19.24101705662906,
+    18.754982040263712,
+    20.657840999774635,
+    19.407549989409745,
+    19.27121204789728,
+    19.349119975231588,
+    20.35636897198856,
+    19.340497092343867,
+    19.41229496151209,
+    19.209291087463498,
+    19.234362989664078,
+    19.478514092043042,
+    19.862476037815213,
+    19.685033010318875,
+    25.732986046932638,
+    21.024837042205036,
+    62.87668400909752,
+    62.32619902584702,
+    61.78742798510939,
+    62.28843401186168,
+    62.4942280119285,
+    61.87841505743563,
+    61.24127807561308,
+    62.97354004345834,
+    63.574064057320356,
+    63.25711507815868,
+    61.819158028811216,
+    60.69516693241894,
+    61.04927998967469,
+    66.55711494386196,
+    62.53866502083838,
+    64.0173009596765,
+    67.36680003814399,
+    64.94186201598495,
+    225.73108307551593,
+    221.4175140252337,
+    216.55263099819422,
+    214.4655380398035,
+    211.94534294772893,
+    209.52967007178813,
+    207.52157899551094,
+    226.30385495722294,
+    204.03683395124972,
+    212.17665902804583,
+    218.6698050936684,
+    214.6064389962703,
+    208.55662401299924,
+    205.77978307846934,
+    214.26206594333053,
+    208.822840009816,
+    210.335728013888,
+    212.93231600429863,
+    848.0608450481668,
+    825.0530819641426,
+    812.0404820656404,
+    817.2862839419395,
+    802.6156988926232,
+    838.2717769127339,
+    831.2934199348092,
+    788.4137380169705,
+    853.9408210199326,
+    869.962121010758,
+    827.322592958808,
+    890.770270023495,
+    888.5154149029404,
+    808.456527069211,
+    845.0542960781604,
+    794.9883950641379,
+    803.5358130000532,
+    815.9813659731299,
+    3343.9996540546417,
+    3326.254892977886,
+    3468.5617400100455,
+    3375.5812710151076,
+    3486.4518040558323,
+    3298.9523640135303,
+    3513.150369981304,
+    3348.251656978391,
+    3501.9268669420853,
+    3346.9400759786367,
+    3471.5288559673354,
+    3441.1905839806423,
+    3405.6642989162356,
+    3278.5228959983215,
+    3227.0270970184356,
+    3282.31126407627,
+    3352.355351904407,
+    3549.3063640315086
+  ],
+  "precision": {
+    "parameters": "float32",
+    "forward_autocast": "bfloat16",
+    "device_type": "cuda"
+  },
+  "data_files": [
+    {
+      "file": "dev.jsonl",
+      "sha256": "bc021f8b5479ab1fd8cfc3cf7e351414dc7e7b429e44b1d0b098d23104aca1d2"
+    },
+    {
+      "file": "dev.jsonl",
+      "sha256": "2768c6155a31a5e50c3ef3929fa2bd5ea0a45ea2ed49f4318be3743a69b30b85"
+    }
+  ]
+}

evaluation/migration-dev-vela.json

@@ -0,0 +1,1410 @@
+{
+  "rows": 1001,
+  "accuracy": 0.8411588411588412,
+  "macro_f1": 0.8456991326446053,
+  "macro_f1_present_labels": 0.8456991326446053,
+  "per_label": {
+    "AR": {
+      "support": 473,
+      "tp": 314,
+      "fp": 0,
+      "fn": 159,
+      "precision": 1.0,
+      "recall": 0.6638477801268499,
+      "f1": 0.7979669631512071
+    },
+    "DIFFUSION": {
+      "support": 264,
+      "tp": 264,
+      "fp": 66,
+      "fn": 0,
+      "precision": 0.8,
+      "recall": 1.0,
+      "f1": 0.8888888888888888
+    },
+    "BOTH": {
+      "support": 264,
+      "tp": 264,
+      "fp": 93,
+      "fn": 0,
+      "precision": 0.7394957983193278,
+      "recall": 1.0,
+      "f1": 0.8502415458937198
+    }
+  },
+  "confusion": [
+    [
+      314,
+      66,
+      93
+    ],
+    [
+      0,
+      264,
+      0
+    ],
+    [
+      0,
+      0,
+      264
+    ]
+  ],
+  "breakdowns": {
+    "source": {
+      "academic-text-answer": {
+        "rows": 180,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 180,
+            "tp": 180,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            180,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "authored-context-on-frozen-source": {
+        "rows": 72,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 24,
+            "tp": 12,
+            "fp": 0,
+            "fn": 12,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 24,
+            "tp": 24,
+            "fp": 12,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 24,
+            "tp": 24,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            12,
+            12,
+            0
+          ],
+          [
+            0,
+            24,
+            0
+          ],
+          [
+            0,
+            0,
+            24
+          ]
+        ]
+      },
+      "authored-output-contract-with-gallery-caption": {
+        "rows": 720,
+        "accuracy": 0.7958333333333333,
+        "macro_f1": 0.7650437660438248,
+        "macro_f1_present_labels": 0.7650437660438248,
+        "per_label": {
+          "AR": {
+            "support": 240,
+            "tp": 93,
+            "fp": 0,
+            "fn": 147,
+            "precision": 1.0,
+            "recall": 0.3875,
+            "f1": 0.5585585585585585
+          },
+          "DIFFUSION": {
+            "support": 240,
+            "tp": 240,
+            "fp": 54,
+            "fn": 0,
+            "precision": 0.8163265306122449,
+            "recall": 1.0,
+            "f1": 0.898876404494382
+          },
+          "BOTH": {
+            "support": 240,
+            "tp": 240,
+            "fp": 93,
+            "fn": 0,
+            "precision": 0.7207207207207207,
+            "recall": 1.0,
+            "f1": 0.837696335078534
+          }
+        },
+        "confusion": [
+          [
+            93,
+            54,
+            93
+          ],
+          [
+            0,
+            240,
+            0
+          ],
+          [
+            0,
+            0,
+            240
+          ]
+        ]
+      },
+      "aya-original-human": {
+        "rows": 29,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 29,
+            "tp": 29,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            29,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      }
+    },
+    "language": {
+      "de": {
+        "rows": 150,
+        "accuracy": 0.9666666666666667,
+        "macro_f1": 0.9680464778503994,
+        "macro_f1_present_labels": 0.9680464778503994,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 65,
+            "fp": 0,
+            "fn": 5,
+            "precision": 1.0,
+            "recall": 0.9285714285714286,
+            "f1": 0.9629629629629629
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 5,
+            "fn": 0,
+            "precision": 0.8888888888888888,
+            "recall": 1.0,
+            "f1": 0.9411764705882353
+          }
+        },
+        "confusion": [
+          [
+            65,
+            0,
+            5
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "deu": {
+        "rows": 2,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 2,
+            "tp": 2,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            2,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "en": {
+        "rows": 186,
+        "accuracy": 0.7903225806451613,
+        "macro_f1": 0.8009399198931909,
+        "macro_f1_present_labels": 0.8009399198931909,
+        "per_label": {
+          "AR": {
+            "support": 82,
+            "tp": 43,
+            "fp": 0,
+            "fn": 39,
+            "precision": 1.0,
+            "recall": 0.524390243902439,
+            "f1": 0.688
+          },
+          "DIFFUSION": {
+            "support": 52,
+            "tp": 52,
+            "fp": 36,
+            "fn": 0,
+            "precision": 0.5909090909090909,
+            "recall": 1.0,
+            "f1": 0.7428571428571429
+          },
+          "BOTH": {
+            "support": 52,
+            "tp": 52,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.9454545454545454,
+            "recall": 1.0,
+            "f1": 0.9719626168224299
+          }
+        },
+        "confusion": [
+          [
+            43,
+            36,
+            3
+          ],
+          [
+            0,
+            52,
+            0
+          ],
+          [
+            0,
+            0,
+            52
+          ]
+        ]
+      },
+      "eng": {
+        "rows": 5,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 5,
+            "tp": 5,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            5,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "es": {
+        "rows": 150,
+        "accuracy": 0.7333333333333333,
+        "macro_f1": 0.751191952597492,
+        "macro_f1_present_labels": 0.751191952597492,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 30,
+            "fp": 0,
+            "fn": 40,
+            "precision": 1.0,
+            "recall": 0.42857142857142855,
+            "f1": 0.6
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 2,
+            "fn": 0,
+            "precision": 0.9523809523809523,
+            "recall": 1.0,
+            "f1": 0.975609756097561
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 38,
+            "fn": 0,
+            "precision": 0.5128205128205128,
+            "recall": 1.0,
+            "f1": 0.6779661016949152
+          }
+        },
+        "confusion": [
+          [
+            30,
+            2,
+            38
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "fr": {
+        "rows": 150,
+        "accuracy": 0.76,
+        "macro_f1": 0.7811671087533156,
+        "macro_f1_present_labels": 0.7811671087533156,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 34,
+            "fp": 0,
+            "fn": 36,
+            "precision": 1.0,
+            "recall": 0.4857142857142857,
+            "f1": 0.6538461538461539
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 36,
+            "fn": 0,
+            "precision": 0.5263157894736842,
+            "recall": 1.0,
+            "f1": 0.6896551724137931
+          }
+        },
+        "confusion": [
+          [
+            34,
+            0,
+            36
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "ja": {
+        "rows": 150,
+        "accuracy": 0.9333333333333333,
+        "macro_f1": 0.9373219373219372,
+        "macro_f1_present_labels": 0.9373219373219372,
+        "per_label": {
+          "AR": {
+            "support": 70,
+            "tp": 60,
+            "fp": 0,
+            "fn": 10,
+            "precision": 1.0,
+            "recall": 0.8571428571428571,
+            "f1": 0.9230769230769231
+          },
+          "DIFFUSION": {
+            "support": 40,
+            "tp": 40,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "BOTH": {
+            "support": 40,
+            "tp": 40,
+            "fp": 10,
+            "fn": 0,
+            "precision": 0.8,
+            "recall": 1.0,
+            "f1": 0.8888888888888888
+          }
+        },
+        "confusion": [
+          [
+            60,
+            0,
+            10
+          ],
+          [
+            0,
+            40,
+            0
+          ],
+          [
+            0,
+            0,
+            40
+          ]
+        ]
+      },
+      "spa": {
+        "rows": 19,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 19,
+            "tp": 19,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            19,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      },
+      "zh": {
+        "rows": 186,
+        "accuracy": 0.8440860215053764,
+        "macro_f1": 0.8545133878467212,
+        "macro_f1_present_labels": 0.8545133878467212,
+        "per_label": {
+          "AR": {
+            "support": 82,
+            "tp": 53,
+            "fp": 0,
+            "fn": 29,
+            "precision": 1.0,
+            "recall": 0.6463414634146342,
+            "f1": 0.7851851851851852
+          },
+          "DIFFUSION": {
+            "support": 52,
+            "tp": 52,
+            "fp": 28,
+            "fn": 0,
+            "precision": 0.65,
+            "recall": 1.0,
+            "f1": 0.7878787878787878
+          },
+          "BOTH": {
+            "support": 52,
+            "tp": 52,
+            "fp": 1,
+            "fn": 0,
+            "precision": 0.9811320754716981,
+            "recall": 1.0,
+            "f1": 0.9904761904761905
+          }
+        },
+        "confusion": [
+          [
+            53,
+            28,
+            1
+          ],
+          [
+            0,
+            52,
+            0
+          ],
+          [
+            0,
+            0,
+            52
+          ]
+        ]
+      },
+      "zho": {
+        "rows": 3,
+        "accuracy": 1.0,
+        "macro_f1": 0.3333333333333333,
+        "macro_f1_present_labels": 1.0,
+        "per_label": {
+          "AR": {
+            "support": 3,
+            "tp": 3,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          },
+          "DIFFUSION": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          },
+          "BOTH": {
+            "support": 0,
+            "tp": 0,
+            "fp": 0,
+            "fn": 0,
+            "precision": 0.0,
+            "recall": 0.0,
+            "f1": 0.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ],
+          [
+            0,
+            0,
+            0
+          ]
+        ]
+      }
+    },
+    "length_bucket": {
+      "16384": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "32768": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "4096": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "8192": {
+        "rows": 18,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 6,
+            "tp": 3,
+            "fp": 0,
+            "fn": 3,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 6,
+            "tp": 6,
+            "fp": 3,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 6,
+            "tp": 6,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            3,
+            3,
+            0
+          ],
+          [
+            0,
+            6,
+            0
+          ],
+          [
+            0,
+            0,
+            6
+          ]
+        ]
+      },
+      "short": {
+        "rows": 929,
+        "accuracy": 0.8417653390742734,
+        "macro_f1": 0.8469445749752597,
+        "macro_f1_present_labels": 0.8469445749752597,
+        "per_label": {
+          "AR": {
+            "support": 449,
+            "tp": 302,
+            "fp": 0,
+            "fn": 147,
+            "precision": 1.0,
+            "recall": 0.6726057906458798,
+            "f1": 0.8042609853528628
+          },
+          "DIFFUSION": {
+            "support": 240,
+            "tp": 240,
+            "fp": 54,
+            "fn": 0,
+            "precision": 0.8163265306122449,
+            "recall": 1.0,
+            "f1": 0.898876404494382
+          },
+          "BOTH": {
+            "support": 240,
+            "tp": 240,
+            "fp": 93,
+            "fn": 0,
+            "precision": 0.7207207207207207,
+            "recall": 1.0,
+            "f1": 0.837696335078534
+          }
+        },
+        "confusion": [
+          [
+            302,
+            54,
+            93
+          ],
+          [
+            0,
+            240,
+            0
+          ],
+          [
+            0,
+            0,
+            240
+          ]
+        ]
+      }
+    },
+    "position": {
+      "head": {
+        "rows": 24,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            0,
+            8,
+            0
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "middle": {
+        "rows": 24,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            0,
+            8,
+            0
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "tail": {
+        "rows": 24,
+        "accuracy": 0.8333333333333334,
+        "macro_f1": 0.8222222222222223,
+        "macro_f1_present_labels": 0.8222222222222223,
+        "per_label": {
+          "AR": {
+            "support": 8,
+            "tp": 4,
+            "fp": 0,
+            "fn": 4,
+            "precision": 1.0,
+            "recall": 0.5,
+            "f1": 0.6666666666666666
+          },
+          "DIFFUSION": {
+            "support": 8,
+            "tp": 8,
+            "fp": 4,
+            "fn": 0,
+            "precision": 0.6666666666666666,
+            "recall": 1.0,
+            "f1": 0.8
+          },
+          "BOTH": {
+            "support": 8,
+            "tp": 8,
+            "fp": 0,
+            "fn": 0,
+            "precision": 1.0,
+            "recall": 1.0,
+            "f1": 1.0
+          }
+        },
+        "confusion": [
+          [
+            4,
+            4,
+            0
+          ],
+          [
+            0,
+            8,
+            0
+          ],
+          [
+            0,
+            0,
+            8
+          ]
+        ]
+      },
+      "unspecified": {
+        "rows": 929,
+        "accuracy": 0.8417653390742734,
+        "macro_f1": 0.8469445749752597,
+        "macro_f1_present_labels": 0.8469445749752597,
+        "per_label": {
+          "AR": {
+            "support": 449,
+            "tp": 302,
+            "fp": 0,
+            "fn": 147,
+            "precision": 1.0,
+            "recall": 0.6726057906458798,
+            "f1": 0.8042609853528628
+          },
+          "DIFFUSION": {
+            "support": 240,
+            "tp": 240,
+            "fp": 54,
+            "fn": 0,
+            "precision": 0.8163265306122449,
+            "recall": 1.0,
+            "f1": 0.898876404494382
+          },
+          "BOTH": {
+            "support": 240,
+            "tp": 240,
+            "fp": 93,
+            "fn": 0,
+            "precision": 0.7207207207207207,
+            "recall": 1.0,
+            "f1": 0.837696335078534
+          }
+        },
+        "confusion": [
+          [
+            302,
+            54,
+            93
+          ],
+          [
+            0,
+            240,
+            0
+          ],
+          [
+            0,
+            0,
+            240
+          ]
+        ]
+      }
+    }
+  },
+  "actual_tokens": {
+    "min": 9,
+    "max": 32768
+  },
+  "forward_batches_ms": [
+    7138.325362931937,
+    20.877185044810176,
+    19.67246795538813,
+    19.299722043797374,
+    19.489847007207572,
+    19.313878961838782,
+    18.220369005575776,
+    18.069003941491246,
+    17.690050997771323,
+    18.120278022252023,
+    18.159667029976845,
+    18.83825403638184,
+    18.094961997121572,
+    18.554485985077918,
+    21.646736073307693,
+    20.772389019839466,
+    18.628726014867425,
+    17.715877038426697,
+    18.23048701044172,
+    18.000961048528552,
+    18.17912806291133,
+    18.317410023882985,
+    18.020904972217977,
+    17.96193898189813,
+    17.93277298565954,
+    57.140661985613406,
+    18.297206959687173,
+    18.85265507735312,
+    17.86524197086692,
+    18.02249299362302,
+    18.041340983472764,
+    18.24610901530832,
+    17.97376002650708,
+    18.182258936576545,
+    17.512104008346796,
+    18.063965952023864,
+    17.891981988213956,
+    18.07054504752159,
+    20.899881958030164,
+    20.305432961322367,
+    17.980684991925955,
+    19.004215020686388,
+    18.027939018793404,
+    18.247300991788507,
+    17.516574007458985,
+    19.29691794794053,
+    18.164891051128507,
+    18.128087976947427,
+    18.290097010321915,
+    17.779922927729785,
+    19.383021048270166,
+    18.370570964179933,
+    18.288260092958808,
+    20.001595956273377,
+    19.974722061306238,
+    19.60317906923592,
+    18.46761896740645,
+    25.879641994833946,
+    21.32161194458604,
+    67.82619096338749,
+    66.8274499475956,
+    67.0328710693866,
+    67.87156604696065,
+    66.34944898542017,
+    65.6514149159193,
+    64.8158120457083,
+    69.92864399217069,
+    67.61880894191563,
+    61.89206801354885,
+    69.69579495489597,
+    65.14124805107713,
+    72.03296897932887,
+    71.39926496893167,
+    76.04432699736208,
+    74.31276992429048,
+    70.53207501303405,
+    72.93997798115015,
+    242.4739560810849,
+    237.631244934164,
+    236.15267500281334,
+    243.34610905498266,
+    233.1851359922439,
+    247.32551199849695,
+    245.15614099800587,
+    228.48063590936363,
+    237.3392099980265,
+    222.71971602458507,
+    224.52398692257702,
+    230.29644892085344,
+    251.83466100133955,
+    246.70857994351536,
+    235.7939110370353,
+    232.3068710975349,
+    233.56033593881875,
+    229.35457597486675,
+    898.6606820253655,
+    884.4740580534562,
+    885.0953739602119,
+    917.1203889418393,
+    901.3165140058845,
+    899.4212590623647,
+    884.0509259607643,
+    909.6224809763953,
+    865.516652003862,
+    862.0706489309669,
+    865.1024099672213,
+    843.7351549509913,
+    889.8438949836418,
+    837.8107619937509,
+    879.9146349774674,
+    820.4324760008603,
+    907.9147179145366,
+    908.1653489265591,
+    3256.707448977977,
+    3420.562503975816,
+    3417.6524529466406,
+    3594.7950570844114,
+    3433.8663649978116,
+    3413.9636090258136,
+    3344.1693080822006,
+    3134.1756300535053,
+    3304.6368720242754,
+    3353.6964000668377,
+    3370.9448899608105,
+    3342.783577973023,
+    3258.953638956882,
+    3131.3253020634875,
+    3339.7319950163364,
+    3427.004609024152,
+    3479.9413479631767,
+    3547.474607010372
+  ],
+  "precision": {
+    "parameters": "float32",
+    "forward_autocast": "bfloat16",
+    "device_type": "cuda"
+  },
+  "data_files": [
+    {
+      "file": "dev.jsonl",
+      "sha256": "bc021f8b5479ab1fd8cfc3cf7e351414dc7e7b429e44b1d0b098d23104aca1d2"
+    },
+    {
+      "file": "dev.jsonl",
+      "sha256": "2768c6155a31a5e50c3ef3929fa2bd5ea0a45ea2ed49f4318be3743a69b30b85"
+    }
+  ]
+}

evaluation/migration-plan.json

@@ -0,0 +1,19 @@
+{
+  "candidate_step": 400,
+  "adapter_sha256": "5d2e619fe94be057a19931ad43122d68e07b34924ff487b4aab89346ff8f4e94",
+  "test_used": false,
+  "precision": "bfloat16 autocast with FP32 parameters",
+  "criteria": {
+    "short_macro_f1_max_drop": 0.005,
+    "long_source_macro_f1_max_drop": 0.01,
+    "each_long_length_max_correct_loss": 1,
+    "natural_aya_max_correct_loss": 1,
+    "academic_and_authored_source_macro_f1_max_drop": 0.01
+  },
+  "dev_files": {
+    "corpora/modality-v2/dev.jsonl": "bc021f8b5479ab1fd8cfc3cf7e351414dc7e7b429e44b1d0b098d23104aca1d2",
+    "corpora/modality-v2-context-v2/dev.jsonl": "2768c6155a31a5e50c3ef3929fa2bd5ea0a45ea2ed49f4318be3743a69b30b85"
+  },
+  "source_revision": "72a23a6640489471eb4ff7ad3ec5bc80af8a27de",
+  "target_revision": "5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114"
+}

label_mapping.json

@@ -0,0 +1,22 @@
+{
+  "label2id": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "id2label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  },
+  "label_to_idx": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "idx_to_label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  }
+}

lora/README.md

@@ -0,0 +1,32 @@
+# Modality LoRA variant
+
+These adapter weights reproduce the same frozen task model as the merged weights
+in the parent repository. The base is `llm-semantic-router/Vela-1.0-Encoder-307M` at
+`5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114`. Use Transformers 4.57.6, PEFT 0.18.1 and Torch 2.10.
+The adapter and task config are coupled: architecture, label IDs and pooling must
+be loaded from this directory. In particular, Domain uses CLS pooling; loading
+only the base configuration gives a different model.
+
+Download a pinned revision of this model repository, then run from its root:
+
+```bash
+python lora/load_adapter.py --adapter lora --text "Explain cellular respiration."
+python lora/verify_adapter.py --adapter lora --merged . --output adapter-verification.json
+```
+
+By default, the helper downloads the exact parent revision. `--base-directory`
+can supply an already verified local snapshot of that same parent. It is the
+caller's responsibility to verify a supplied local snapshot against the parent
+weight hash in `adapter-qualification.json` in the repository root.
+
+The verification uses FP32 CPU SDPA on three padded two-row English/Chinese
+fixtures, including Unicode text. It requires probability atol 2e-4 / rtol 1e-4
+and identical argmax labels. This checks short numerical parity, not held-out
+quality, 32K execution, performance or all inference engines. The model card
+contains separate task-quality and engine qualification evidence.
+
+`config.json`, tokenizer files and runtime mappings match the frozen merged model.
+The load helper emits logits; PII entity decoding and a serving threshold are
+separate operations. Maximum model capacity does not choose a device memory budget.
+The root model license also applies to these adapter weights; bundled helper code
+is Apache-2.0 under `reproducibility/LICENSE`.

lora/adapter_config.json

@@ -0,0 +1,58 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "llm-semantic-router/Vela-1.0-Encoder-307M",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "head",
+    "classifier",
+    "classifier",
+    "score",
+    "classifier",
+    "score",
+    "classifier",
+    "score",
+    "classifier",
+    "score",
+    "classifier",
+    "score",
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": "5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114",
+  "target_modules": [
+    "attn.Wo",
+    "mlp.Wo",
+    "mlp.Wi",
+    "attn.Wqkv"
+  ],
+  "target_parameters": null,
+  "task_type": "SEQ_CLS",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52fec9bffae938380605bd1da242fcfded6c75964b89ce112de5206f14f4eb00
+size 29427500

lora/config.json

@@ -0,0 +1,57 @@
+{
+  "architectures": [
+    "ModernBertForSequenceClassification"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 1,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 1,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "id2label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  },
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "label2id": {
+    "AR": 0,
+    "BOTH": 2,
+    "DIFFUSION": 1
+  },
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 160000,
+  "mask_token_id": 4,
+  "max_position_embeddings": 32768,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 0,
+  "position_embedding_type": "sans_pos",
+  "problem_type": "single_label_classification",
+  "repad_logits_with_grad": false,
+  "sep_token_id": 1,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "transformers_version": "4.57.6",
+  "vocab_size": 256000
+}

lora/label_mapping.json

@@ -0,0 +1,22 @@
+{
+  "label2id": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "id2label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  },
+  "label_to_idx": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "idx_to_label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  }
+}

lora/load_adapter.py

@@ -0,0 +1,71 @@
+"""Load a task LoRA using its explicit frozen task config and parent revision."""
+
+import argparse
+import json
+from pathlib import Path
+
+import torch
+from peft import PeftConfig, PeftModel
+from transformers import (
+    AutoConfig,
+    AutoModelForSequenceClassification,
+    AutoModelForTokenClassification,
+    AutoTokenizer,
+)
+
+
+def load_adapter(adapter, base_directory=None, device="cpu"):
+    adapter = Path(adapter)
+    parent = PeftConfig.from_pretrained(adapter)
+    if not parent.revision or len(parent.revision) != 40:
+        raise ValueError("Adapter must pin an immutable parent revision")
+    config = AutoConfig.from_pretrained(adapter)
+    architectures = config.architectures or []
+    if architectures == ["ModernBertForTokenClassification"]:
+        factory = AutoModelForTokenClassification
+    elif architectures == ["ModernBertForSequenceClassification"]:
+        factory = AutoModelForSequenceClassification
+        if config.classifier_pooling not in {"cls", "mean"}:
+            raise ValueError("Unknown sequence pooling contract")
+    else:
+        raise ValueError("Unknown task architecture")
+    if set(map(int, config.id2label)) != set(range(config.num_labels)):
+        raise ValueError("Non-contiguous label IDs")
+    for index, label in config.id2label.items():
+        if config.label2id[label] != int(index):
+            raise ValueError("Inconsistent task label mapping")
+    base = factory.from_pretrained(
+        base_directory or parent.base_model_name_or_path,
+        revision=None if base_directory else parent.revision,
+        config=config,
+        torch_dtype=torch.float32,
+        attn_implementation="sdpa",
+    )
+    model = (
+        PeftModel.from_pretrained(base, adapter, is_trainable=False).to(device).eval()
+    )
+    tokenizer = AutoTokenizer.from_pretrained(adapter)
+    return model, tokenizer
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--adapter", type=Path, required=True)
+    parser.add_argument(
+        "--base-directory",
+        type=Path,
+        help="Optional already verified snapshot of the pinned parent",
+    )
+    parser.add_argument("--text", required=True)
+    args = parser.parse_args()
+    model, tokenizer = load_adapter(args.adapter, args.base_directory)
+    inputs = tokenizer(args.text, return_tensors="pt", truncation=False)
+    if inputs["input_ids"].shape[1] > model.config.max_position_embeddings:
+        raise ValueError("Input exceeds the task model token budget")
+    with torch.inference_mode():
+        logits = model(**inputs).logits.float()
+    print(json.dumps({"labels": model.config.id2label, "logits": logits.tolist()}))
+
+
+if __name__ == "__main__":
+    main()

lora/modality_mapping.json

@@ -0,0 +1,12 @@
+{
+  "label_to_idx": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "idx_to_label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  }
+}

lora/special_tokens_map.json

@@ -0,0 +1,55 @@
+{
+  "additional_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

lora/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22fd4a60565f25fee8ebd2866aab07456dd46cbbeb45eaf506529ed8451ee58f
+size 34363343

lora/tokenizer_config.json

@@ -0,0 +1,2018 @@
+{
+  "add_bos_token": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<unused60>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<unused61>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<unused62>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<unused63>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁���▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<start_of_turn>",
+    "<end_of_turn>"
+  ],
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<bos>",
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 32768,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sep_token": "<eos>",
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "<unk>"
+}

lora/verify_adapter.py

@@ -0,0 +1,93 @@
+"""Reproduce the short CPU adapter/merged numerical contract; not a quality benchmark."""
+
+import argparse
+import hashlib
+import json
+from pathlib import Path
+
+import torch
+from transformers import (
+    AutoModelForSequenceClassification,
+    AutoModelForTokenClassification,
+)
+from load_adapter import load_adapter
+
+CASES = [
+    ["Contact Ada Example at ada.example@example.org.", "请解释细胞呼吸如何产生能量。"],
+    [
+        "Résumé — 王明在办公室阅读报告。 Contact test.user@example.org.",
+        "The committee discussed public history and economic policy.",
+    ],
+    ["A short request.", "A fictional setting for a story about a quiet garden. " * 9],
+]
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--adapter", type=Path, required=True)
+    parser.add_argument("--merged", type=Path, required=True)
+    parser.add_argument("--base-directory", type=Path)
+    parser.add_argument("--output", type=Path, required=True)
+    args = parser.parse_args()
+    if args.output.exists():
+        raise ValueError("Refusing to overwrite evidence")
+    torch.set_num_threads(8)
+    model, tokenizer = load_adapter(args.adapter, args.base_directory)
+    token_task = model.config.architectures == ["ModernBertForTokenClassification"]
+    factory = (
+        AutoModelForTokenClassification
+        if token_task
+        else AutoModelForSequenceClassification
+    )
+    reference = factory.from_pretrained(
+        args.merged, torch_dtype=torch.float32, attn_implementation="sdpa"
+    ).eval()
+    results = []
+    for texts in CASES:
+        inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=False)
+        with torch.inference_mode():
+            a, b = model(**inputs).logits.float(), reference(**inputs).logits.float()
+        ap, bp = a.softmax(-1), b.softmax(-1)
+        flips = int((a.argmax(-1) != b.argmax(-1)).sum())
+        passed = bool(
+            torch.isfinite(a).all()
+            and torch.isfinite(b).all()
+            and torch.allclose(ap, bp, atol=2e-4, rtol=1e-4)
+            and flips == 0
+        )
+        results.append(
+            {
+                "batch": len(texts),
+                "padded_tokens": inputs["input_ids"].shape[1],
+                "attention_mask_counts": inputs["attention_mask"].sum(-1).tolist(),
+                "input_sha256": hashlib.sha256(
+                    json.dumps(texts, ensure_ascii=False).encode()
+                ).hexdigest(),
+                "maximum_absolute_logit_difference": float((a - b).abs().max()),
+                "maximum_absolute_probability_difference": float((ap - bp).abs().max()),
+                "argmax_differences": flips,
+                "passed": passed,
+            }
+        )
+    args.output.write_text(
+        json.dumps(
+            {
+                "dtype": "float32",
+                "provider": "Torch CPU SDPA",
+                "probability_atol": 2e-4,
+                "probability_rtol": 1e-4,
+                "all_argmax_equal_required": True,
+                "quality_benchmark": False,
+                "long_context_qualification": False,
+                "results": results,
+            },
+            indent=2,
+        )
+        + "\n"
+    )
+    if not all(row["passed"] for row in results):
+        raise RuntimeError("Adapter/merged numerical contract failed")
+
+
+if __name__ == "__main__":
+    main()

modality_mapping.json

@@ -0,0 +1,12 @@
+{
+  "label_to_idx": {
+    "AR": 0,
+    "DIFFUSION": 1,
+    "BOTH": 2
+  },
+  "idx_to_label": {
+    "0": "AR",
+    "1": "DIFFUSION",
+    "2": "BOTH"
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7
+size 1230144500

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74a08c52ce47f522071fd71acee080c0a3a74ad12e3d5fb19ae1dfd244f1059c
+size 555905

onnx/model.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0372b44dd9a9ae0d93f0ee0211ec27644dd2ad94351c8022e94c953bd14c399
+size 1230130176

publication-manifest.json

@@ -0,0 +1,154 @@
+{
+  "model_id": "llm-semantic-router/Vela-1.0-Encoder-307M-Modality",
+  "base_model": "llm-semantic-router/Vela-1.0-Encoder-307M",
+  "base_revision": "5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114",
+  "weight_sha256": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+  "parameter_count": 307532547,
+  "encoder_parameter_count": 306939648,
+  "classifier_pooling": "mean",
+  "weights_unchanged_after_final_evaluation": true,
+  "quality_protocol": "FP32 complete-input argmax; authored three-way contracts, natural AR requests, academic AR retention and long stress separately reported",
+  "native_and_onnx_artifacts": "Native FP32, frozen standalone LoRA, and qualified static-B1 FP32 blocked ONNX; failed experimental graphs excluded",
+  "model_files": {
+    "config.json": "4cc261fd20f7acf8eb78c7c81491a57eaff5cbf5dbe73ed738fc87af97e729f2",
+    "label_mapping.json": "63c62f883903b470c24ab5c760f593d0dbe4a01823376ee312dbf77fa9b7686a",
+    "modality_mapping.json": "40890ec869e786e4ca6510367bd1a6cdc2feedaf8564b345b24b38bbce7e0edd",
+    "model.safetensors": "d6fdd660a7bbecbb95e91956ce8f2f1c12b2424b2a0a42ca65bab2f858de07e7",
+    "special_tokens_map.json": "6e3204c5a4004719c185007a745d1940471a6fb02521cfa0a00306dbb6bc5903",
+    "tokenizer.json": "22fd4a60565f25fee8ebd2866aab07456dd46cbbeb45eaf506529ed8451ee58f",
+    "tokenizer_config.json": "74a259bb1a3811a7e3028adcd07a65765d866d5e66e0e883f0994ccfa67e8455"
+  },
+  "support_files": {
+    "LICENSE": "f1e043aceb9aaca635570145a4246cba490f87a907715806ea020d05f45838a9",
+    "NOTICE.md": "35140a10f7fcbb826428daf9b6e767123ea58d740ac95b9d236f884cc86d668e",
+    "README.md": "5e93a2f8be652a59b23c0f3374a01558ad7fa08244668c7532d696771e07ad8b",
+    "evaluation/candidate-config.json": "4cc261fd20f7acf8eb78c7c81491a57eaff5cbf5dbe73ed738fc87af97e729f2",
+    "evaluation/candidate-label_mapping.json": "63c62f883903b470c24ab5c760f593d0dbe4a01823376ee312dbf77fa9b7686a",
+    "evaluation/candidate-lock.json": "34c6015eb91e0fd500371f9bad66cf28600533acb0c6197022eff3b3eeaa3b5f",
+    "evaluation/candidate-modality_mapping.json": "40890ec869e786e4ca6510367bd1a6cdc2feedaf8564b345b24b38bbce7e0edd",
+    "evaluation/comparison.json": "619fcba21a3eb603c28aab2445055b60c9d7a67f9cef5f963bf29983196ee457",
+    "evaluation/engines/blocked-storage-provenance.json": "ee4de9798b5277218618b453bc8bea3884c877462b5a6c6f86ca07ddfe3f82a3",
+    "evaluation/engines/vela-modality-blocked-b1-native-gpu-32k-v1.json": "315d6053bf4af6f4a5eaa68fcc4c06dfb938fd7d4250c74700fe92fa6647c467",
+    "evaluation/engines/vela-modality-blocked-b1-onnx-gpu-v1.json": "7264fa901a81c6078cd8cabb825d38b18a39746b19eff3ecc692813a256ec1c6",
+    "evaluation/engines/vela-modality-candle-cpu-v1.json": "f1cd9c3c58fd9af874157374a9c30d6ebda07175bd8a52e5fd4a675d7f801a97",
+    "evaluation/engines/vela-modality-standalone-engine-reproduction-v1.json": "6ccab48c6f0c6594c083f6fc12a71388128a57f72b7ccc9af2aac53f2a343abc",
+    "evaluation/final-baseline-fp32.json": "33cabd73d0ed8d8677f072a59c6470a2d00f5ff0ea32b966851ef41ede6e31c8",
+    "evaluation/final-baseline-fp32.predictions.jsonl": "2fe80dcebad0334c54c8d1bd064346d2e9e702a00fb01fba4b7186cd518d8c4e",
+    "evaluation/final-candidate-fp32.json": "2af3b6eb92e55d5e834f741dde44320c249800f302cf5eaff45094c78ea98006",
+    "evaluation/final-candidate-fp32.predictions.jsonl": "2f28bc2e3f7c9a6cd3898efe6210202fc75c4d265158210c1f148635cf8a131c",
+    "evaluation/final-evaluation-plan.json": "2dfa9c27d0f05236b9eb8bdb943c84007751407cdc4d8b73b8a4ff046538f4d0",
+    "evaluation/final-label-manifest.jsonl": "2bf2b4787248172f9df19fdc2203b20e5c83f1967f355922e5e9ba5025064fe0",
+    "evaluation/freeze-decision.json": "b768ec118e5f334d525712d7db215957e0ffdf4f517f462cbbef131e725d5e3b",
+    "evaluation/migration-adaptation-plan.json": "e80a24d2537fb99f767ca032b7ce010af587f01849b2d63332024ad094251463",
+    "evaluation/migration-decision.json": "7a90a2271692a92b358872e5854e5fca2b9809d32b8c30dbf4064bb150ea0776",
+    "evaluation/migration-dev-old.json": "97d807cce2873da2c3bb00ca9aab4b58d66a26f4f9ed06cbef5658118acd83b7",
+    "evaluation/migration-dev-vela.json": "ecaa7927e83815e289eb0434cde024991798da0e6f14e04ecd17cf5b0c352059",
+    "evaluation/migration-plan.json": "453ca8360bb857a207ba383701312f64fa43cb2ab67442b3d049f016ad9eebb0",
+    "reproducibility/LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
+    "reproducibility/README.md": "07530eb8403cb6238e73f1d2cab784922d347236a6f1d8b2baa63f7dead29c54",
+    "reproducibility/bundle-checks.json": "80fe40424704431af730fdbf4d62734c65f9471d6ade580b7f8e9102abde8d12",
+    "reproducibility/contract-tests.txt": "5633287c6133137bb9c0bd8200d445deb1c14766cd532891a6c157bea19a92bc",
+    "reproducibility/corpora/domain-v1/contract.json": "95026e8dde09856c0c099106fed22579d97f2e30157e51cc4777af2771e43073",
+    "reproducibility/corpora/domain-v1/manifest.json": "2f5242918ef2ec6ea97aa9b87ea8dc0dbb21e06b10adf5ec2c490c345b2d63fb",
+    "reproducibility/corpora/modality-v1/contract.json": "87c8b1584d114e307901a3e1a203d93433cffbea0e62e3810825009a5f280a2a",
+    "reproducibility/corpora/modality-v1/manifest.json": "f202327c04bcbc28b0e8984ad00d8f33274cd744954c98bfa20f605bd2a9d833",
+    "reproducibility/corpora/modality-v2/contract.json": "87c8b1584d114e307901a3e1a203d93433cffbea0e62e3810825009a5f280a2a",
+    "reproducibility/corpora/modality-v2/manifest.json": "390faf835e37715d31a2a5160911c2e0e18c30a86c1be131321d33c5cf2a38e0",
+    "reproducibility/corpora/modality-v2-context-v2/contract.json": "87c8b1584d114e307901a3e1a203d93433cffbea0e62e3810825009a5f280a2a",
+    "reproducibility/corpora/modality-v2-context-v2/manifest.json": "43e2339c706d9bb921ffeadf37d855e1b1a37ec659146774fc57da7e99c91d85",
+    "reproducibility/data-file-hashes.json": "adde77781834a0802912b4a8a65264b15317cf2c34f506e8a621f0445ee3c0cd",
+    "reproducibility/data-reproduction.json": "96d5fd587d37ccc0c8d06d10bfa8b539228b8ea70f739dab65c2054ec17c76e5",
+    "reproducibility/environment.json": "cdffff561c770544bc6c3629baea14acc3cb9eb5f463d38d0ca7c3eae04dc7bc",
+    "reproducibility/evaluate_final.py": "c89ef86bc194618d832beaccf7d805fe15dd05bc65c702625f2d17ac3898083c",
+    "reproducibility/initialization-reproduction.json": "0611a8ea461086f3bbaa7df5e696811d639463b886e3a1b45301ed9e698e6bad",
+    "reproducibility/modality-source-files.json": "24c12b23d8877456516ec1ae317e9cc258eef8b1451bd29ba93e622a4fc96782",
+    "reproducibility/onnx/README.md": "6d4643f7e9803b7143ab555d89bd46c33136345bb69b28f7302618991536901b",
+    "reproducibility/onnx/export_classifier.py": "15837367a7ac3a6af308b34c6e72cf1a9a6d1f90ce3bab9a9fdec8d3c117d211",
+    "reproducibility/onnx/generate_engine_reference.py": "d5f9c09553e16aad0181a01f0793f7490993e2a6bf0992802780e62fd41f53af",
+    "reproducibility/onnx/onnx_artifacts.py": "324d3d4f1de13643e09d64e69e72a562fad9e0d30eada9f793d0aab64e2e56d0",
+    "reproducibility/onnx/qualify_onnx_fp32.py": "1bbc9da83314da564d0c0b1d43cb593954f28cb67b8a0fa16716288831d80733",
+    "reproducibility/onnx/requirements.txt": "1eda4cc7cb6ecfce45f2c646cc8c78ffaf08da08d310b8f1877e03fc9fcf5768",
+    "reproducibility/onnx/rewrite_blocked_attention.py": "cbb8d05dab9c257c4977d071fef11e0fba54b8d63c0fa7ada5f28608b13aec5d",
+    "reproducibility/onnx/rewrite_graph.py": "34672e6260b00a6c43298e55e9d082ee0e6c9e3d6102785f3e16bcfd3e56cfd8",
+    "reproducibility/onnx/stable_pooling.py": "8b67fdd8b11a975fe6a2a09e33970b7969c3f682f81f0e6cdb1ec534c90bb221",
+    "reproducibility/onnx/test_rewrite_blocked_attention.py": "7992368e87c4b3291157301bb907dbff273371d0e913ae37113a04d85c6c0336",
+    "reproducibility/prepare_data.py": "033d3b8095bcf50a90f6d12b25ef56a61202396ae487dea07d1977c068d4d302",
+    "reproducibility/release-lineage.json": "693e7564b755efb974b40b1bf72f873134ce3edc27effd08dd4a68781146e54a",
+    "reproducibility/requirements.txt": "fc5132424eb84042d86dca954ad29febd6fa3daec417a1e788bedd9526e41a1c",
+    "reproducibility/src/training/model_classifier/domain_classifier/README.md": "c6c2ab4cfe748dfcf643eb1f9b758dde897737996ccc75b8666b51a97ff1f5c5",
+    "reproducibility/src/training/model_classifier/domain_classifier/__init__.py": "c9744448e6d8dfc7555bf9dc5ca79ee913c5f45d7203e2a84316d5f020db8437",
+    "reproducibility/src/training/model_classifier/domain_classifier/prepare_data.py": "5a3bb37f15b6fcd0e9982224734bfd978a90fdc7a03b9025f40caa4ea9a7d6e4",
+    "reproducibility/src/training/model_classifier/modality_routing_classifier/extend_contracts.py": "b92671dc63baee1b94e32b7489587f2da5131f4a91a3ba04a40faa4fbb01cd1e",
+    "reproducibility/src/training/model_classifier/modality_routing_classifier/prepare_vela_data.py": "8e527a7f5190ae755b7c400a58caf0fa97a23d94a3711e668bd87225b9bd734b",
+    "reproducibility/src/training/model_classifier/sequence_repair/README.md": "258b6cab0851e54c33300d61df8ece8b6f346245cabaedc3623fdeeae4eb73a8",
+    "reproducibility/src/training/model_classifier/sequence_repair/__init__.py": "04cd2a4c037dd1cc2070d57bea979cacb76d4660313a190fdf612007cb33b88e",
+    "reproducibility/src/training/model_classifier/sequence_repair/complete_adapter.py": "f03c9c7743e1bfc051204c454d1d8230bfed8d4664dbcdeac48de3e7ea53f89f",
+    "reproducibility/src/training/model_classifier/sequence_repair/context.py": "8e51031478aca4a825a77d057b2a66a9d09d80e42fb66ccbec3c527f3a891303",
+    "reproducibility/src/training/model_classifier/sequence_repair/corpus.py": "ba3d4f91607db085e55ae1c76701b9b1319fea5ee7dece9851d4ee7375bca108",
+    "reproducibility/src/training/model_classifier/sequence_repair/data/README.md": "11960d9ca18463ccecd12bce5da30eba80b8bc3ab56b1ec5484320c6bedcd6ca",
+    "reproducibility/src/training/model_classifier/sequence_repair/data/aya-fact-reviewed-v2.json": "46127dfeaca954b0a38eef7a93188b736eec9fb1db5eaab833f2d89e51f98449",
+    "reproducibility/src/training/model_classifier/sequence_repair/data/aya-reviewed-labels-v1.json": "5ba7154d4ee94c15c2b6cd5fd494ea981fadc378048f0ad0436f99824c8745c3",
+    "reproducibility/src/training/model_classifier/sequence_repair/data/dolly-fact-reviewed-v2.json": "72aa07f8641b1fd369b365f54172fd5da2ceeb4a776bee8c8777a84e43e081cc",
+    "reproducibility/src/training/model_classifier/sequence_repair/data/source-files.json": "189c532e841346aa17191633f230be4eae0a69caa576c3385a1f2b75cbf9e346",
+    "reproducibility/src/training/model_classifier/sequence_repair/data/task-context-backgrounds.json": "b687f879fa4a5fa56d7fb7b9bef6a6f7b31fa7ad0cb3804365e3e45a68c97101",
+    "reproducibility/src/training/model_classifier/sequence_repair/data.py": "84f922c20a2f3b029f7d363cf7f96f98cd239669f7777de099c63cfa1a1ef199",
+    "reproducibility/src/training/model_classifier/sequence_repair/evaluate.py": "41617631838a1f72e33d402470a9bebd1ab0d7b42b80ac7e8c424a487068c562",
+    "reproducibility/src/training/model_classifier/sequence_repair/export.py": "334d5f27cbd953ef79f6a38b36c5ae9ba615eaf2449f46c16103f8800c04e662",
+    "reproducibility/src/training/model_classifier/sequence_repair/fetch_sources.py": "1abd30d33f456357553facdbde11116134835a087009c63c433faeccf8214638",
+    "reproducibility/src/training/model_classifier/sequence_repair/hydrate_annotations.py": "ea0c44b51afddbeb7dabf084db006eed545022ed819fa40682f4be846b207d62",
+    "reproducibility/src/training/model_classifier/sequence_repair/initialize.py": "d4e3a5ef2ee17759fac1a1bbfb95d165555dfd3d13c0334f38ae6464f2b82406",
+    "reproducibility/src/training/model_classifier/sequence_repair/model.py": "282b64def05cd9157062e4c1936848e30e0e7f18d5883605ad6d01e7f4828742",
+    "reproducibility/src/training/model_classifier/sequence_repair/prepare_context.py": "07d122e4025a25b30bf9c27a29881b4c18db5d908d135ee3f4df78115e54ffe7",
+    "reproducibility/src/training/model_classifier/sequence_repair/runtime_mapping.py": "ec0c553ef625001c09c53a27659cc5162629e303bdef6a93c6091767f49b7fd9",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_annotations.py": "5b331daaf2444b297752d5109dfe098ee05bb847c7fe8d350e826b208c72e34b",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_context.py": "b050b58fbba0bb62dc628d97703a071b5fde4a696a8031755e38f024ac4642a5",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_contracts.py": "3e1fd5e2fd7329c54b6a94a651050202666cf6e63a0555486e604c88a0a24d8f",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_recipe_sources.py": "2687d68f6b55021cbe81dbb35e90ded951918568e70fe46ec38cb55301ac62d4",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_runtime_mapping.py": "46e937fc5c43929c457a1d8709112da331fb557ec4c610e9ff17e0530757e927",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_sampling.py": "dc760c7459fb37ea007caf71e40232ad88854d36d4c20f1475c1f26b3d7ff5a9",
+    "reproducibility/src/training/model_classifier/sequence_repair/tests/test_token_budget.py": "abb0cee3bfaa1e19aadd7336c35d58caad7b1d6e92d2a0511c47dac86bea3262",
+    "reproducibility/src/training/model_classifier/sequence_repair/train.py": "2b6a160522dd6efd24893f6146bb641d7f1a8413bd690587b728c028e85d7e3e",
+    "reproducibility/summarize.py": "913832bf454e4c29c5a6c579d7999b564799a2b88e61499915c2a95cbb6e9a0b",
+    "reproducibility/train_stages.py": "87902e0e2d3f1729a32794179f0469134e3e740f1dad59d4faaaed15253f3249",
+    "reproducibility/training-receipts/modality-v1/run.json": "61232f37e3a58d7ac90798cf8ed75f184fcc8d2a7405b7449b09989ab78d6e58",
+    "reproducibility/training-receipts/modality-v1/selection.json": "cb56165aaa2c6db4d8ed10015a2b14f4ba5e977d1b720301d33d1b390e9d123b",
+    "reproducibility/training-receipts/modality-v2-diversity/run.json": "70d958965b08928bed3d54a982d29a2767a2cb9ba2bd4182c999081ae2ac7eba",
+    "reproducibility/training-receipts/modality-v2-diversity/selection.json": "137c71e36dc8629cbcca1665a4d0d8793e22490afc6f746a810aea2c7a1a63cb",
+    "reproducibility/training-receipts/modality-v3-context/run.json": "0104f13c1a43ec534edfec55661de81118b370c23d9eb40028153b47951940de",
+    "reproducibility/training-receipts/modality-v3-context/selection.json": "076c584384d8784253660d926e308489ec825a3055a1ebec0614b356b7b1fb8f",
+    "reproducibility/training-receipts/modality-v4-diverse-4k/run.json": "0f0f414b3bf84a3d166060f8772d52af92e646ea0335b0fb8dafedeec38e9200",
+    "reproducibility/training-receipts/modality-v4-diverse-4k/selection.json": "a9b4a16ee64ecb0f03f33c92b24bcceecf241e48ca72d6b478f8f8976455f70a",
+    "reproducibility/training-receipts/modality-v5-diverse-context/run.json": "d6d5959c097abed615e5a22770088056ec3a6502520ed1e4e3e0104ad022d7d7",
+    "reproducibility/training-receipts/modality-v5-diverse-context/selection.json": "a6d85b02bf9e5e1b07c7b8eff56ce00ee14e36ea0ade7ada4f742eb290ed7054",
+    "reproducibility/training-receipts/modality-v6-vela-base/dev-step-0.json": "589d9a69d3f81eb5e6624f36ed90b88197b167aba8ebbaea5656d7b200f6603c",
+    "reproducibility/training-receipts/modality-v6-vela-base/dev-step-200.json": "dbe84e85a80e91b1a7f307f2560531a7dc85db59b50017170602356b58171731",
+    "reproducibility/training-receipts/modality-v6-vela-base/run.json": "a6e0136fa8a08b0e03dd581518c2cda1d1a3468c5e32d84be0690888b350baf8",
+    "reproducibility/training-receipts/modality-v6-vela-base/selection.json": "c3c4f945b184eae3556c6931a13a23d22bac3155225cbee98e031121a3d27191"
+  },
+  "onnx_variant": {
+    "precision": "float32",
+    "attention": "query-blocked, complete keys/values",
+    "static_batch_size": 1,
+    "qualified_max_sequence_length": 32768,
+    "custom_op_required": false,
+    "independent_task_quality": false
+  },
+  "additional_files": {
+    "onnx/model.onnx": "74a08c52ce47f522071fd71acee080c0a3a74ad12e3d5fb19ae1dfd244f1059c",
+    "onnx/model.onnx.data": "f0372b44dd9a9ae0d93f0ee0211ec27644dd2ad94351c8022e94c953bd14c399",
+    "adapter-qualification.json": "1c3f3675e975a08f26685ff955263f2c17e6de624115b3a627ea90ca1b5f9c98",
+    "lora/README.md": "46387be05f1fa1bd1fd92c09f615bf8e3bd8d9b499089f785148c1c186f0bd8e",
+    "lora/adapter_config.json": "426d2ca8619487565d658930462e30a0d156641f0800c255f84b3610f44139d5",
+    "lora/adapter_model.safetensors": "52fec9bffae938380605bd1da242fcfded6c75964b89ce112de5206f14f4eb00",
+    "lora/config.json": "4cc261fd20f7acf8eb78c7c81491a57eaff5cbf5dbe73ed738fc87af97e729f2",
+    "lora/label_mapping.json": "63c62f883903b470c24ab5c760f593d0dbe4a01823376ee312dbf77fa9b7686a",
+    "lora/load_adapter.py": "759023bdc0bd563b422068c6b3dd28361a5c2ecdd3f4b8073ef931e040940298",
+    "lora/modality_mapping.json": "40890ec869e786e4ca6510367bd1a6cdc2feedaf8564b345b24b38bbce7e0edd",
+    "lora/special_tokens_map.json": "6e3204c5a4004719c185007a745d1940471a6fb02521cfa0a00306dbb6bc5903",
+    "lora/tokenizer.json": "22fd4a60565f25fee8ebd2866aab07456dd46cbbeb45eaf506529ed8451ee58f",
+    "lora/tokenizer_config.json": "74a259bb1a3811a7e3028adcd07a65765d866d5e66e0e883f0994ccfa67e8455",
+    "lora/verify_adapter.py": "ad9a6a32d8bc488dd56976936c4cccc217939a9c29256a069d4e9430c901b85f",
+    "standalone-verification.json": "1f1004e815211afa6ca714f169e311a54a6b27cc539de304f0d1064ebb6b1421",
+    "adapter-variant-manifest.json": "eeef8cc8f526a7a2c06778e9b8b0c8149670335f8c4be2f8638a4ac93ca59efc"
+  }
+}

reproducibility/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

reproducibility/README.md

@@ -0,0 +1,114 @@
+# Reproduce Vela Modality text-output routing
+
+This text-only encoder classifies a requested output as `AR`, `DIFFUSION`, or
+`BOTH`. It does not process images or audio. The released task starts from a
+fresh adapter and head, then passes a development-only migration to the published
+Vela Base. The old Modality weights are used only as a baseline.
+
+Use Python 3.12, an appropriate Torch 2.10 build, and `requirements.txt`.
+Training uses FP32 parameters with BF16 autocast, SDPA and checkpointing; final
+scoring uses FP32 throughout. Reproducing the recipe does not guarantee identical
+weights across hardware, kernels or package versions. Run the commands below
+from this directory. No training command automatically evaluates final data.
+
+## Reconstruct data and initialize
+
+```bash
+python -m src.training.model_classifier.sequence_repair.fetch_sources \
+  --manifest modality-source-files.json --output artifacts/sources
+hf download llm-semantic-router/mmbert-32k-yarn \
+  --revision 72a23a6640489471eb4ff7ad3ec5bc80af8a27de \
+  --local-dir artifacts/base
+hf download llm-semantic-router/Vela-1.0-Encoder-307M \
+  --revision 5fe5bbb1a88b7fdcc93bb5b9d546c574564eb114 \
+  --local-dir artifacts/vela-base
+python prepare_data.py --sources artifacts/sources \
+  --tokenizer artifacts/base --output artifacts/data
+python train_stages.py --stage initialize --artifacts artifacts
+```
+
+Source files and revisions are pinned for Global-MMLU, Aya, historical-source
+screening and DiffusionDB. Only caption metadata are used from DiffusionDB; no
+images are downloaded. Authored multilingual task contracts are combined with
+captions. Text-answer academic questions and reviewed Aya requests provide AR
+retention examples. They are not naturally balanced three-way modality labels.
+Natural source requests are individually assistant-reviewed; this is not expert
+human annotation. Data keep parent source groups together, and development/final
+contract families are separated from training. Captions translated or reused
+across variants do not count as independent examples.
+
+The diversity extension adds training-only contract families and neutral negative
+contexts. Long data wrap a selected task payload in authored background prose,
+retaining its source split and group. Train and evaluation backgrounds differ.
+The 4K stage selects the 4K subset of the already frozen long data. All 14 data
+files were reproduced byte-for-byte; their hashes are checked by the wrapper.
+The public fresh initializer also reproduced the initial adapter weight hash.
+
+## Train the selected lineage
+
+```bash
+python train_stages.py --stage modality-v1 --artifacts artifacts
+python train_stages.py --stage modality-v2-diversity --artifacts artifacts
+python train_stages.py --stage modality-v4-diverse-4k --artifacts artifacts
+python train_stages.py --stage modality-v5-diverse-context --artifacts artifacts
+python train_stages.py --stage modality-v6-vela-base --artifacts artifacts
+python train_stages.py --stage export --artifacts artifacts
+```
+
+The original selected steps are 100, 300, 400, 400 and 200. Exact data hashes,
+learning rates, sampling, seeds and selection metrics are in `training-receipts/`.
+A rerun must select using its own development results. The omitted v3-context
+trial is a diagnostic branch, not a parent of v4. The model retains standard
+mean pooling throughout; complete task head and classifier weights are saved in
+the adapter.
+
+The final 200-step stage uses the exact published Vela Base and preserves the
+selected task adapter/head. Its direct-transfer baseline did not pass retention;
+the bounded adaptation passes every predeclared short/source/length criterion.
+The same source-macro F1 rule chooses step 200. It uses the explicit token-budget
+microbatch option: the same pre-sampled global four examples are grouped to
+reduce padding while preserving each example's CE weight. Dropout random
+trajectories can differ across regroupings. The earlier stages retain their
+original ungrouped execution path.
+
+The exporter preserves `ModernBertForSequenceClassification`, mean pooling,
+`AR=0`, `DIFFUSION=1`, `BOTH=2`, and both generic and `modality_mapping.json`
+runtime mappings. It locks config, tokenizer, mappings and weights together.
+
+## Independently evaluate the frozen final corpus
+
+```bash
+hf download llm-semantic-router/mmbert32k-modality-router-merged \
+  --revision 9f05f15889d32987d29bbe2e71bff945902ff553 \
+  --local-dir artifacts/baseline
+python evaluate_final.py --artifacts artifacts --model baseline
+python evaluate_final.py --artifacts artifacts --model candidate
+```
+
+FP32 is the default. Both evaluations use identical gold, complete input without
+truncation, fixed label IDs and argmax decisions. Existing outputs are not
+overwritten. Final data are hash-checked against the freeze decision; there is
+no final-based checkpoint or threshold selection. Report academic/AR retention,
+natural AR requests, authored output contracts and long stress separately.
+Long position/length variants are correlated stress cases, not a natural
+long-document benchmark. Recorded forward time excludes tokenization, HTTP,
+queueing and router execution.
+
+The package's contract tests are CPU-only and do not run training:
+
+```bash
+python -m unittest discover \
+  -s src/training/model_classifier/sequence_repair/tests -v
+```
+
+Recompute the source-separated results from the supplied text-free manifest:
+
+```bash
+python summarize.py --manifest ../evaluation/final-label-manifest.jsonl \
+  --baseline ../evaluation/final-baseline-fp32.predictions.jsonl \
+  --candidate ../evaluation/final-candidate-fp32.predictions.jsonl \
+  --contract corpora/modality-v2/contract.json --output comparison.json
+```
+
+All 33 supplied CPU contract tests passed, including the actual tiny-ModernBERT
+gradient checks. The public log redacts its execution directory.

reproducibility/bundle-checks.json

@@ -0,0 +1,15 @@
+{
+  "python_syntax": "all supplied Python modules parsed",
+  "markdown_links": "all relative links resolved",
+  "contract_tests": {
+    "tests": 33,
+    "passed": true,
+    "skipped": 0,
+    "original_log_sha256": "79a494a95cc00d6399518acf48943cf78c1e2df97636dbf379fdc60cd8cd43bc",
+    "public_log_path_redaction": true
+  },
+  "data_reconstruction": "14 frozen data files reproduced byte-for-byte",
+  "initializer": "public fresh initializer weight hash matches original initialization",
+  "frozen_predictions": "all prediction/report bytes verified against their original SHA256; source-separated metrics recomputed",
+  "test_used_for_selection": false
+}

reproducibility/contract-tests.txt

@@ -0,0 +1,48 @@
+test_aya_author_and_text_both_identify_the_reviewed_request (test_annotations.AnnotationTests.test_aya_author_and_text_both_identify_the_reviewed_request) ... ok
+test_dolly_context_is_reconstructed_and_verified_not_discarded (test_annotations.AnnotationTests.test_dolly_context_is_reconstructed_and_verified_not_discarded) ... ok
+test_estimated_search_matches_complete_paragraph_brute_force (test_context.ContextTests.test_estimated_search_matches_complete_paragraph_brute_force) ... ok
+test_oversized_payload_is_rejected_without_truncation (test_context.ContextTests.test_oversized_payload_is_rejected_without_truncation) ... ok
+test_payload_selection_preserves_distinct_source_groups_and_language (test_context.ContextTests.test_payload_selection_preserves_distinct_source_groups_and_language) ... ok
+test_payload_survives_every_position_at_exact_attended_budget (test_context.ContextTests.test_payload_survives_every_position_at_exact_attended_budget) ... ok
+test_accumulated_mean_cross_entropy_matches_full_batch_gradient (test_contracts.ContractTests.test_accumulated_mean_cross_entropy_matches_full_batch_gradient) ... /opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directory
+ok
+test_conflicting_annotations_are_rejected (test_contracts.ContractTests.test_conflicting_annotations_are_rejected) ... ok
+test_explicit_pooling_preserves_legacy_default_and_rejects_unknown_modes (test_contracts.ContractTests.test_explicit_pooling_preserves_legacy_default_and_rejects_unknown_modes) ... ok
+test_label_ids_are_bijective_not_resorted (test_contracts.ContractTests.test_label_ids_are_bijective_not_resorted) ... ok
+test_length_sampling_keeps_a_rare_long_bucket_visible (test_contracts.ContractTests.test_length_sampling_keeps_a_rare_long_bucket_visible) ... ok
+test_length_selection_cannot_be_dominated_by_dense_bucket (test_contracts.ContractTests.test_length_selection_cannot_be_dominated_by_dense_bucket) ... ok
+test_macro_f1_exposes_failure_on_minority_label (test_contracts.ContractTests.test_macro_f1_exposes_failure_on_minority_label) ... ok
+test_present_source_selection_does_not_penalize_unobserved_labels (test_contracts.ContractTests.test_present_source_selection_does_not_penalize_unobserved_labels) ... ok
+test_small_reviewed_source_remains_visible_with_length_and_label_balance (test_contracts.ContractTests.test_small_reviewed_source_remains_visible_with_length_and_label_balance) ... ok
+test_source_selection_does_not_hide_a_reviewed_source_regression (test_contracts.ContractTests.test_source_selection_does_not_hide_a_reviewed_source_regression) ... ok
+test_unicode_normalized_text_and_translation_groups_cannot_leak (test_contracts.ContractTests.test_unicode_normalized_text_and_translation_groups_cannot_leak) ... ok
+test_unicode_separator_inside_json_string_is_not_a_record_boundary (test_contracts.ContractTests.test_unicode_separator_inside_json_string_is_not_a_record_boundary) ... ok
+test_caption_recovery_keeps_embedded_colons_and_rejects_changed_source (test_recipe_sources.SourceContractTests.test_caption_recovery_keeps_embedded_colons_and_rejects_changed_source) ... ok
+test_domain_question_contains_choices_but_not_answer_key (test_recipe_sources.SourceContractTests.test_domain_question_contains_choices_but_not_answer_key) ... ok
+test_translated_duplicate_groups_are_unioned_transitively (test_recipe_sources.SourceContractTests.test_translated_duplicate_groups_are_unioned_transitively) ... ok
+test_domain_uses_category_loader_schema_and_same_ids (test_runtime_mapping.RuntimeMappingTests.test_domain_uses_category_loader_schema_and_same_ids) ... ok
+test_factcheck_polarity_is_preserved_in_both_directions (test_runtime_mapping.RuntimeMappingTests.test_factcheck_polarity_is_preserved_in_both_directions) ... ok
+test_modality_cannot_reorder_native_label_indices (test_runtime_mapping.RuntimeMappingTests.test_modality_cannot_reorder_native_label_indices) ... ok
+test_prompt_guard_and_dynamic_labels_use_router_layout (test_runtime_mapping.RuntimeMappingTests.test_prompt_guard_and_dynamic_labels_use_router_layout) ... ok
+test_sparse_or_duplicate_ids_fail_before_writing (test_runtime_mapping.RuntimeMappingTests.test_sparse_or_duplicate_ids_fail_before_writing) ... ok
+test_invalid_or_incomplete_source_weights_are_rejected (test_sampling.SourceSamplingTests.test_invalid_or_incomplete_source_weights_are_rejected) ... ok
+test_missing_weights_preserve_legacy_rng_sequence (test_sampling.SourceSamplingTests.test_missing_weights_preserve_legacy_rng_sequence) ... ok
+test_unique_coverage_keeps_unseen_rows_visible (test_sampling.SourceSamplingTests.test_unique_coverage_keeps_unseen_rows_visible) ... ok
+test_weight_order_uses_source_occurrence_and_controls_frequency (test_sampling.SourceSamplingTests.test_weight_order_uses_source_occurrence_and_controls_frequency) ... ok
+test_fp32_modernbert_gradient_matches_full_batch_with_dropout_disabled (test_token_budget.TokenBudgetTests.test_fp32_modernbert_gradient_matches_full_batch_with_dropout_disabled) ... artifacts/reference-venv/lib/python3.12/site-packages/torch/jit/_script.py:362: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
+  warnings.warn(
+ok
+test_prefetch_preserves_sampling_and_rng_across_optimizer_steps (test_token_budget.TokenBudgetTests.test_prefetch_preserves_sampling_and_rng_across_optimizer_steps) ... ok
+test_stable_groups_preserve_duplicate_draws_and_bound_padding (test_token_budget.TokenBudgetTests.test_stable_groups_preserve_duplicate_draws_and_bound_padding) ... ok
+
+----------------------------------------------------------------------
+Ran 33 tests in 4.012s
+
+OK