Add BaseDecision weights, encoder config, tokenizer and model card

README.md

@@ -0,0 +1,206 @@
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- answerdotai/ModernBERT-large
+pipeline_tag: text-classification
+tags:
+- Safetensors
+- SystemOne
+- post-training
+- fine-tine
+- classification
+- structured-output
+- Sentiment Analysis
+- Intent classification
+---
+# BaseDecision Model Card
+
+BaseDecision is a compact, text-only decision model for selecting among caller-provided options, answering boolean questions, and assigning discrete ratings. It combines a ModernBERT-large encoder additional task adaptation. It processes up to **8,192 packed tokens** per request and is approximately **0.42 billion parameters**.
+
+BaseDecision received **approximately 320 million tokens of additional training**. In the supplied third-party evaluation, it achieved the highest unweighted average score among the five compared models and led five of eight benchmarks.
+
+## Model details
+
+| Property | Description |
+|---|---|
+| Model name | BaseDecision |
+| Developer | Hrudayaditya “Aady” Jallu |
+| Repository | [hrudayaditya/BaseDecision](https://github.com/hrudayaditya/BaseDecision) |
+| Model family | ModernBERT-large encoder |
+| Approximate size | 0.42B parameters; rounded model size, not an exact parameter inventory |
+| Architecture | Bidirectional transformer encoder with typed decision heads |
+| Input | Text context, question/instructions, and candidate options or rating levels |
+| Output | Selected answer, option ID, raw logits, option probabilities, and request metadata |
+| Decision types | `choice`, `noul` (boolean), and `score` (discrete rating) |
+| Candidate options | 2–255, subject to the total token budget |
+| Context limit | 8,192 packed tokens, including context, instructions, options, and special tokens |
+| Primary evaluated language | English |
+| Additional training | Approximately 320M tokens |
+| Public Python package | [`basedecision`](https://pypi.org/project/basedecision/); published version reviewed: **0.1.1**, Python **3.10+** |
+
+This is a discriminative decision model. It does not generate free-form explanations, expose a causal language-model `.generate()` interface, or provide the general-purpose capabilities of a chat model. Multiple questions over one context are evaluated independently; shared transformer-state computation is not claimed.
+
+## Intended uses
+
+- Intent classification and workflow routing with explicit candidate actions.
+- Entity-specific sentiment and stance classification.
+- Textual entailment and document-based decision support.
+- Boolean checks and discrete ratings over supplied text.
+- Local applications that need structured decisions without a remote API call.
+
+Plain-text option labels are supported. Descriptions can help disambiguate labels but are not mandatory. Evaluate the actual question wording, options, input lengths, and data distribution used by the application.
+
+The model should not be treated as an autonomous authority for medical, legal, financial, or security decisions. A correct output schema does not establish factual correctness, safe action selection, or suitability for a high-impact workflow.
+
+## Training
+
+### Training volume
+
+**Approximately 320 million tokens were processed during BaseDecision training, as verified by the developer.** This is the additional BaseDecision training volume, not the upstream ModernBERT pretraining corpus size. 
+
+## Third-party evaluation
+
+The table below reproduces the third-party evaluation results supplied by the developer and published in the repository's [benchmark chart](https://github.com/hrudayaditya/BaseDecision/blob/main/docs/assets/benchmarks.png). **Higher is better.** Values are reported benchmark scores; the chart does not identify every row's metric, so they are not uniformly labeled accuracy or F1 here.
+
+| Benchmark | GLiNER 2.5 base | GLiNER2.5-Decide | Decision 1.0 Kai 0.6B | Laya | BaseDecision |
+|---|---:|---:|---:|---:|---:|
+| BANKING77 | 23.8 | 65.6 | 40.7 | 14.3 | **68.2** |
+| FinEntity | 70.1 | 66.2 | 37.0 | 61.0 | **71.3** |
+| ContractNLI | 22.9 | 21.8 | 36.6 | 29.0 | **66.8** |
+| VAST | 35.8 | 35.3 | 20.8 | 40.5 | **66.2** |
+| SGD / SGD-X | 45.9 | 0.8 | 48.5 | 42.4 | **51.1** |
+| MuSR | 34.8 | **45.2** | **45.2** | 43.2 | 44.6 |
+| NLI4CT | 39.8 | **48.0** | 29.8 | 47.7 | 42.3 |
+| PhishNChips | 50.4 | 50.0 | 49.9 | 50.1 | 50.0 |
+| **Unweighted average** | 40.4 | 41.6 | 38.6 | 41.0 | **57.6** |
+
+BaseDecision leads **5 of 8 benchmarks** in this comparison: BANKING77, FinEntity, ContractNLI, VAST, and SGD/SGD-X. Its reported average is **16.0 points above the next-highest reported average**,
+
+## Confidence and calibration
+
+Local inference returns raw softmax probabilities by default. These are relative scores over the supplied choices, not guaranteed probabilities that an answer is correct. Changing the candidate set can change those scores. A high score does not establish evidence completeness or eliminate the possibility of a confidently wrong answer.
+
+Optional scalar-temperature profiles are available for specific assessed workloads:
+
+| Profile | Release behavior | Temperature | Assessment NLL, raw → scaled | Equal-width ECE, raw → scaled |
+|---|---|---:|---:|---:|
+| SGD identifier | Explicit opt-in | 1.4646 | 0.6116 → 0.5830 | 8.05% → 2.97% |
+| SGD schema | Explicit opt-in | 1.4793 | 0.6034 → 0.5606 | 7.88% → 2.96% |
+| VAST | Raw recommended; negligible gain | 1.0071 | 0.6246 → 0.6242 | 7.68% → 7.43% |
+
+These are internal calibration assessments, separate from the third-party benchmark table. Each SGD format was assessed on 1,270 questions from 125 dialogue groups; the formats share groups. VAST used 118 questions/32 groups; CLINC used 156 questions/groups. SGD's short and two-option slices regressed despite aggregate gains.
+
+## Deployment and usage
+
+Install the published Python package:
+
+```bash
+pip install basedecision
+```
+
+Version **0.1.1** is published on [PyPI](https://pypi.org/project/basedecision/). Python **3.10 or newer** is required. The default installation includes PyTorch, Transformers, SafeTensors, Tokenizers, Hugging Face Hub, OpenAI, and Anthropic dependencies. The initial download can therefore be large. To reproduce the reviewed package release, use `pip install basedecision==0.1.1`; pin the runtime and model revision separately when exact reproducibility matters.
+
+The older `runtime`, `hub`, and provider extras remain compatibility aliases; they are no longer needed to install those dependencies. Advanced users who already manage their runtime can use `pip install --no-deps basedecision` and install the dependencies they need themselves. From a repository clone, use `pip install .`.
+
+**Installing the package does not install model weights.** Supply a local checkpoint folder. The README's Hugging Face model identifier remains a placeholder; `load_from_hub(repository_id, revision=commit_hash)` supports downloading a separately published checkpoint, with `local_files_only=True` available for cached operation. Do not use the placeholder as an actual model ID.
+
+Load an exported checkpoint containing `model.safetensors`, its encoder configuration, decision configuration, and tokenizer:
+
+```python
+from basedecision import load
+
+model = load('/path/to/model')
+result = model.choose(
+    context='I was charged twice. Please refund the duplicate payment.',
+    question='What does the customer request?',
+    options=['Refund request', 'Delivery status', 'Change address'],
+)
+print(result.answer)
+print(result.probabilities)  # Raw softmax by default; not calibrated confidence.
+model.close()
+```
+
+The current public SDK selects a suitable CUDA path when available and otherwise uses CPU FP32. Explicit device choices are supported. Apple-silicon Macs use the CPU path; native MPS acceleration is not claimed. The opt-in `cpu_fast` backend uses BF16 and local-attention optimizations, is experimental, checks compatibility at load time, and may be substantially slower on unsuitable hardware. It does not silently substitute another backend. The normal runtime allows Transformers `>=4.48,<6`, while `cpu_fast` supports only its checked Transformers 4.48–4.57 range. A default installation can therefore resolve to Transformers 5 and work with the normal backend while `cpu_fast` raises `CPUFastUnavailable`. Use a compatible runtime deliberately if testing this experimental path.
+
+Inputs are packed with intact options; over-budget requests raise `ContextLengthError` rather than being silently truncated. The 8,192-token limit covers the **entire packed request**, not 8,192 context tokens plus unlimited instructions and options. Simple chunking or voting does not guarantee preservation of cross-document reasoning.
+
+OpenAI and Anthropic adapters provide an alternative SDK backend. They call those providers' models, not BaseDecision weights; their results must not be attributed to BaseDecision. They require explicit selection and send inputs to the chosen provider. Cloud results do not supply BaseDecision's local option probabilities.
+
+### Python interfaces and resource management
+
+| Interface | Purpose |
+|---|---|
+| `model.choose(...)` | Select an option from plain strings or `Option` objects |
+| `model.check(...)` | Return a boolean decision |
+| `model.score(...)` | Select a rating level; optionally return its numeric value and expected value |
+| `decide(model, context=..., questions=...)` | Answer several typed questions independently |
+| `model.predict_batch(...)` / `model.predict_iter(...)` | Process request collections or streams |
+| `result.to_dict()` | Export a result as ordinary JSON-compatible values |
+| `model.count_tokens(request)` | Inspect the packed request length before inference |
+
+Use `with load('/path/to/model') as model:` or call `model.close()` to release resources. Closing is idempotent; subsequent prediction on a closed model raises `InputError`.
+
+The `SystemOne` adapter accepts the Jev/SystemOne `state` and typed `questions` wire format. It is a local-model interface with one forward pass per question; it does not make BaseDecision a Jev model or add shared-context computation. A reference HTTP server is provided in the repository's `examples/` directory and binds to localhost. Review [SystemOne documentation](https://github.com/hrudayaditya/BaseDecision/blob/main/docs/SYSTEMONE.md) before exposing it over a network.
+
+### Errors and privacy
+
+- `ContextLengthError` reports an over-budget input, including required and maximum token counts.
+- `InputError` reports invalid request types, duplicate options, or other malformed input. Context must be text; serialize structured data explicitly.
+- `CPUFastUnavailable` reports an unsupported or failed experimental CPU configuration.
+- Cloud failures raise `ProviderError` with `code`, `retryable`, and `status_code`. The documented error representation excludes API keys and request text.
+- Refused or invalid provider responses raise errors rather than producing a guessed label or silently returning `False`.
+
+Local inference requires no provider API key. Explicit cloud-backend selection sends text to that provider; there is no automatic cloud fallback. See the [API reference](https://github.com/hrudayaditya/BaseDecision/blob/main/docs/API.md), [cloud documentation](https://github.com/hrudayaditya/BaseDecision/blob/main/docs/CLOUD.md), and [security guidance](https://github.com/hrudayaditya/BaseDecision/blob/main/SECURITY.md).
+
+## Performance and hardware
+
+Latency depends on sequence length, option count, batching, processor instruction support, runtime, and system load. No universal CPU latency guarantee is made.
+
+The repository reports these warmed, synthetic, similar-length throughput measurements on an **A100-SXM4-80GB**, using Torch 2.9.1+cu126, BF16 autocast and FP32 weights:
+
+| Approximate packed length | Batch 1 requests/s | Fastest tested batch | Requests/s at that batch |
+|---|---:|---:|---:|
+| 256 | 43.84 | 8 | 273.31 |
+| 512 | 41.88 | 8 | 157.26 |
+| 2,048 | 13.24 | 8 | 30.67 |
+| 4,096 | 4.98 | 8 | 11.44 |
+| 8,176 | 1.49 | 4 | 2.96 |
+
+These are throughput measurements, not service-level tail-latency guarantees. Mixed-length batching can be slower than sequential inference. See [benchmark methodology](https://github.com/hrudayaditya/BaseDecision/blob/main/BENCHMARKS.md).
+
+The current README reports the following approximate timings on one **10-core Apple-silicon laptop using the normal CPU path**:
+
+| Reported input length | Approximate time per answer | Approximate peak memory |
+|---|---:|---:|
+| A few dozen tokens | 0.05 s | 2 GiB |
+| 512 tokens | 0.4 s | 2 GiB |
+| 4,096 tokens | 8 s | 5 GiB |
+| 8,191 tokens | 30 s | 8 GiB |
+
+These are README usage estimates, not a controlled benchmark or a latency guarantee. The complete packed request must still fit within 8,192 tokens. The README estimates model loading at 5–25 seconds depending on the PyTorch version. Earlier experimental server measurements and RC5 verification timings used different implementations and hardware and should not be treated as measurements of the published 0.1.1 package. In particular, the experimental CPU backend can be slower than the normal backend on Apple silicon; measure on the deployment machine.
+
+## Known limitations
+
+- Accepting an 8k input does not guarantee reliable use of all evidence at all positions.
+- Option wording, ordering, domain, and input format can affect predictions.
+- Training and evaluation are primarily English; broad multilingual quality is not established.
+- CPU, GPU, precision, and batching changes can affect logits and occasionally selected labels. Existing GPU calibration must not be assumed to transfer.
+- Bias, subgroup fairness, adversarial robustness, and prompt-injection resistance are not comprehensively characterized by the reported benchmark chart.
+
+For deployment, assess representative labeled data, false-positive and false-negative costs, and escalation behavior. Keep deterministic validation around exact arithmetic, schema constraints, and consequential actions when needed.
+
+## License and attribution
+
+The **BaseDecision software repository is licensed under Apache-2.0**. BaseDecision builds on ModernBERT; retain applicable upstream notices. A software repository license alone does not establish the licensing of separately distributed model weights or training datasets. Consult the weight release's license and each dataset's terms before redistribution or deployment.
+
+## Sources
+
+- [Published Python package, version 0.1.1](https://pypi.org/project/basedecision/0.1.1/)
+- [BaseDecision repository and usage](https://github.com/hrudayaditya/BaseDecision)
+- [Third-party evaluation chart](https://github.com/hrudayaditya/BaseDecision/blob/main/docs/assets/benchmarks.png)
+- [Calibration profiles and assessment scope](https://github.com/hrudayaditya/BaseDecision/blob/main/docs/CALIBRATION.md)
+- [Measured GPU throughput](https://github.com/hrudayaditya/BaseDecision/blob/main/BENCHMARKS.md)
+
+*Updated October 5, 2026 (America/New_York), using the current README and PyPI 0.1.1 package metadata. Benchmark scores are reproduced as reported.*

checksums.txt

@@ -0,0 +1 @@
+SHA-256 - f58f0f12978599f70a6d4956940c7d4be93c8d8a7e729f1143758718cf5cdaf6

encoder/config.json

@@ -0,0 +1,85 @@
+{
+  "architectures": [
+    "ModernBertForMaskedLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 1024,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 2624,
+  "layer_norm_eps": 1e-05,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 16,
+  "num_hidden_layers": 28,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "repad_logits_with_grad": false,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 160000.0,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "transformers_version": "4.57.6",
+  "vocab_size": 50368
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f58f0f12978599f70a6d4956940c7d4be93c8d8a7e729f1143758718cf5cdaf6
+size 1685197088

rl_agent_config.json

@@ -0,0 +1,6 @@
+{
+  "head_layers": 2,
+  "act_costs": {
+    "escalate": 0.5
+  }
+}

tokenizer/special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer/tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}