Text Classification
Transformers
Safetensors
English
basedecision_dyno
feature-extraction
decision-model
system-one
decision-index
multiple-choice
calibration
read-twice
custom-code
lora-merged
custom_code
Instructions to use onlyaady/BaseDecision-Dyno with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use onlyaady/BaseDecision-Dyno with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="onlyaady/BaseDecision-Dyno", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("onlyaady/BaseDecision-Dyno", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from onlyaady/BaseDecision-Dyno: direct link, hf CLI and curl.
- Browser
- Download file 11.7 kB
-
https://huggingface.co/onlyaady/BaseDecision-Dyno/resolve/main/README.md
- Command line
-
hf download hf://onlyaady/BaseDecision-Dyno/README.md
-
curl -L -o README.md https://huggingface.co/onlyaady/BaseDecision-Dyno/resolve/main/README.md
11.7 kB
| library_name: transformers | |
| base_model: | |
| - Qwen/Qwen3.5-4B | |
| base_model_relation: finetune | |
| language: | |
| - en | |
| tags: | |
| - decision-model | |
| - system-one | |
| - decision-index | |
| - text-classification | |
| - multiple-choice | |
| - calibration | |
| - read-twice | |
| - custom-code | |
| - lora-merged | |
| license: apache-2.0 | |
| # BaseDecision-Dyno | |
| **A 4B-class decision model with calibrated option probabilities.** | |
| Give BaseDecision-Dyno a document, conversation, policy, or structured record—and questions with caller-defined options. It returns the selected option and a probability for every option. It scores decisions directly, without generating explanations or decoding an answer sequence. | |
| The calibrated BF16 checkpoint scores **54.80 on the full public Decision Index 0.3 suite**, using the official evaluation kit. | |
| ## At a glance | |
| | Property | Release | | |
| |---|---| | |
| | Model family | Qwen3.5-4B | | |
| | Modality | Text only | | |
| | Weights | Standalone merged checkpoint; no adapter stacking required | | |
| | Runtime | Custom Hugging Face Transformers `AutoModel` | | |
| | Default inference | One scoring pass per question | | |
| | Optional inference | Explicit `mode="flash"`, single read | | |
| | Output | Option key and normalized option probabilities | | |
| | Choice options | 2–255 through the evaluated Decision Index adapter | | |
| | Context | 16,384 **rendered tokens per question**, including prompt overhead | | |
| | Precision | BF16 backbone; FP32 decision readout and softmax | | |
| | Attention | Bidirectional full-attention layers; native recurrence in linear-attention layers | | |
| | Generation / decoding cache | No autoregressive generation; decoding KV cache disabled | | |
| ## Quickstart | |
| The release includes custom model code. Review it before enabling `trust_remote_code=True`. Use `AutoModel` and the structured `decide()` interface to preserve the trained prompt, attention, option mapping, and calibration. | |
| The validated environment used **Python 3.12.15, PyTorch 2.14.0+cu130, and Transformers 5.17.0**. Package loading and sampled protocol checks also passed on an NVIDIA A100-SXM4-80GB. The examples below require a compatible NVIDIA CUDA GPU. | |
| ```python | |
| import torch | |
| from transformers import AutoModel | |
| MODEL_PATH = "onlyaady/BaseDecision-Dyno" | |
| # Match the backend setting used by the evaluated engine. | |
| torch.backends.cuda.enable_cudnn_sdp(False) | |
| model = AutoModel.from_pretrained( | |
| MODEL_PATH, | |
| trust_remote_code=True, | |
| dtype=torch.bfloat16, | |
| device_map={"": "cuda:0"}, | |
| ).eval() | |
| request = { | |
| "state": "The customer's invoice contains the same charge twice.", | |
| "questions": { | |
| "team": { | |
| "type": "choice", | |
| "instructions": "Which team should handle this request?", | |
| "criteria": { | |
| "billing": "Charges, invoices, and refunds", | |
| "technical": "Technical integration problems", | |
| "sales": "Questions about buying the product", | |
| }, | |
| }, | |
| "duplicate": { | |
| "type": "noul", | |
| "instructions": "The customer was charged twice for the same item.", | |
| }, | |
| }, | |
| } | |
| # Read twice is the default. Calibration is applied internally. | |
| result = model.decide(request) | |
| assert result["read_twice"] is True | |
| print(result["answers"]["team"]["choice"]) | |
| print(result["answers"]["team"]["probabilities"]) | |
| print(result["answers"]["duplicate"]["probabilities"]["true"]) | |
| # Explicit single-read mode, with its own calibration applied internally. | |
| flash_result = model.decide(request, mode="flash") | |
| assert flash_result["read_twice"] is False | |
| ``` | |
| For a single question, `{"state": ..., "question": ...}` is also accepted; its answer is returned under `answers["q"]`. Multiple questions are processed independently, each in its own sequence and forward pass. They do not share a single prefill pass. | |
| The response includes `answers`, `mode`, `read_twice`, `sequence_tokens`, `calibrated`, and `release_id`. For both decision types, the native API returns `choice` and `probabilities`. A `noul` answer uses the probability keys `"true"` and `"false"`; the Decision Index adapter converts this distribution into the benchmark’s scalar `noul` response. | |
| ### Persistent inference | |
| Download the model and its included JSONL runner: | |
| ```bash | |
| hf download onlyaady/BaseDecision-Dyno --local-dir BaseDecision-Dyno | |
| cd BaseDecision-Dyno | |
| # Default: read twice, with calibration applied internally. | |
| python inference.py --input requests.jsonl | |
| # Explicit single-read mode, with its own calibration. | |
| python inference.py --mode flash --input requests.jsonl | |
| ``` | |
| Each input line is one structured JSON request; each output line is one JSON response. Create `requests.jsonl` with your requests before running these commands. | |
| The runner loads the model once and keeps it resident across successive requests. Omit `--input` to read requests from stdin until EOF. Use the validated CUDA environment. No compilation, quantization, or cross-request batching is enabled by this runner. | |
| ## Read twice by default | |
| Read twice repeats the state and question/options content within one rendered sequence, followed by **one scoring pass per question**. It is not two independent predictions, voting, or generated reasoning. | |
| - `model.decide(request)`, `model(request)`, and `model.prepare_request(request)` default to read twice. | |
| - `mode="flash"` explicitly selects a single read. | |
| - The 16,384-token limit applies **after rendering and repetition**. Default mode therefore accepts less original content than flash mode. | |
| - Oversized requests raise a length error. Inputs are not silently truncated, and default mode never falls back automatically to flash. | |
| - Keep state and questions structured; do not manually repeat the input before calling `decide()`. | |
| The default is implemented in the supplied model code rather than left to the caller's prompt. Someone who modifies the code or directly uses lower-level weights can change that behavior. | |
| ## Decision Index results | |
| **Completed self-run public evaluation with the official kit, edition 0.3.** The exact calibrated model and strict default protocol above were used. These are public results, not a maintainer-run private or Full leaderboard score. | |
| | Metric | Result | | |
| |---|---:| | |
| | Public Decision Index | **53.83** | | |
| | Raw index | 65.19 | | |
| | Breadth skill score | 52.16 | | |
| | Requests processed before official exclusions | 140,620 | | |
| | Requests after official exclusions | 140,178 | | |
| | Successful scored requests | 140,016 | | |
| | Unsupported scored requests | 162 | | |
| | Errors | 0 | | |
| Index values are **scores, not accuracy percentages**. The public composite includes 37 panel benchmarks. The official report also contains additional diagnostic benchmark entries. | |
| | Area | Skill score (0–100) | Raw score (0–100) | | |
| |---|---:|---:| | |
| | Knowledge & Reasoning | 35.96 | 50.26 | | |
| | Language Understanding | 62.37 | 73.23 | | |
| | Retrieval & Classification | 59.15 | 70.28 | | |
| | Tools & Automation | 71.66 | 74.19 | | |
| | Arts & Human Taste | 34.70 | 56.36 | | |
| Raw scores combine benchmark-specific metrics; they should not all be read as plain accuracy. See the accompanying [`scores.json`](scores.json) for individual benchmarks, coverage, and scoring details. | |
| Unsupported requests were retained in the official scoring path: POP909-CL (158), ToolRet (2), API-Bank (1), and HLE (1). The summary reports alone do not establish the individual refusal reason for every row. | |
| ## Calibration | |
| Mode-specific scalar temperatures are applied **once to raw logits**, followed by a softmax over the current question's options: | |
| ```text | |
| probabilities = softmax(raw_logits / temperature) | |
| ``` | |
| | Mode | Temperature | | |
| |---|---:| | |
| | Default / read twice | 2.0561549881751153 | | |
| | Flash / single read | 1.9163401529278457 | | |
| `decide()` already applies the appropriate temperature. Do not apply another temperature to its returned probabilities. These values replace the earlier 1.26 setting. | |
| Temperatures were fitted on 10,000 decisions and audited on 10,000 separate decisions: 8,000 held out by group and 2,000 from families absent from temperature fitting. Those families were not necessarily absent from backbone training. | |
| | Mode | Audit metric ↓ | Earlier T = 1.26 | Calibrated | | |
| |---|---|---:|---:| | |
| | Default | Negative log-likelihood | 0.897542 | **0.817370** | | |
| | Default | Brier score, summed over options | 0.390352 | **0.365954** | | |
| | Default | 15-bin hard-label ECE | 0.107526 | **0.028381** | | |
| | Flash | Negative log-likelihood | 0.876103 | **0.823770** | | |
| | Flash | Brier score, summed over options | 0.385906 | **0.368202** | | |
| | Flash | 15-bin hard-label ECE | 0.091027 | **0.030229** | | |
| Accuracy on the 9,266 hard-labeled audit decisions was unchanged by calibration: 70.26% in default mode and 69.78% in flash mode. Positive scalar temperature scaling preserves the selected option. These audit accuracies are separate from the Decision Index score. | |
| Calibration improved on this audit distribution; it is not guaranteed on every domain. Training, quantization, or a material backend change requires renewed evaluation and may require recalibration. | |
| ## Performance and deployment | |
| The completed public run measured the following request latencies on shared **NVIDIA RTX PRO 6000 Blackwell** resources: | |
| | Default read-twice latency | Time | | |
| |---|---:| | |
| | Median | 53.3 ms | | |
| These are CUDA-synchronized individual-request timings from the public evaluation. Shared physical GPU contention and the benchmark request mix affect the measurements. They are not the maintainers' dedicated-hardware latency result, and they exclude model loading and external network/queueing overhead. | |
| On an earlier short-input sample (60 distinct requests per mode, each measured three times; up to 512 original single-read rendered tokens and at most 32 options), the uncompiled runtime measured: | |
| | Mode | Median | p95 | | |
| |---|---:|---:| | |
| | Default / read twice | 29.58 ms | 63.43 ms | | |
| | Flash / single read | 26.15 ms | 36.32 ms | | |
| The short-input results do not establish a universal 150 ms default or 100 ms flash bound. Long contexts and many-question requests can take substantially longer. | |
| Keep the model loaded across requests. The supplied runner does this without changing the scoring path. Compilation and quantized inference are not qualified for this release. A 16GB laptop deployment remains a target; this BF16 GPU release does not establish laptop memory fit or latency. | |
| ## Limitations | |
| - Text-only release; image, audio, and video inference are not supported. | |
| - The evaluated adapter supports `choice` and `noul`; a dedicated ordinal `score` response interface is not qualified by this release. | |
| - Default repetition consumes context and compute. The context limit is per rendered question, not 16k original tokens plus a second copy. | |
| - Option probabilities are normalized over the supplied options. They do not prove that the option set contains a correct answer. | |
| - English-focused evaluation does not establish comparable multilingual performance. | |
| - Factual, numerical, ambiguous, and out-of-domain decisions can be wrong, including when confidence is high. | |
| - Adversarial robustness and resistance to instructions planted in the state have not been established. | |
| - Owners can modify the supplied runtime; read-twice enforcement is an API behavior, not a cryptographic restriction on weights. | |
| ## Licensing and acknowledgements | |
| The **BaseDecision-Dyno software repository & model weights are licensed under Apache-2.0**. | |
| This release builds on [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B); Qwen3.5-4B's base license does not by itself settle the licensing of this derived release or all training sources. Preserve applicable upstream notices and dataset terms. | |
| Special Thanks to [ezjev](https://github.com/everettjf/ezjev) & Maintainer of tasksource. | |