---
license: apache-2.0
library_name: peft
base_model: Qwen/Qwen3.5-4B
language:
- en
tags:
- lora
- qwen3.5
- decision-model
- classification
- structured-decisions
- openresearch
---

# OR-decision-4B

OR-decision-4B is a rank-16 LoRA adapter for **Qwen/Qwen3.5-4B**, trained to answer
typed decisions: yes/no (`noul`), multiple choice (`choice`), and ordinal ratings
(`score`). This repository contains the trained **adapter**, tokenizer, and a
small inference helper. Loading it also downloads the original Qwen base model.

The model represents each supplied option as a single choice-identifier token.
It scores those identifiers at the end of a fixed prompt and normalizes their
logits over the supplied options. This is a specialized vocabulary-readout
classifier; ordinary chat generation is not the evaluated inference procedure.

## Usage

Install the packages in `requirements.txt`, then run:

```python
import sys
import torch
from huggingface_hub import snapshot_download
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

folder = snapshot_download("alphaXiv/OR-decision-4B")
sys.path.insert(0, folder)
from decision import decision_probabilities

tokenizer = AutoTokenizer.from_pretrained(folder, trust_remote_code=False)
base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3.5-4B",
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=False,
)
model = PeftModel.from_pretrained(base, folder).eval()

probabilities = decision_probabilities(
    model,
    tokenizer,
    context="The customer needs a copy of an invoice.",
    instructions="Which department should handle this request?",
    options=[
        ("billing", "Invoices and payments"),
        ("technical_support", "Software and device problems"),
    ],
    kind="choice",
)
print(probabilities)
print("Decision:", max(probabilities, key=probabilities.get))
```

For yes/no questions use `kind="noul"` and options labeled `false` and `true`.
For ordinal ratings use `kind="score"` and ordered, numeric option labels. The
helper returns the probability distribution; applications can calculate its
expected ordinal value or apply a decision threshold. Keep the supplied option
descriptions and full context. The helper rejects inputs exceeding the evaluated
4,096-token window and supports up to 62 distinct option labels.

## Training and checkpoint selection

- Backbone: Qwen/Qwen3.5-4B.
- LoRA rank: 16; learning rate: 0.00001; global batch size: 128.
- Corpus: 117,186 training examples after a content-hash split of 123,339 source
  examples. The mixture includes classification, reasoning, adequacy judgments,
  and synthetic examples, with frozen Qwen teacher distributions where available.
- Objective: gold-label loss, teacher KL weight 1.0, and frozen-base KL weight 0.3.
  Ordinal gold targets include adjacent-label smoothing.
- Trained across two epochs. The released checkpoint is update 925 of the second
  epoch, selected by minimum gold-label negative log likelihood on a fixed
  1,024-example validation set. It is not the checkpoint with the highest
  validation accuracy.
- Checkpoint export uses Spindle/Miles' PEFT export. Optimizer state and the base
  model weights are not included in this adapter repository.

## Evaluation

The released checkpoint was evaluated using Spindle's native vocabulary readout.
The exported Transformers/PEFT runtime has not been separately rebenchmarked;
runtime, precision, and batching can produce numerical differences.

| Evaluation | Result |
|---|---:|
| Fixed validation set | 754 / 1,024, **73.63%** |
| Validation gold-label NLL | 0.6549 |
| JevBench, primary 4,096-token window | 193 / 231, **83.55%** |
| JevBench, secondary 3,072-token window | 194 / 231, **83.98%** |
| Base Qwen3.5-4B, primary JevBench window | 176 / 231, **76.19%** |

JevBench used the public tasks and original scoring implementation pinned at
[`1bcc55eb6c8cffde2306b3db03ede39b61c6152a`](https://github.com/fstandhartinger/jevbench/tree/1bcc55eb6c8cffde2306b3db03ede39b61c6152a).
Raw and temperature-calibrated correctness counts were identical. Temperature
fitting used a separate 4,000-example calibration subset, with a separate
4,000-example calibration test. No training updates occurred during evaluation.
The helper above returns raw probabilities, without this fitted temperature.

## Limitations

The validation set was reused to select among checkpoints, so its score is not
an independent test estimate. The public benchmark has also been reused across
experiments. Results are from one training seed and 231 benchmark tasks; small
accuracy differences should not be treated as robust improvements. Exact-input
overlap checks do not establish absence of semantic contamination. Confidence
values are not guaranteed to be calibrated in a new application.

This adapter was trained and evaluated on text decisions. The base model's
multimodal capabilities were not evaluated as part of this release.

## License

Apache-2.0. The base model is distributed separately by Qwen under its own
Apache-2.0 license. See `LICENSE` and the
[Qwen3.5-4B model card](https://huggingface.co/Qwen/Qwen3.5-4B).