README.md

---
license: bsd-3-clause
language:
- en
library_name: transformers
pipeline_tag: text-classification
tags:
- bert
- bert-mini
- classification
- decision-model
- dynamic-classification
- calibrated-classification
- option-scoring
- oryn
datasets:
- zefang-liu/phishing-email-dataset
- fancyzhx/ag_news
- google/boolq
- SetFit/sst5
metrics:
- accuracy
- brier_score
- expected_calibration_error
base_model:
- boltuix/bert-mini
---

# Oryn

Oryn is a compact, low-latency decision model built on
[`boltuix/bert-mini`](https://huggingface.co/boltuix/bert-mini).

Unlike a conventional classifier with a fixed output layer, Oryn receives the
classification question and candidate answers at inference time. This allows
the same model to handle binary decisions, multiclass classification, ordinal
scoring, and other option-based tasks.

Oryn is inspired by the publicly described architecture and training ideas
behind Laya.

## Highlights

- Approximately 21 MB of trained weights
- Approximately 3 ms warmed-up single-request latency in the author's local test
- Dynamically supplied classification options
- Supports binary, categorical, and ordinal questions
- Uses one `[MASK]` marker for each candidate answer
- Scores all supplied options relative to one another
- Trained on phishing detection, BoolQ, AG News, and SST-5
- Designed for local and CPU-friendly inference

Latency depends on the device, runtime, input length, and measurement method.
The reported 3 ms result should not be treated as a universal benchmark.

## Architecture

Oryn consists of:

1. A pretrained BERT Mini encoder
2. Two additional Transformer encoder layers
3. One `[MASK]` marker for each candidate answer
4. A learned linear option scorer
5. A softmax over the options provided with the request

A request is represented approximately as:

```text
question: Which category best describes this text?
state: The company reported record quarterly revenue.
options: business [MASK], sports [MASK], technology [MASK], politics [MASK]
```

The contextual representation at each option's `[MASK]` token is passed through
the shared scoring layer. The resulting logits are normalized across the
candidate options.

Because the answer space is supplied at request time, it is not permanently
encoded into a fixed classification head.

## Evaluation

Oryn was evaluated on a random sample of 1,000 held-out examples from its
multitask validation set.

| Metric | Result |
|---|---:|
| Overall accuracy | 71.7% |
| Brier score | 0.3468 |
| Negative log-likelihood | 0.6228 |
| Expected calibration error | 0.0381 |
| Average confidence | 0.7518 |
| Batched throughput | 129.9 examples/second |

### Results by question type

| Question type | Examples | Accuracy | Brier score |
|---|---:|---:|---:|
| Binary / `noul` | 591 | 85.62% | 0.2083 |
| Categorical / `choice` | 196 | 79.59% | 0.2793 |
| Ordinal / `score` | 213 | 25.82% | 0.7933 |

The ordinal sentiment task is currently the model's main weakness. The model is
substantially stronger on binary and categorical decisions.

The benchmark was performed with batched GPU inference. Batched latency is not
equivalent to end-to-end single-request latency.

## Training data

Oryn was trained on a mixture of:

- `zefang-liu/phishing-email-dataset`
- `fancyzhx/ag_news`
- `google/boolq`
- `SetFit/sst5`

These datasets expose the model to:

- Phishing and fraud detection
- Binary reading comprehension
- News topic classification
- Five-level ordinal sentiment classification

## Training method

Training used two phases:

1. Supervised cross-entropy warmup
2. Calibration-oriented optimization using differentiable proper-scoring
   objectives

For nominal decisions, the calibration objective combines cross-entropy with a
spherical scoring term. For ordinal decisions, cross-entropy is combined with
ranked probability score.

This training procedure is Laya-inspired, but it does not reproduce the
original RLCD/REINFORCE procedure exactly.

## Intended uses

Oryn is suitable for experimentation with:

- Message triage
- Phishing detection
- Topic classification
- Sentiment classification
- Support-request routing
- Urgency classification
- Small local decision systems
- Dynamic classification research
- Low-latency edge or local inference

## General classification

Oryn can technically receive any question and any set of options:

```python
result = predict(
    text="The company reported record quarterly revenue.",
    question="Which category best describes this text?",
    options=[
        "business",
        "sports",
        "technology",
        "politics",
    ],
)
```

However, architectural support for arbitrary options does not guarantee
reliable knowledge of arbitrary domains.

Oryn is most reliable on tasks similar to its training mixture. New domains
should be evaluated and usually fine-tuned before production use.

## Example output

```python
{
    "prediction": "business",
    "confidence": 0.8421,
    "probabilities": {
        "business": 0.8421,
        "sports": 0.0314,
        "technology": 0.0987,
        "politics": 0.0278
    }
}
```

## Files


The custom `inference.py` implementation is required because Oryn uses a custom
option-scoring head rather than a standard Transformers sequence-classification
head.

## Installation

```bash
pip install torch transformers
```

## Inference

```bash
python inference.py \
  --text "The company reported record quarterly revenue." \
  --question "Which category best describes this text?" \
  --options business sports technology politics
```

## Citation

If you use Oryn, cite this model repository:

```bibtex
@misc{oryn_bert_mini,
  title        = {Oryn: A compact low-latency BERT Mini option-scoring model},
  author       = {Timo Sarkar},
  year         = {2026},
  howpublished = {Hugging Face},
  note         = {A compact low-latency BERT Mini option-scoring model}
}
```