Text Classification
Transformers
English
bert
bert-mini
classification
decision-model
dynamic-classification
calibrated-classification
option-scoring
oryn
Instructions to use timosarkar/oryn with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use timosarkar/oryn with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="timosarkar/oryn")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("timosarkar/oryn", device_map="auto") - Notebooks
- Google Colab
- Kaggle
added readme
Browse files
README.md
CHANGED
|
@@ -1,3 +1,226 @@
|
|
| 1 |
---
|
| 2 |
license: bsd-3-clause
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: bsd-3-clause
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
library_name: transformers
|
| 6 |
+
pipeline_tag: text-classification
|
| 7 |
+
tags:
|
| 8 |
+
- bert
|
| 9 |
+
- bert-mini
|
| 10 |
+
- classification
|
| 11 |
+
- decision-model
|
| 12 |
+
- dynamic-classification
|
| 13 |
+
- calibrated-classification
|
| 14 |
+
- option-scoring
|
| 15 |
+
- oryn
|
| 16 |
+
datasets:
|
| 17 |
+
- zefang-liu/phishing-email-dataset
|
| 18 |
+
- fancyzhx/ag_news
|
| 19 |
+
- google/boolq
|
| 20 |
+
- SetFit/sst5
|
| 21 |
+
metrics:
|
| 22 |
+
- accuracy
|
| 23 |
+
- brier_score
|
| 24 |
+
- expected_calibration_error
|
| 25 |
+
base_model:
|
| 26 |
+
- boltuix/bert-mini
|
| 27 |
---
|
| 28 |
+
|
| 29 |
+
# Oryn
|
| 30 |
+
|
| 31 |
+
Oryn is a compact, low-latency decision model built on
|
| 32 |
+
[`boltuix/bert-mini`](https://huggingface.co/boltuix/bert-mini).
|
| 33 |
+
|
| 34 |
+
Unlike a conventional classifier with a fixed output layer, Oryn receives the
|
| 35 |
+
classification question and candidate answers at inference time. This allows
|
| 36 |
+
the same model to handle binary decisions, multiclass classification, ordinal
|
| 37 |
+
scoring, and other option-based tasks.
|
| 38 |
+
|
| 39 |
+
Oryn is inspired by the publicly described architecture and training ideas
|
| 40 |
+
behind Laya.
|
| 41 |
+
|
| 42 |
+
## Highlights
|
| 43 |
+
|
| 44 |
+
- Approximately 21 MB of trained weights
|
| 45 |
+
- Approximately 3 ms warmed-up single-request latency in the author's local test
|
| 46 |
+
- Dynamically supplied classification options
|
| 47 |
+
- Supports binary, categorical, and ordinal questions
|
| 48 |
+
- Uses one `[MASK]` marker for each candidate answer
|
| 49 |
+
- Scores all supplied options relative to one another
|
| 50 |
+
- Trained on phishing detection, BoolQ, AG News, and SST-5
|
| 51 |
+
- Designed for local and CPU-friendly inference
|
| 52 |
+
|
| 53 |
+
Latency depends on the device, runtime, input length, and measurement method.
|
| 54 |
+
The reported 3 ms result should not be treated as a universal benchmark.
|
| 55 |
+
|
| 56 |
+
## Architecture
|
| 57 |
+
|
| 58 |
+
Oryn consists of:
|
| 59 |
+
|
| 60 |
+
1. A pretrained BERT Mini encoder
|
| 61 |
+
2. Two additional Transformer encoder layers
|
| 62 |
+
3. One `[MASK]` marker for each candidate answer
|
| 63 |
+
4. A learned linear option scorer
|
| 64 |
+
5. A softmax over the options provided with the request
|
| 65 |
+
|
| 66 |
+
A request is represented approximately as:
|
| 67 |
+
|
| 68 |
+
```text
|
| 69 |
+
question: Which category best describes this text?
|
| 70 |
+
state: The company reported record quarterly revenue.
|
| 71 |
+
options: business [MASK], sports [MASK], technology [MASK], politics [MASK]
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
The contextual representation at each option's `[MASK]` token is passed through
|
| 75 |
+
the shared scoring layer. The resulting logits are normalized across the
|
| 76 |
+
candidate options.
|
| 77 |
+
|
| 78 |
+
Because the answer space is supplied at request time, it is not permanently
|
| 79 |
+
encoded into a fixed classification head.
|
| 80 |
+
|
| 81 |
+
## Evaluation
|
| 82 |
+
|
| 83 |
+
Oryn was evaluated on a random sample of 1,000 held-out examples from its
|
| 84 |
+
multitask validation set.
|
| 85 |
+
|
| 86 |
+
| Metric | Result |
|
| 87 |
+
|---|---:|
|
| 88 |
+
| Overall accuracy | 71.7% |
|
| 89 |
+
| Brier score | 0.3468 |
|
| 90 |
+
| Negative log-likelihood | 0.6228 |
|
| 91 |
+
| Expected calibration error | 0.0381 |
|
| 92 |
+
| Average confidence | 0.7518 |
|
| 93 |
+
| Batched throughput | 129.9 examples/second |
|
| 94 |
+
|
| 95 |
+
### Results by question type
|
| 96 |
+
|
| 97 |
+
| Question type | Examples | Accuracy | Brier score |
|
| 98 |
+
|---|---:|---:|---:|
|
| 99 |
+
| Binary / `noul` | 591 | 85.62% | 0.2083 |
|
| 100 |
+
| Categorical / `choice` | 196 | 79.59% | 0.2793 |
|
| 101 |
+
| Ordinal / `score` | 213 | 25.82% | 0.7933 |
|
| 102 |
+
|
| 103 |
+
The ordinal sentiment task is currently the model's main weakness. The model is
|
| 104 |
+
substantially stronger on binary and categorical decisions.
|
| 105 |
+
|
| 106 |
+
The benchmark was performed with batched GPU inference. Batched latency is not
|
| 107 |
+
equivalent to end-to-end single-request latency.
|
| 108 |
+
|
| 109 |
+
## Training data
|
| 110 |
+
|
| 111 |
+
Oryn was trained on a mixture of:
|
| 112 |
+
|
| 113 |
+
- `zefang-liu/phishing-email-dataset`
|
| 114 |
+
- `fancyzhx/ag_news`
|
| 115 |
+
- `google/boolq`
|
| 116 |
+
- `SetFit/sst5`
|
| 117 |
+
|
| 118 |
+
These datasets expose the model to:
|
| 119 |
+
|
| 120 |
+
- Phishing and fraud detection
|
| 121 |
+
- Binary reading comprehension
|
| 122 |
+
- News topic classification
|
| 123 |
+
- Five-level ordinal sentiment classification
|
| 124 |
+
|
| 125 |
+
## Training method
|
| 126 |
+
|
| 127 |
+
Training used two phases:
|
| 128 |
+
|
| 129 |
+
1. Supervised cross-entropy warmup
|
| 130 |
+
2. Calibration-oriented optimization using differentiable proper-scoring
|
| 131 |
+
objectives
|
| 132 |
+
|
| 133 |
+
For nominal decisions, the calibration objective combines cross-entropy with a
|
| 134 |
+
spherical scoring term. For ordinal decisions, cross-entropy is combined with
|
| 135 |
+
ranked probability score.
|
| 136 |
+
|
| 137 |
+
This training procedure is Laya-inspired, but it does not reproduce the
|
| 138 |
+
original RLCD/REINFORCE procedure exactly.
|
| 139 |
+
|
| 140 |
+
## Intended uses
|
| 141 |
+
|
| 142 |
+
Oryn is suitable for experimentation with:
|
| 143 |
+
|
| 144 |
+
- Message triage
|
| 145 |
+
- Phishing detection
|
| 146 |
+
- Topic classification
|
| 147 |
+
- Sentiment classification
|
| 148 |
+
- Support-request routing
|
| 149 |
+
- Urgency classification
|
| 150 |
+
- Small local decision systems
|
| 151 |
+
- Dynamic classification research
|
| 152 |
+
- Low-latency edge or local inference
|
| 153 |
+
|
| 154 |
+
## General classification
|
| 155 |
+
|
| 156 |
+
Oryn can technically receive any question and any set of options:
|
| 157 |
+
|
| 158 |
+
```python
|
| 159 |
+
result = predict(
|
| 160 |
+
text="The company reported record quarterly revenue.",
|
| 161 |
+
question="Which category best describes this text?",
|
| 162 |
+
options=[
|
| 163 |
+
"business",
|
| 164 |
+
"sports",
|
| 165 |
+
"technology",
|
| 166 |
+
"politics",
|
| 167 |
+
],
|
| 168 |
+
)
|
| 169 |
+
```
|
| 170 |
+
|
| 171 |
+
However, architectural support for arbitrary options does not guarantee
|
| 172 |
+
reliable knowledge of arbitrary domains.
|
| 173 |
+
|
| 174 |
+
Oryn is most reliable on tasks similar to its training mixture. New domains
|
| 175 |
+
should be evaluated and usually fine-tuned before production use.
|
| 176 |
+
|
| 177 |
+
## Example output
|
| 178 |
+
|
| 179 |
+
```python
|
| 180 |
+
{
|
| 181 |
+
"prediction": "business",
|
| 182 |
+
"confidence": 0.8421,
|
| 183 |
+
"probabilities": {
|
| 184 |
+
"business": 0.8421,
|
| 185 |
+
"sports": 0.0314,
|
| 186 |
+
"technology": 0.0987,
|
| 187 |
+
"politics": 0.0278
|
| 188 |
+
}
|
| 189 |
+
}
|
| 190 |
+
```
|
| 191 |
+
|
| 192 |
+
## Files
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
The custom `inference.py` implementation is required because Oryn uses a custom
|
| 196 |
+
option-scoring head rather than a standard Transformers sequence-classification
|
| 197 |
+
head.
|
| 198 |
+
|
| 199 |
+
## Installation
|
| 200 |
+
|
| 201 |
+
```bash
|
| 202 |
+
pip install torch transformers
|
| 203 |
+
```
|
| 204 |
+
|
| 205 |
+
## Inference
|
| 206 |
+
|
| 207 |
+
```bash
|
| 208 |
+
python inference.py \
|
| 209 |
+
--text "The company reported record quarterly revenue." \
|
| 210 |
+
--question "Which category best describes this text?" \
|
| 211 |
+
--options business sports technology politics
|
| 212 |
+
```
|
| 213 |
+
|
| 214 |
+
## Citation
|
| 215 |
+
|
| 216 |
+
If you use Oryn, cite this model repository:
|
| 217 |
+
|
| 218 |
+
```bibtex
|
| 219 |
+
@misc{oryn_bert_mini,
|
| 220 |
+
title = {Oryn: A compact low-latency BERT Mini option-scoring model},
|
| 221 |
+
author = {Timo Sarkar},
|
| 222 |
+
year = {2026},
|
| 223 |
+
howpublished = {Hugging Face},
|
| 224 |
+
note = {A compact low-latency BERT Mini option-scoring model}
|
| 225 |
+
}
|
| 226 |
+
```
|