File size: 3,328 Bytes
6cf033d 7cb6b85 6cf033d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
library_name: pytorch
license: mit
tags:
- liquid-neural-network
- intent-classification
- edge
- calibrated
---
# O1-Flash
**A liquid-core intent router — 4.53M parameters answering 77-way
customer-service routing as a calibrated multiple-choice decision.**
Part of the AwareLiquid O-Series. The recurrent core carries per-channel
learnable time constants (τ, geometrically spaced), so one small model holds
both phoneme-scale and utterance-scale state with a fixed-size hidden state.
## Honest status
Measured on **banking77** (10,003 train / 3,080 test rows, 77 intents,
byte-level tokenizer, no external vocab), default config d256/4L/4.53M params,
batch 64:
| steps | test acc | ECE | ECE (temperature-scaled) |
|---|---|---|---|
| 1,500 | 23.28% | — | — |
| 20,000 (lr 1e-3) | 30.23% | 0.485 | 0.161 |
| 20,000 (scheduled 3e-3) | 33.28% | 0.469 | 0.161 |
| **200,000 (scheduled)** | **40.75%** | 0.456 | **0.127** |
- Chance = 1.3%. The default-config model is far from a production router;
40.75% at 4.53M params is the honest ceiling measured so far.
- **Calibration works**: temperature scaling (LBFGS on a held-out half) cuts
ECE by ~3.6× on the final model (T=4.41).
- **Training-stability finding (2026-10-04)**: at lr 3e-3, training diverges
under GPU contention (learns, then collapses to chance — reproduced on two
machines). Fix: `--lr 1e-3` default plus linear-warmup + cosine schedule.
The 200k run above ran fully stable under the schedule.
- The temperature parameter (T=4.41) is saved with the evaluation artifact,
not baked into the checkpoint: apply it to the softmax only when calibrated
probabilities are needed.
## Files
- `o1flash_banking77_200k_sched.pt` — PyTorch checkpoint (18.2 MB,
4.53M params, trained 200k steps with the lr schedule). Contains
`model` state dict + `config`.
## Decision heads (`heads/`)
Fine-tuned decision readouts for the JevBench decision benchmarks — full
harness numbers in `docs/JEVBENCH_RESULTS.md` (GitHub):
- `decision_v1.pt` — from-scratch 4.5M decision model (synthetic decision
corpus, 8k steps); 16–21 ms per decision (Speed axis 100).
- `decision_head_2b.pt` — readout head on the frozen 2B core; best official
configuration (easy 39.6% / orig 34.7%, ECE 0.20, p50 0.168 s).
- `decision_head_2b_v2.pt` — warm-started +20k-step variant (overfits: 33.3%).
- `decision_head_v2b_core.pt` — head on the SFT-v2b core (35.0%).
- `decision_head_realdata.pt` — real-data-family corpus (CLINC150+MNLI+SST-2);
negative result (28.5%).
## Usage
```python
import torch
from benchmarks.real_data_bench import O1Flash, FlashConfig # github.com/AwareLiquid/O1-Flash
ck = torch.load("o1flash_banking77_200k_sched.pt", map_location="cpu",
weights_only=True)
model = O1Flash(FlashConfig(**ck["config"]))
model.load_state_dict(ck["model"])
```
## Limits
- English customer-service text only; byte-level tokenizer.
- 40.75% top-1 on 77 intents — not production. The point of this checkpoint
is reproducibility of the measured numbers and the stability/calibration
findings, not deployment.
- The same architecture/discussion as the rest of the O-Series: see
[EverestAn/MT-LNN](https://huggingface.co/EverestAn/MT-LNN) for the family
card and the awareliquid.ai benchmark page.
|