| --- |
| license: apache-2.0 |
| library_name: tflite |
| pipeline_tag: text-classification |
| tags: |
| - edge-tpu |
| - coral |
| - tflite |
| - int8 |
| - typed-decisions |
| - systemone |
| - decision-index |
| - reef |
| base_model: minishlab/potion-base-8M |
| --- |
| |
| # Reef |
|
|
| **A general typed-decision model that runs entirely on a Google Coral Edge TPU.** You send a |
| state (text or JSON) and typed questions: yes/no (`noul`), one-of-N (`choice`) or an ordinal rubric |
| (`score`). You get a calibrated probability for every option. About 11M parameters in total, |
| int8, one Edge TPU subgraph that fits in the Coral's on-chip memory. |
|
|
| Part of **[Reef](https://github.com/SeraphimSerapis/reef)**: give your retired Coral a new job. |
|
|
| ## Decision Index 0.2.1 |
|
|
| Self-measured with the public harness ([apolinario/decision-index](https://github.com/apolinario/decision-index), |
| MIT) on the full local suite (150,246 requests). **This is not an official leaderboard entry.** |
| The comparison rows are the published board scores. |
|
|
| | Model | Params (total) | Index | |
| |---|---|---| |
| | **Reef (this model, int8)** | **11.2M** | **4.73** | |
| | GLiNER 2.5 small ([fastino/gliner2.5-small-v1](https://huggingface.co/fastino/gliner2.5-small-v1)) | 73.9M | 3.82 | |
| | Reef v1 (previous version, a homelab specialist) | 10.3M | 2.38 | |
|
|
| How the Reef score was measured: |
|
|
| - **Serving mode.** Measured with the server's default (lenient) mode: over-long inputs are truncated to the trained limits instead of refused. With `--strict`, the score is about 0.15–0.2 lower (measured on a 19,640-row subset with an earlier checkpoint). |
| - **HLE.** HLE has no rows in the local suite and is scored 0, so the number is a lower bound for the board. |
| - **Run-to-run variance.** Retraining moved single benchmarks by up to ±0.1 skill (PhishNChips most), so treat differences of about ±0.3 Index points between checkpoints as noise. |
| - **Params.** 7.38M are in the int8 encoder on the Edge TPU and 3.78M in the host's token-embedding table (frozen [potion-base-8M](https://huggingface.co/minishlab/potion-base-8M) vectors projected to 128 dimensions). All neural-network compute runs on the Coral. The host tokenises, looks up token vectors and applies a softmax. |
|
|
| | Area | Skill (×100) | |
| |---|---| |
| | Retrieval | 10.47 | |
| | Tools | 5.56 | |
| | Language | 4.79 | |
| | Arts & human taste | 1.62 | |
| | Knowledge | 0.84 | |
|
|
| **Where it does well:** intent detection (BANKING77 0.26 skill, CLINC150+OOS 0.14 with 77/151 |
| options), When2Call (0.25), FinEntity (0.37), PhishNChips (0.18), ContractNLI (0.10). |
|
|
| **Where it doesn't:** it sits at chance, with no detectable signal, on tool retrieval and |
| selection (BFCL, ToolRet), BRIGHT, HoVer, ANLI, VAST, Amazon ESCI and the knowledge and reasoning |
| benchmarks (MMLU-Pro, BBH, GSM8K, CLadder, CRUXEval). An 11M model trained from scratch has little |
| world knowledge. |
|
|
| **Shortcuts deliberately not used.** We did not rank documents by the harness's BM25 order |
| (ToolRet/BRIGHT), and we did not decode BFCL as "exactly one tool". Both would have added Index |
| points without any model skill. |
|
|
| ## On the Edge TPU |
|
|
| Same int8 model on a Coral USB Accelerator: |
|
|
| | Test set (held out) | Accuracy | KL ↓ | |
| |---|---|---| |
| | Homelab packs (5 packs, Gemma 4 soft labels) | 0.807 | 0.212 | |
| | Answer-audit (judge for agent transcripts) | 0.705 | 0.634 | |
| | typed-decisions test | 0.535 | 0.294 | |
| | Open-Jev OOD | 0.695 | 0.610 | |
|
|
| For the narrow homelab packs alone, the previous homelab-specialist version is still slightly |
| better (0.84). This model is the generalist. |
|
|
| ## Quickstart |
|
|
| ```bash |
| pip install "reef-decide @ git+https://github.com/SeraphimSerapis/reef" |
| reef serve --host 0.0.0.0 --port 8080 # fetches this model; Edge TPU if attached, else CPU |
| ``` |
|
|
| ```bash |
| curl -s localhost:8080/v1/systemone -d '{ |
| "state": {"source": "uptime-kuma", "monitor": "nextcloud", "status": "down", "down_for": "14m"}, |
| "questions": {"page": {"type": "noul", "instructions": "Should someone be woken up for this?"}} |
| }' |
| ``` |
|
|
| ## Files |
|
|
| | File | What | |
| |---|---| |
| | `reef_int8_edgetpu.tflite` | encoder compiled for the Edge TPU (1 subgraph, 7.36 MiB on-chip, 0 B off-chip) | |
| | `reef_int8.tflite` | the same int8 encoder for CPU (LiteRT, XNNPACK) | |
| | `embeddings.npz` | token / role / level embedding tables (looked up on the host) | |
| | `tokenizer.json`, `config.json` | tokenizer, shapes and per-type calibration temperatures | |
| | `reef_float.weights.h5` | float training weights (Keras), for `reef train --init seraphimserapis/reef` | |
|
|
| ## Model |
|
|
| - **Convolutional encoder**, no attention (the Edge TPU compiler cannot map it): 11 blocks, width 256, MLP expansion 3, a 3×5 depthwise conv and mean+max state summaries, plus two question bands with their own summaries and a multiplicative gate. |
| - **Input**: a 384-token state region plus a 384-token question region, folded into a 24×32 grid. Long states are read in overlapping windows, and questions with many options as a knockout. |
| - **Training stages**: |
| 1. pre-training on about 300k public decision cases; |
| 2. fine-tuning on homelab packs, answer-audit, typed-decisions, Open-Jev and the public train sets below; |
| 3. quantisation-aware fine-tuning, so the int8 model stays within about 1–2 accuracy points of the float model. |
|
|
| ## Training data |
|
|
| All public data comes from **train splits only**, deduplicated against the Decision Index test |
| inputs (normalised exact-match on distinguishing inputs, lines and options), with permissive |
| licences. It uses human labels with label smoothing. One exception was kept on purpose: tool |
| descriptions shared by public tool catalogs (Glaive, ToolACE, When2Call) also appear in the |
| ToolRet/BRIGHT candidate corpus; the model scores at chance on ToolRet. |
|
|
| - **Intents:** MASSIVE en/de, SNIPS, BANKING77, CLINC150, MINDS-14 |
| - **NLI and contracts:** WANLI, MultiNLI (OANC genres only), ContractNLI |
| - **Tools:** Glaive function calling v2, ToolACE, When2Call, API-Bank |
| - **Retrieval:** Amazon ESCI |
| - **Sentiment, finance and toxicity:** GoEmotions, Civil Comments, SMS Spam, SEntFiN, Twitter financial news topic/sentiment, iSarcasmEval |
| - **Commonsense:** HellaSwag, WinoGrande, GSM8K |
| - **Benchmark data:** [ZefanCai/Open-Jev](https://huggingface.co/datasets/ZefanCai/Open-Jev) (CC0) and [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) (Apache-2.0) |
| - **Generated home-appliance households:** the harness's generator, MIT, with seeds disjoint from the test households |
|
|
| Licences: CC BY 4.0, CC BY 3.0, CC0, Apache-2.0 and MIT. |
|
|
| The homelab packs and the answer-audit pack are synthetic (no real home data). Their soft labels |
| are answer distributions from **Gemma 4**: |
|
|
| - homelab packs: E4B and 26B-A4B, an ensemble of option logprobs; |
| - answer-audit: E4B. |
|
|
| Thank you to the Gemma team. Question wording for the public sets mirrors the Decision Index |
| adapters (MIT). |
|
|
| ## Limitations |
|
|
| - **Decisions, not reasoning.** It doesn't know facts or do arithmetic. Use it to filter, route and rank cheap, frequent decisions, and send low-confidence ones to a bigger model. |
| - **Inherited judgement.** Distilled labels inherit the teacher's judgement and blind spots. Keep a human in the loop for anything safety-critical. |
| - **Speed.** On a modern laptop CPU it runs at a similar speed. The Coral makes it cheap to run next to Home Assistant on a Pi, an N100 or a NAS. |
|
|
| ## License |
|
|
| Apache-2.0. See [NOTICE](https://github.com/SeraphimSerapis/reef/blob/main/NOTICE) for third-party credits. |
|
|
