| --- |
| license: apache-2.0 |
| language: |
| - en |
| library_name: safetensors |
| pipeline_tag: text-classification |
| tags: |
| - artoru |
| - artoru-edge |
| - artoru-desk |
| - edge-ai |
| - intent-routing |
| - agentic |
| - on-device |
| - local-inference |
| - safetensors |
| - onnx-runtime |
| - int8 |
| - quantization |
| --- |
| |
| # Artoru # EDGE 65M |
| <img src="./banner.svg" alt="Artoru EDGE 65M Banner" width="100%" /> |
|
|
| **First preview. The local agentic brand starts here.** |
|
|
| Not another cloud chat clone. **Artoru** is the local-first, multi-agent line you run on your own machine — and **EDGE 65M** is the opening chapter: the tiny routing engine inside **Artoru Desk**, shipped as a public first preview with LoRAs and more already on the road. |
|
|
| The arena already has its myths. Artoru is our line in that sky — built to live on *your* hardware. |
|
|
| --- |
|
|
| | | | |
| | --- | --- | |
| | **Brand** | **Artoru** | |
| | **Model** | **Artoru EDGE 65M** (first EDGE preview) | |
| | **Product** | **Artoru Desk** — local-first multi-agent workspace | |
| | **Routing win** | **~98.66%** product-scored · **~99.16%** handled (**EDGE + Chief**) | |
| | **Speed** | ORT CUDA FP32 **~134,462 tok/s** (batch 32, RTX 5090) | |
| | **Params** | ~**65.39M** | |
|
|
| --- |
|
|
| ## Why this card exists |
|
|
| Most model cards apologize for being small. This one doesn’t. |
|
|
| **EDGE 65M** is not trying to write your novel, debate philosophy, or impersonate a search engine. It has one job and it does that job like a blade: **read the turn, name the intent, route the desk** — on-device, at reflex speed, without asking the cloud for permission to think. |
|
|
| If Desk is the stage, EDGE is the stage manager with a stopwatch. |
|
|
| --- |
|
|
| ## The win |
|
|
| Cloud models talk. **EDGE routes.** |
|
|
| **EDGE + Chief** on the product eval (n=372): |
|
|
| - **~98.66%** product-scored accuracy |
| - **~99.16%** handled-rate |
|
|
| That’s the number. Own it. |
|
|
| **Throughput:** ORT CUDA FP32 batch 32 ≈ **134k tok/s** (~0.11 ms/sample, ~8.9k samples/s) on short routing utterances. Batched PyTorch FP16 ≈ **84k tok/s**. |
|
|
| Tiny model. Massive local routing power. |
|
|
| No asterisk that says “but only if the API is up.” No waiting room. No meter ticking while you decide whether to draft an email or ask the Chief. |
|
|
| --- |
|
|
| ## The story: from Micro to EDGE |
|
|
| Before Artoru had a name people could shout, there was a quieter public router — **AMS Micro** on `vvarghese/ams-micro-70m`. Honest work. On-device. Already thinking in intents. |
|
|
| **Artoru EDGE 65M** is that lineage stepping into the light with a brand, a desk, and a scoreboard: |
|
|
| | | Prior public Micro | **Artoru EDGE 65M** | |
| | --- | --- | --- | |
| | **Identity** | Earlier published on-device router (`vvarghese/ams-micro-70m`) | **Today’s Artoru preview** — Desk’s local routing engine | |
| | **Product routing** | Earlier public baseline | **~98.66% / ~99.16%** handled with **EDGE + Chief** | |
| | **Speed** | Prior CPU ORT full-test ~9.2 s (older publish) | **~134k tok/s** ORT CUDA FP32 b32 | |
| | **This pack** | — | FP32 + int8 + Q4 | |
|
|
| Same obsession — understand first, spend heavy compute second. New costume. Sharper teeth. Product attached. |
|
|
| --- |
|
|
| ## Why local still wins |
|
|
| Because your desk is not a demo. |
|
|
| Because “upload everything to someone else’s GPU” is a lifestyle choice, not a law of physics. |
|
|
| Because routing is a *reflex*, and reflexes that cross the ocean feel like lag even when the chat model is brilliant. |
|
|
| Local means: |
|
|
| - **Privacy by architecture** — intent classification happens where the keystrokes already are |
| - **Latency you can feel** — ~134k tok/s class throughput on the FP32 ORT path (batch 32, RTX 5090) |
| - **A desk that still works when the wifi sulks** |
| - **Agents that ask permission** instead of shipping your draft into the void |
|
|
| Artoru is not anti-cloud. Artoru is **pro-sovereignty**. Cloud models are invited guests. EDGE is the bouncer who knows your name. |
|
|
| --- |
|
|
| ## Artoru Desk — the product frame |
|
|
| **Artoru Desk** is the product: a **local-first multi-agent workspace** — agents, tools, approvals, panic, memory, and a full surface that lives on your hardware. |
|
|
| **EDGE 65M** is the fast intent/routing model in that stack. |
| **Chief** is the handoff layer that rides with EDGE on hard turns — one piece of Desk, not the whole story. |
|
|
| ```text |
| [ You type ] → [ EDGE 65M names the intent ] → [ Desk routes ] |
| └─ low confidence / hard intent → [ Chief ] |
| ``` |
|
|
| Desk > any single model. |
|
|
| EDGE is how Desk decides *what to do next* without round-tripping the cloud for every reflex. Chief is how Desk stays honest when the turn is sharper than a tiny router should pretend. |
|
|
| Chat is never empty theater. Someone is always addressed — usually the Chief — and EDGE is the quiet nerve that decides whether the turn is a greeting, a task assign, an escalate, a skill suggest, or “ask the human.” |
|
|
| --- |
|
|
| ## How EDGE + Chief feels |
|
|
| Imagine opening Desk. |
|
|
| You say something ordinary. EDGE fires. The room already knows whether this is smalltalk, a meeting, a fetch, or a handoff. |
|
|
| You say something that matters — escalate, assign, a Chief question. Confidence dips or the intent lands in the handoff set. **Chief** steps in. Visible. Ownable. Not a silent model swap behind a spinner. |
|
|
| That duet is the product eval story: |
|
|
| | Metric | Value | |
| | --- | --- | |
| | Product-scored accuracy | **~98.66%** | |
| | Handled-rate (**EDGE + Chief**) | **~99.16%** | |
| | Eval size | n=372 | |
|
|
| EDGE alone is the sprinter. |
| EDGE + Chief is the relay team that finishes the race. |
|
|
| If routing were only accuracy cosplay, we’d ship a bigger model and call it a day. This pack is about *feel*: instant local triage, then a named handoff when the desk needs a brain with more room. |
|
|
| --- |
|
|
| ## What’s in the pack (and why the drama) |
|
|
| Four files. Four moods. Same EDGE soul. |
|
|
| | File | Precision | Use when | |
| | --- | --- | --- | |
| | `artoru-edge-65m.safetensors` | **FP32** | Flagship PyTorch load + vocab metadata | |
| | `artoru-edge-65m.onnx` | **FP32** | ORT serve / GPU peak throughput (self-contained) | |
| | `artoru-edge-65m-int8.onnx` | **INT8** | Smaller CPU-oriented ORT (~63 MB) | |
| | `artoru-edge-65m-q4.safetensors` | **INT4** | Smallest download (~33 MB); dequant then load | |
|
|
| **FP32** is the headline — quality plus the **~134k tok/s** ORT CUDA story. |
| **Int8** is the everyday carry for lean CPU desks. |
| **Q4** is the suitcase that still somehow holds the whole outfit: a custom RTN int4 pack — dequant before `load_state_dict`. After dequant, compute is FP32-class. |
|
|
| Download the flagship if you want the poster on the wall. |
| Download int8 if your fans already hate you. |
| Download Q4 if you believe small can still be serious. |
|
|
| --- |
|
|
| ## Architecture |
|
|
| ~**65.39M** params · vocab **9225** · up to **2048** positions · **768-d** · **8** heads · **8** layers · intents + skills + slots + risk. |
|
|
| Built for agentic routing inside Desk: register, assign/resume, browse/search/fetch, skills, memory/meeting, calculate/time/weather. |
|
|
| Not a toy transformer cosplaying as a chat model. A routing encoder with a job description carved into the heads. |
|
|
| --- |
|
|
| ## Load — FP32 Safetensors |
|
|
| ```python |
| import json |
| from pathlib import Path |
| import torch |
| from safetensors import safe_open |
| from safetensors.torch import load_file |
| |
| from scripts.train_canonical import build_canonical_encoder |
| from ams.tokenize import tokenize_legacy |
| |
| path = Path("Model/artoru-edge-65m-hf/artoru-edge-65m.safetensors") |
| tensors = load_file(str(path)) |
| with safe_open(str(path), framework="pt") as f: |
| meta = f.metadata() or {} |
| |
| vocab = json.loads(meta["vocab_json"]) |
| canonical = json.loads(meta["canonical_json"]) |
| skills = json.loads(meta["skills_json"]) |
| |
| model = build_canonical_encoder(len(vocab), len(canonical), len(skills), dropout=0.0) |
| model.load_state_dict(tensors, strict=False) |
| model.eval() |
| |
| def encode(text: str, pack: int = 512): |
| content = tokenize_legacy(text) |
| room = pack - 1 |
| if len(content) > room: |
| content = content[-room:] |
| ids = [vocab["<cls>"]] + [vocab.get(tok, 1) for tok in content] |
| return torch.tensor([ids], dtype=torch.long) |
| ``` |
|
|
| --- |
|
|
| ## Load — FP32 ONNX |
|
|
| ```python |
| import onnxruntime as ort |
| |
| sess = ort.InferenceSession( |
| "Model/artoru-edge-65m-hf/artoru-edge-65m.onnx", |
| providers=["CUDAExecutionProvider", "CPUExecutionProvider"], |
| ) |
| ``` |
|
|
| --- |
|
|
| ## Load — INT8 ONNX |
|
|
| ```python |
| import onnxruntime as ort |
| |
| sess = ort.InferenceSession( |
| "Model/artoru-edge-65m-hf/artoru-edge-65m-int8.onnx", |
| providers=["CPUExecutionProvider"], |
| ) |
| ``` |
|
|
| Pad/truncate to the graph width when needed (bench used seq≈42). |
|
|
| --- |
|
|
| ## Load — Q4 |
|
|
| Unpack `*.q4` + `*.scale` from safetensors metadata (`layouts_json`), rebuild FP32 tensors, then `load_state_dict` into the EDGE encoder. After dequant, compute is FP32-class. |
|
|
| --- |
|
|
| ## Roadmap swagger |
|
|
| This is the **1st public preview** of Artoru EDGE on the path to Desk. |
|
|
| **Coming next — proudly:** |
|
|
| - LoRAs that specialize the router without replacing the soul |
| - More EDGE drops as the desk surface deepens |
| - Richer agent lanes, sharper handoffs, louder product feel |
| - The boring-but-holy work: eval harnesses, calibration, thresholds you can actually trust |
|
|
| The brand ladder is just starting. Watch Artoru climb. |
|
|
| We are not shipping “a 65M curiosity.” We are shipping the **first named nerve** of a local multi-agent brand — and inviting you to load it, break it, score it, and tell us where it flinches. |
|
|
| --- |
|
|
| ## Naming |
|
|
| | Name | What it is | |
| | --- | --- | |
| | **Artoru** | The brand line — local / agentic | |
| | **Artoru EDGE 65M** | This model (first EDGE preview weights) | |
| | **Chief** | Handoff layer with EDGE | |
| | **Artoru Desk** | Local-first multi-agent product | |
|
|
| --- |
|
|
| ## Intended use (short and honest) |
|
|
| **Yes:** on-device intent / routing for agentic desks; triage before expensive generation; local-first pipelines that want a fast first decision. |
|
|
| Use EDGE to decide. Use your generative stack to speak. Use Chief when the turn deserves a longer brain. |
|
|
| --- |
|
|
|
|
| **Artoru EDGE 65M — first preview. The ladder begins.** |
