--- license: apache-2.0 pipeline_tag: text-generation language: - en tags: - conversational - deltanet - flare - hybrid - north-ml - safetensors - starlight thumbnail: flare.png --- # Flare (preview) Flare is North ML's merged hybrid architecture: a Qwen3.5-derived backbone of mostly gated DeltaNet linear-attention layers with periodic full GQA attention, a stateful agent-memory register after every full-attention layer, a Fiber draft/memory trunk with delta-rule associative memory blocks, a learned bridge between the two widths, and a small search/think router head — all in one `model.safetensors` (arch `flare-1`; legacy artifacts read as `fiber-starlight-1`). ``` token ids │ ├─────────────► shared embed (tied LM head) │ ├─► Starlight backbone ├─► Fiber trunk (158M) │ 24 layers: │ 24 blocks: 21 attn + │ ~18 gated DeltaNet │ 3 DeltaMemory layers │ ~6 full GQA attention │ (delta-rule S[H,D,D]) │ +AgentRegister per attn ────┘ │ (carried mem slots) │ ├─► router head → {p_search, p_think} └─► Era: lossless state snapshots / prefix checkpoints / rollback over attention KV + recurrent S + register mem + Fiber state ``` ## Tiers ``` ──────────────────────┬───────────┬──────────────┬──────────────────────────── API id │ Params │ Backbone │ Extra ──────────────────────┼───────────┼──────────────┼──────────────────────────── ink-1 │ ~725M │ Small (1024h)│ router + retrieval fiber-1 │ ~1.63B │ Pro (2048h)│ router + retrieval fiber-1-pro │ ~1.63B │ Pro (2048h)│ + fact-verify pass ──────────────────────┴───────────┴──────────────┴──────────────────────────── ``` Weights ship 8-bit quantized in a single safetensors file with JSON metadata (prompts, generation config, router thresholds) plus the tokenizer. ## What is actually measured Scores below are the merged artifacts running through the MLX runtime, not the raw backbones. Subsets and settings are noted; treat them as indicative, not official leaderboard numbers. ``` ────────────────────────┬────────┬─────────┬───────────┬────────────────────── Model │ ARC-C │ MMLU │ HumanEval │ GSM8K ────────────────────────┼────────┼─────────┼───────────┼────────────────────── ink-1 (725M) │ 63.6% │ 39.3% │ 18.3% │ 40.3% fiber-1 (1.63B) │ 73.5% │ 48.9% │ 37.2% │ ~11–40%† ────────────────────────┴────────┴─────────┴───────────┴────────────────────── ARC-Challenge 1165q · MMLU 280q subset · HumanEval 164 · GSM8K 300q † GSM8K varied sharply across runs/settings — flagged as unstable, not a typo. ``` Nothing here supports claims of beating much larger frontier models; the point of the architecture is capability per parameter, not absolute SOTA. ## Era Era is the state manager, not a new attention. It checkpoints and restores the exact model state — attention KV, DeltaNet/SSM state, register memory, Fiber KV and DeltaMemory matrices — keyed by token-prefix hashes, so prompts can resume losslessly and speculative branches can roll back exactly. Prefix reuse is off by default pending parity verification. ## Use This is a research preview. The merged format loads with the Kestrel MLX runtime (`runtime/kestrel`), not stock transformers: ```python from kestrel import load m = load("model.safetensors") for ev in m.run("How tall is Mount Everest?", search="auto"): ... # stream of {type: thinking|text|agent} events ``` ## 1M proof-of-concept `experiments/flare1m/flare_mini.py` trains a ~1M-parameter byte-level Flare (gated delta layers, registers, DeltaMemory, carried state) against a parameter-matched vanilla transformer on the same data and steps. First-run results (1500 steps, RTX 5060 Ti): ``` ──────────────────────────┬───────────┬───────────── metric │ Flare-1M │ GPT (981k) ──────────────────────────┼───────────┼───────────── params │ 965,908 │ 981,312 heldout CE │ 0.0004 │ 0.0003 passkey @0.1/0.5/0.9 │ 0/.08/.08 │ 0/0/.08 passkey stream-512 │ .08/0/0 │ 0/0/0 associative recall │ .146 │ .104 train time │ 576s │ 36s ──────────────────────────┴───────────┴───────────── ``` Read honestly: **inconclusive-to-negative**. Both models sit near chance — the passkey docs (2600 tokens) were 7× beyond the training window (T=384), so neither could reach the planted fact; the recall edge (0.146 vs 0.104, chance 0.111) is within noise at n=48; and the delta-rule scan cost 16× more wall time per step. The 1M demo does *not* prove the mechanisms beat a plain transformer — a fair test needs training at longer context so the recall task is within reach. ## Limitations - Small models: expect hallucination, weak multi-hop reasoning, and shaky math relative to frontier models. Retrieval grounding helps but does not verify. - English only; 64k vocabulary; 8k context in current runtime. - The search/think router is a linear probe on top of the embeddings — it is calibrated, not intelligent. - Preview: the artifact format and API surface may change. --- *Part of [North ML](https://huggingface.co/North-ML1) - open weights, honest numbers.*