| --- |
| license: apache-2.0 |
| tags: |
| - north-ml |
| - flare-1 |
| - deltanet |
| - hybrid-attention |
| thumbnail: banner.png |
| --- |
| |
| # Fiber Lite — North ML Flare-1 (~1.6B) |
|
|
| The mid Flare-1 tier: Qwen3.5-derived gated **DeltaNet + GQA hybrid** |
| backbone (hidden 2048) with **AgentRegisters** — persistent learned state |
| slots for agent-style memory. This is the full-precision training |
| checkpoint (HF `transformers` format). The production 8-bit Kestrel build |
| with the learned router, Fiber memory trunk, and search pipeline is at |
| [North-ML1/Fiber](https://huggingface.co/North-ML1/Fiber). |
|
|
| ## Benchmarks (measured, same harness — Modal L4, greedy, Nov-2025) |
|
|
| | Model | ARC-C | MMLU | GSM8K | HumanEval | |
| |---|---|---|---|---| |
| | ink (725M) | 63.8% | 32.0% | 30.0% | 37.5% | |
| | **fiber-lite (1.6B)** | 72.9% | 38.0% | 16.0% | 37.5% | |
| | fiber-pro (1.6B) | 80.9% | 40.0% | 14.0% | 48.4% | |
| | Qwen3-0.6B (raw) | 49.2% | 29.0% | 4.0% | 4.7% | |
| | Qwen3-1.7B (raw) | 72.9% | 39.5% | 43.0% | 68.8% | |
| | SmolLM3-3B (raw) | 77.9% | 36.0% | 51.0% | 71.9% | |
|
|
| Subsets: ARC-C n=199, MMLU n=200, GSM8K n=100, HumanEval n=64. Honest read: |
| ties Qwen3-1.7B on ARC-C and MMLU at the same size class; trails on math |
| and code — math reasoning is the known weak spot. |
|
|
| ## Specs |
|
|
| | | | |
| |---|---| |
| | Parameters | ~1.63B (1.41B backbone + 158M Fiber trunk + registers + bridge) | |
| | Layers | 24 | |
| | Hidden | 2048 | |
| | Attention | GQA 8 heads / 2 KV, DeltaNet:GQA = 3:1 | |
| | Context | 262,144 tokens | |
| | Vocab | 65,536 | |
|
|
| ## Interactive chat |
|
|
| ```bash |
| pip install torch transformers safetensors huggingface_hub |
| python3 chat.py --model North-ML1/fiber-lite |
| ``` |
|
|
| `chat.py` (in this repo) auto-downloads the weights, attaches the |
| AgentRegisters, and starts a REPL with optional two-phase thinking. |
|
|
| ## Links |
|
|
| - Architecture report: [Flare-1 paper](https://huggingface.co/spaces/North-ML1/flare-1-paper) |
| - Runtime artifacts: [Ink-1](https://huggingface.co/North-ML1/Ink-1) · [Fiber](https://huggingface.co/North-ML1/Fiber) · [Fiber-pro](https://huggingface.co/North-ML1/Fiber-pro) |
|
|
