Spaces:
Running
Running
|
Download README.md from agentic-underground/README: direct link, hf CLI and curl.
- Browser
- Download file 14.7 kB
-
https://huggingface.co/spaces/agentic-underground/README/resolve/main/README.md
- Command line
-
hf download hf://spaces/agentic-underground/README/README.md
-
curl -L -o README.md https://huggingface.co/spaces/agentic-underground/README/resolve/main/README.md
14.7 kB
| title: README | |
| emoji: πΈ | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: static | |
| pinned: false | |
| <div align="center"> | |
| # πΈ agentic-underground | |
| **Open tools and research for building software with AI agents** β | |
| disciplined workflows, safety, token efficiency, and reproducible model research. | |
| <a href="https://huggingface.co/collections/agentic-underground/prattle-not-6af7ac5e95e0fc80de563d61"><img alt="Collection: prattle_not" src="https://img.shields.io/badge/collection-prattle__not-6366f1"></a> | |
| <a href="https://agentic-underground.github.io/prattle_not/"><img alt="Live demo in your browser" src="https://img.shields.io/badge/demo-runs%20in%20your%20browser-a855f7"></a> | |
| <a href="https://github.com/agentic-underground"><img alt="Source on GitHub" src="https://img.shields.io/badge/source-GitHub-24292f?logo=github"></a> | |
| <a href="https://huggingface.co/papers/2609.25053"><img alt="Builds on LatentPort by Simon Villani (arXiv 2609.25053)" src="https://img.shields.io/badge/builds%20on-LatentPort%20by%20Simon%20Villani-b31b1b"></a> | |
| </div> | |
| --- | |
| > **β οΈ Security note for every bridge here.** A bridged model does not read the prompt itself, the system | |
| > prompt included, and rules set in a system prompt are broken more often bridged (measured on six pairs). | |
| > Do not rely on system-prompt instructions as guardrails for a bridged model; enforce policy outside it. | |
| ## On the Hub: the prattle_not family | |
| **prattle_not** reproduces [**LatentPort**](https://huggingface.co/papers/2609.25053), Simon Villani's method for | |
| handing a conversation from one language model to another *through its memory*. A small model reads | |
| the prompt; a fitted **bridge** translates its key/value cache into the memory a larger model would | |
| have built; the larger model answers from it β **without reading the prompt itself**, so its first | |
| token arrives sooner. | |
| What we publish here are the **bridges**: the fitted weights that make that handoff work between a | |
| specific pair of models. Everything is gathered in the | |
| **[prattle_not collection β](https://huggingface.co/collections/agentic-underground/prattle-not-6af7ac5e95e0fc80de563d61)** | |
| ### β¨ Featured β [`smollm2-135m-to-360m-bridge`](https://huggingface.co/agentic-underground/smollm2-135m-to-360m-bridge) | |
| SmolLM2-135M reads; SmolLM2-360M, almost three times larger, answers. Two models with different | |
| shapes, bridged with the paper's **one-token handoff** and its learned correction. | |
| | | | | |
| |:--|:--| | |
| | β‘ **Time to first token** | **1.87Γ** sooner (95% CI 1.83β1.91), in WebAssembly, on a CPU | | |
| | πͺ **Bridge tokens** | **1** β the 360M reads one token of the prompt itself | | |
| | π― **Held-out questions** | **6 of 7** correct β as many as the 360M reading everything | | |
| | π **Fidelity (NCR)** | **0.930** β the paper's corrected bridge on its own pair: 0.918 | | |
| βΆοΈ **[Try it in your browser](https://agentic-underground.github.io/prattle_not/)** β both models and the | |
| bridge download once and run locally. Turn the bridge on, ask anything, and watch the counters show | |
| who read what. | |
| ### π The reproduction behind it | |
| Before carrying the method to new models, we reproduced LatentPort's own **4B β 27B** pair on two | |
| RTX 3090s with upstream's timing code and statistic: **1.36Γ** sooner at 128 prompt tokens, rising to | |
| **2.94Γ** at ~2K and **3.39Γ** at ~16K. The full tables, every run, and a step-by-step guide to | |
| repeat them are in the [prattle_not repository](https://github.com/agentic-underground/prattle_not). | |
| ### π The paper's own pair β [`qwen3.5-4b-to-9b-bridge`](https://huggingface.co/agentic-underground/qwen3.5-4b-to-9b-bridge) | |
| Qwen3.5-4B reads; Qwen3.5-9B answers. This is the pair Simon Villani's paper fits, re-fitted | |
| independently with its method. The attention layers' keys and values are mapped, and the 24 | |
| recurrent layers cross as the 4B's own state. | |
| | | | | |
| |:--|:--| | |
| | π **Fidelity (NCR)** | **0.989** with the paper's learned correction (0.974 from the maps alone), our highest yet | | |
| | π§ͺ **Control (wrong donor)** | 0.308, high because the recurrent state carries general context whichever prompt made it | | |
| | π― **Held-out questions** | **7 of 7**, the same as each model alone | | |
| | β‘ **Time to first token** | **1.16Γ** sooner (95% CI 1.16β1.16) on two RTX 3090s at ~256 tokens; ceiling 1.38Γ. Small: the two models share a recurrent core that costs the same in both | | |
| ### 𧬠Qwen3 at a larger size β [`qwen3-4b-to-8b-bridge`](https://huggingface.co/agentic-underground/qwen3-4b-to-8b-bridge) | |
| Qwen3-4B reads; Qwen3-8B answers, across all 36 layers. | |
| | | | | |
| |:--|:--| | |
| | π **Fidelity (NCR)** | **0.890** with the paper's learned correction | | |
| | π§ͺ **Control (wrong donor)** | β0.026 | | |
| | π― **Held-out questions** | **7 of 7**, the same as each model alone | | |
| | β‘ **Time to first token** | **1.12Γ** sooner (95% CI 1.12β1.12) on two RTX 3090s at ~256 tokens; ceiling 1.61Γ | | |
| ### π Fifty-four times larger β [`qwen3-0.6b-to-32b-bridge`](https://huggingface.co/agentic-underground/qwen3-0.6b-to-32b-bridge) | |
| Qwen3-0.6B reads; Qwen3-32B, quantized to Q4K, answers. Published with its miss stated. | |
| | | | | |
| |:--|:--| | |
| | β‘ **Time to first token** | **4.21Γ** sooner (95% CI 4.19β4.23) at ~256 tokens, **7.17Γ** at ~1K, on two RTX 3090s; ceilings 11.60Γ and 14.33Γ | | |
| | π― **Held-out questions** | **6 of 7** β the 32B alone gets 7, so it is **below our answer bar** | | |
| | π **Fidelity (NCR)** | **0.866** with the paper's learned correction (0.794 from the maps alone) | | |
| | π§ͺ **Control (wrong donor)** | 0.197 | | |
| ### πͺ A 13.6Γ gap, at the bar β [`qwen3-0.6b-to-8b-bridge`](https://huggingface.co/agentic-underground/qwen3-0.6b-to-8b-bridge) | |
| Qwen3-0.6B reads; Qwen3-8B, 13.6 times larger, answers. | |
| | | | | |
| |:--|:--| | |
| | β‘ **Time to first token** | **2.82Γ** sooner (95% CI 2.82β2.83) on one RTX 3090 at ~256 tokens; ceiling 6.03Γ | | |
| | π― **Held-out questions** | **7 of 7**, the same as the 8B alone | | |
| | π **Fidelity (NCR)** | **0.803** with the paper's learned correction | | |
| | π§ͺ **Control (wrong donor)** | 0.092 | | |
| ### π A larger gap β [`smollm2-360m-to-1.7b-bridge`](https://huggingface.co/agentic-underground/smollm2-360m-to-1.7b-bridge) | |
| SmolLM2-360M reads; SmolLM2-1.7B, about five times larger and shaped differently, answers from | |
| **one bridge token**. Here the two models differ on the answer test, so it can show a gain. | |
| | | | | |
| |:--|:--| | |
| | π― **Held-out questions** | **7 of 7**, as the 1.7B reading everything (the 360M alone: 6) | | |
| | π **Fidelity (NCR)** | **0.845** with the paper's learned correction (0.611 from the maps alone) | | |
| | π§ͺ **Control (wrong donor)** | β0.266 | | |
| | β‘ **Time to first token** | **1.67Γ** sooner (95% CI 1.66β1.69) on one RTX 3090 at ~256 tokens; ceiling 2.59Γ | | |
| ### π A third family β [`gemma-3-270m-to-1b-bridge`](https://huggingface.co/agentic-underground/gemma-3-270m-to-1b-bridge) | |
| Gemma 3 270M reads; Gemma 3 1B answers, across sliding-window and global attention layers. | |
| | | | | |
| |:--|:--| | |
| | β‘ **Time to first token** | **1.91Γ** sooner (95% CI 1.86β1.97) on one RTX 3090 at ~256 tokens; ceiling 2.79Γ | | |
| | π **Fidelity (NCR)** | **0.870**, but a wrong prompt's memory reaches 0.600 here: an empty Gemma 3 1B is far off, so read the two together | | |
| | π― **Held-out questions** | **6 of 7**, the same as the 270M alone (the 1B: 7) β no answer gain yet | | |
| βοΈ **Gated, and not Apache-2.0:** it is a Gemma Model Derivative, distributed under the | |
| [Gemma Terms of Use](https://ai.google.dev/gemma/terms). | |
| ### π¦ A fourth family β [`llama-3.2-1b-to-3b-bridge`](https://huggingface.co/agentic-underground/llama-3.2-1b-to-3b-bridge) | |
| **Built with Llama.** Llama 3.2 1B reads; Llama 3.2 3B answers, across different head widths (64 | |
| and 128). | |
| | | | | |
| |:--|:--| | |
| | π **Fidelity (NCR)** | **0.965** with the paper's learned correction (0.818 from the maps alone) | | |
| | π§ͺ **Control (wrong donor)** | β0.040 | | |
| | π― **Held-out questions** | **6 of 7**, the same as the 3B alone (the 1B: 7). No answer gain yet | | |
| | β‘ **Time to first token** | **1.70Γ** sooner (95% CI 1.66β1.74) on one RTX 3090 at ~256 tokens; ceiling 2.54Γ | | |
| βοΈ **Gated, and not Apache-2.0:** it is a derivative work of Llama 3.2, distributed under the | |
| [Llama 3.2 Community License](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/blob/main/LICENSE.txt). | |
| ### π Short convolutions β [`lfm2-350m-to-1.2b-bridge`](https://huggingface.co/agentic-underground/lfm2-350m-to-1.2b-bridge) | |
| LFM2-350M reads; LFM2-1.2B answers. Liquid AI's LFM2 mixes short convolutions with attention, and | |
| the two sizes differ in width, so the convolution state cannot be copied: the bridge maps it too. | |
| | | | | |
| |:--|:--| | |
| | β‘ **Time to first token** | **1.88Γ** sooner (95% CI 1.86β1.91) on one RTX 3090 at ~256 tokens; ceiling 2.50Γ | | |
| | π― **Held-out questions** | **7 of 7**, the same as each model alone | | |
| | π **Fidelity (NCR)** | **0.978** (0.982 from the maps alone), but a wrong prompt's memory reaches 0.730: an empty LFM2 is far off, so read the two together | | |
| βοΈ **Gated, and not Apache-2.0:** it is distributed under the base models' | |
| [LFM Open License v1.0](https://huggingface.co/agentic-underground/lfm2-350m-to-1.2b-bridge/blob/main/LICENSE), | |
| which does not license Commercial Use by an entity with USD 10M or more in annual revenue. | |
| ### π Ninety-five times larger, through Mamba-2 β [`granite-4.0-h-350m-to-h-small-bridge`](https://huggingface.co/agentic-underground/granite-4.0-h-350m-to-h-small-bridge) | |
| Granite 4.0 H 350M reads; H Small, a 72-expert 32B quantized to Q4K, answers from translated Mamba-2 | |
| inputs. The paper's correction scores higher but answers 6 of 7, so the uncorrected maps are | |
| published. | |
| | | | | |
| |:--|:--| | |
| | π― **Held-out questions** | **7 of 7**, the same as H Small alone (the 350M alone gets 5) | | |
| | π **Fidelity (NCR)** | **0.767** from the maps alone | | |
| | π§ͺ **Control (wrong donor)** | 0.161 | | |
| | β‘ **Time to first token** | **1.75Γ** sooner (95% CI 1.74β1.75) on two RTX 3090s at ~256 tokens, **1.62Γ** at ~1K; ceilings 7.30Γ and 7.58Γ. H Small rebuilds 36 Mamba-2 states at the handoff | | |
| ### π Gemma 4, two layer shapes β [`gemma-4-e2b-to-31b-bridge`](https://huggingface.co/agentic-underground/gemma-4-e2b-to-31b-bridge) | |
| Gemma 4 E2B reads; Gemma 4 31B, quantized to Q4K, answers. Its 50 sliding layers (16 Γ 256) and 10 | |
| global layers (4 Γ 512) are each fitted in their own shape. Apache-2.0, not gated. | |
| | | | | |
| |:--|:--| | |
| | π― **Held-out questions** | **7 of 7**, the same as the 31B alone | | |
| | π **Fidelity (NCR)** | **0.809** from the maps alone | | |
| | π§ͺ **Control (wrong donor)** | 0.448 | | |
| | β‘ **Time to first token** | **1.97Γ** sooner (95% CI 1.94β2.00) on two RTX 3090s at ~256 tokens; ceiling 5.10Γ. Longer prompts await an attention path that fits a 24 GB card | | |
| ### π Mamba-2, translated β [`granite-4.0-h-1b-to-h-tiny-bridge`](https://huggingface.co/agentic-underground/granite-4.0-h-1b-to-h-tiny-bridge) | |
| Granite 4.0 H 1B reads; H Tiny, a 64-expert mixture, answers. Mamba-2 state cannot be copied between | |
| separately trained sizes, so the bridge translates each recurrent layer's **inputs**, and H Tiny | |
| builds its own state from them. | |
| | | | | |
| |:--|:--| | |
| | π― **Held-out questions** | **7 of 7**, the same as each model alone | | |
| | π **Fidelity (NCR)** | **0.924** with the paper's learned correction (0.916 from the maps alone) | | |
| | π§ͺ **Control (wrong donor)** | β0.175 | | |
| | β‘ **Time to first token** | **2.06Γ** sooner (95% CI 1.95β2.18) on one RTX 3090 at ~256 tokens; ceiling 3.95Γ. Its experts route on the host CPU, so a busy CPU lowers it | | |
| ### π A hybrid model β [`qwen3.5-0.8b-to-2b-bridge`](https://huggingface.co/agentic-underground/qwen3.5-0.8b-to-2b-bridge) | |
| Qwen3.5-0.8B reads; Qwen3.5-2B answers. Qwen3.5 mixes full-attention layers with Gated DeltaNet | |
| layers: the bridge maps the attention layers' keys and values, and the recurrent state crosses as | |
| the 0.8B's own. | |
| | | | | |
| |:--|:--| | |
| | π **Fidelity (NCR)** | **0.917** with the paper's learned correction (0.884 from the maps alone) | | |
| | π§ͺ **Control (wrong donor)** | 0.019 | | |
| | π― **Held-out questions** | **7 of 7**, the same as each model alone | | |
| | β‘ **Time to first token** | **1.36Γ** sooner (95% CI 1.35β1.36) on one RTX 3090 at ~256 tokens; ceiling 1.50Γ | | |
| ### 𧬠A second family β [`qwen3-0.6b-to-1.7b-bridge`](https://huggingface.co/agentic-underground/qwen3-0.6b-to-1.7b-bridge) | |
| Qwen3-0.6B reads; Qwen3-1.7B answers. This is the real test of whether Simon Villani's method | |
| carries beyond one family, fitted with the same tools and measured on held-out questions. | |
| | | | | |
| |:--|:--| | |
| | π **Fidelity (NCR)** | **0.805** with the paper's learned correction (0.645 from the maps alone) | | |
| | π― **Held-out questions** | **7 of 7** correct, as many as the 1.7B reading everything | | |
| | π§ͺ **Control (wrong donor)** | β0.127: the bridge fed another question's memory scores far lower | | |
| | β‘ **Time to first token** | **1.47Γ** sooner (95% CI 1.46β1.48) on one RTX 3090 at ~256 tokens; ceiling 2.03Γ | | |
| Each new pair gets its own repository here and joins the collection once its weights are fitted | |
| and measured. | |
| --- | |
| ## How we publish | |
| - π¬ **Measured** β every number on a card says how it was measured, and the speed-ups can be | |
| measured again on your own hardware with the published runtime. | |
| - π **Pinned** β our pages fetch weights by commit, never by `main`, so a later upload cannot change | |
| what a published result measured. | |
| - π§© **Our own work only** β we host what we fitted. Base models stay with their publishers and are | |
| linked, never mirrored. | |
| - π **Attributed** β prattle_not is a derivative work of [LatentPort](https://latentport.com/) by | |
| Simon Villani ([source](https://github.com/LatentPort/latentport), Apache-2.0). Our bridges are | |
| Apache-2.0, except those fitted from Gemma, which are Gemma Model Derivatives, gated and distributed | |
| under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms), and those fitted from Llama, which | |
| are Built with Llama, gated and distributed under the Llama 3.2 Community License, and those fitted | |
| from LFM2, gated and distributed under the LFM Open License v1.0. | |
| ## Elsewhere | |
| The **source** for everything here, and our other tools for building software with agents (Claude | |
| Code plugins, a static auditor for agent skills, a token-aware scheduler), live on | |
| **[GitHub β agentic-underground](https://github.com/agentic-underground)**. | |
| Questions and corrections are welcome: open a discussion on the model concerned, or an | |
| [issue on prattle_not](https://github.com/agentic-underground/prattle_not/issues). | |