Sentence Similarity
sentence-transformers
context-curation
long-context
memory-management
conversational-ai
retrieval
llm
research-paper
Instructions to use SkyIsNotGreen/splinter-memory with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use SkyIsNotGreen/splinter-memory with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("SkyIsNotGreen/splinter-memory") sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
hive-dev commited on
Commit ·
970b4e6
1
Parent(s): f8589c0
Update the card to the Cascade-edition white paper
Browse filesAdd the Cascade-style title/deck, opening, contents, one-page claim table and three headline results. Rename the sections, reorder the encoder subsections, add explicit limits/what-is-not-claimed and a separate Reproduce section, sync the canonical-paper and repository links, and correct the 35B verdict-set and 93.9% replay recall statements.
- CITATION.cff +2 -2
- README.md +157 -85
CITATION.cff
CHANGED
|
@@ -15,8 +15,8 @@ authors:
|
|
| 15 |
version: "0.1.0"
|
| 16 |
date-released: "2026-09-22"
|
| 17 |
license: MIT
|
| 18 |
-
repository-code: "https://github.com/sky-is-green/
|
| 19 |
-
url: "https://github.com/sky-is-green/
|
| 20 |
keywords:
|
| 21 |
- context-management
|
| 22 |
- memory-management
|
|
|
|
| 15 |
version: "0.1.0"
|
| 16 |
date-released: "2026-09-22"
|
| 17 |
license: MIT
|
| 18 |
+
repository-code: "https://github.com/sky-is-green/hive"
|
| 19 |
+
url: "https://github.com/sky-is-green/hive"
|
| 20 |
keywords:
|
| 21 |
- context-management
|
| 22 |
- memory-management
|
README.md
CHANGED
|
@@ -12,23 +12,75 @@ tags:
|
|
| 12 |
pipeline_tag: sentence-similarity
|
| 13 |
---
|
| 14 |
|
| 15 |
-
#
|
| 16 |
-
|
| 17 |
-
**
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
A long conversation is not a long document. Facts that mattered twenty turns
|
| 34 |
ago fall out of a rolling context window; the model answers from what
|
|
@@ -38,7 +90,7 @@ exactly the old fact the user came back for. Keeping everything is worse -
|
|
| 38 |
the prompt grows, the KV cache grows, the cost grows, and the model
|
| 39 |
"loses the middle" while the interesting fact sits somewhere in the middle.
|
| 40 |
|
| 41 |
-
## What
|
| 42 |
|
| 43 |
Splinter Memory keeps a **managed** memory outside the model and hands the
|
| 44 |
model a **bounded, relevance-ranked** context every turn:
|
|
@@ -65,23 +117,23 @@ answers and the wrong tool for selecting context; measured live, using the
|
|
| 65 |
the hot path, while the 17M-parameter drone did the same job in
|
| 66 |
milliseconds.
|
| 67 |
|
| 68 |
-
## The measured results
|
| 69 |
|
| 70 |
| Metric | Naive FIFO (baseline) | Target | Measured |
|
| 71 |
|---|---|---|---|
|
| 72 |
| Throughput over 500 turns | decays | flat ±10% | **flat** (14.5 → 15.5 tok/s over 308+ turns), 0 OOM events |
|
| 73 |
-
| Retrieval recall (facts the model actually stated) | n/a | ≥75% / ≥90% | **90.3%** live, 93.
|
| 74 |
-
| Retrieval precision (sentence proxy) | near-chance past the window | ≥70% / ≥85% | **10.7%** - *the encoder ceiling* (§
|
| 75 |
| Context sufficiency vs. an equal-size FIFO window | — | ≥80% of turns | **85.1%** of the fact-retrievable turns (4.4:1) |
|
| 76 |
| Routing accuracy | — | ≥80% / ≥92% | **100%** |
|
| 77 |
-
| Added per-turn latency | 0 ms | <50 / <30 ms | **≈18 ms** on a small store (3.4 ms assembly + ~15 ms scoring) - **and it grows with the store**: 22.8 ms at 212 chunks, 156.8 ms at 1,612 (§
|
| 78 |
| Peak RSS over 500 turns | — | small | **34.7 MB** |
|
| 79 |
| Pipeline Efficiency Score | ~30 | ≥65 / ≥80 | **80.0 (GREEN)** live, 73.1 on the 20-conversation evidence run |
|
| 80 |
|
| 81 |
Full protocol, per-hypothesis logic chains and falsification conditions:
|
| 82 |
§5 of the canonical paper.
|
| 83 |
|
| 84 |
-
### The whole protocol, refreshed on the 35B-A3B
|
| 85 |
|
| 86 |
The P1-P11 protocol re-run live against the **35B-A3B ternary** (a
|
| 87 |
12-conversation fixture subset), to ask whether the verdict set is a
|
|
@@ -90,7 +142,7 @@ property of the architecture or of the generator it was measured on:
|
|
| 90 |
| | verdict | the evidence |
|
| 91 |
|---|---|---|
|
| 92 |
| P1 constant throughput | **PASS** | the throughput does not decay |
|
| 93 |
-
| P2 retrieval precision | **FAIL** | the precision clause (the known encoder ceiling, §
|
| 94 |
| P3 context sufficiency | **PASS** | the curated context against an equal-size FIFO window |
|
| 95 |
| P4 domain-dependent decay | **PASS** | code m90 1.8 vs prose m90 1.2 - a gap of 0.6, beyond the 0.2 band |
|
| 96 |
| P5 targeted masking | SKIP | a training experiment |
|
|
@@ -101,29 +153,29 @@ property of the architecture or of the generator it was measured on:
|
|
| 101 |
| P10 drift reset | **FAIL** | the reset changes the recall by 0.0 points |
|
| 102 |
| P11 comb resurrection | **PASS** | the pressure regime: 100% topic-return recall against 20% without the comb |
|
| 103 |
|
| 104 |
-
**PASS 6/11, 2 skipped
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
|
| 110 |
**One of the failures now has a candidate mechanism.** P10 (the drift
|
| 111 |
reset) measures a detector whose threshold was calibrated for a different
|
| 112 |
-
encoder (§
|
| 113 |
effect" may be measuring a detector that fires far too often rather than
|
| 114 |
a reset that does not work. (The exact rate was measured on the
|
| 115 |
-
pre-correction corpus and is withdrawn with the rest of §
|
| 116 |
stands as measured; the mechanism is now in question, which is a
|
| 117 |
different claim.
|
| 118 |
|
| 119 |
**And the one result that does not carry over is P3's margin.** On the
|
| 120 |
27B the curated context won 115 turns outright to 26; on the 35B it is a
|
| 121 |
-
tie (§
|
| 122 |
at least as good on 88.4% of the turns); the *margin* does not. The
|
| 123 |
curation's accuracy advantage is a function of the generator - what
|
| 124 |
survives for a strong one is the bounded context and the cost.
|
| 125 |
|
| 126 |
-
## Use it
|
| 127 |
|
| 128 |
Three ways, in increasing order of control:
|
| 129 |
|
|
@@ -154,7 +206,7 @@ print(result.assembled.content) # the curated context, bounded
|
|
| 154 |
diagnostics and the self-contained run bundles are in the repository, so the
|
| 155 |
claims can be re-measured on your own hardware and your own backend.
|
| 156 |
|
| 157 |
-
## How it works
|
| 158 |
|
| 159 |
| Layer | Function | Core mechanisms |
|
| 160 |
|---|---|---|
|
|
@@ -169,7 +221,7 @@ chunk → Membrane deduplicates and checks the topic → Retention applies decay
|
|
| 169 |
and the remembrance pass → Focal assembles a bounded context → the backend
|
| 170 |
generates → the reply is observed back into the store.
|
| 171 |
|
| 172 |
-
### The four postulates
|
| 173 |
|
| 174 |
1. **Context curation** - for a fixed token budget, a relevance-ranked
|
| 175 |
selection from the full history dominates a contiguous window of the same
|
|
@@ -184,12 +236,12 @@ generates → the reply is observed back into the store.
|
|
| 184 |
the time, which is enough to drive parameter tuning without a human
|
| 185 |
labelling pass.
|
| 186 |
|
| 187 |
-
## The harness
|
| 188 |
|
| 189 |
The memory system is a library; the **HiveBench Studio** is the harness
|
| 190 |
that runs it, and it is where the management pipeline lives.
|
| 191 |
|
| 192 |
-
### The management pipeline
|
| 193 |
|
| 194 |
A long-horizon memory has to be *managed*: every knob that decides what
|
| 195 |
the model sees used to live in three different menus, and a wrong setting
|
|
@@ -209,7 +261,7 @@ The point is not tidiness. It is that **a setting which is not consulted
|
|
| 209 |
is indistinguishable from a setting which does not work**, and the only
|
| 210 |
way to know which one you have is to wire them and then measure.
|
| 211 |
|
| 212 |
-
### The harness itself
|
| 213 |
|
| 214 |
| | |
|
| 215 |
|---|---|
|
|
@@ -219,7 +271,7 @@ way to know which one you have is to wire them and then measure.
|
|
| 219 |
| the cascade console | every runner knob in one form, with the Global tab's Quality block as its starting values |
|
| 220 |
| the harvest | every turn (the query, the reply, the curation that was used) written to a jsonl, off by default - this is the data the encoder is trained on |
|
| 221 |
|
| 222 |
-
### Answer quality reporting and API escalation
|
| 223 |
|
| 224 |
The curation's effect is not only felt, it is reported. Every cascade run
|
| 225 |
writes a per-item report (the verdict, the detector findings, the repair
|
|
@@ -237,13 +289,13 @@ answer, the cost and the reason recorded. The companion paper
|
|
| 237 |
and the measured verdicts; the benchmark and the run reports are
|
| 238 |
published next to this card.
|
| 239 |
|
| 240 |
-
## The evidence, hypothesis by hypothesis
|
| 241 |
|
| 242 |
| | Hypothesis | Verdict |
|
| 243 |
|---|---|---|
|
| 244 |
| P1 | Constant throughput over 500 turns | **PASS** |
|
| 245 |
| P2 | Retrieval precision ≥85% at ≥90% recall | **SPLIT** - recall PASS (90.3%), precision FAIL (the encoder ceiling) |
|
| 246 |
-
| P3 | Context sufficiency ≥80% of turns vs. an equal-size FIFO window | **PASS** (85.1% on the 27B); on the 35B-A3B the answer-level result is a tie (88.4% at-least-as-good, 13 strict wins to 17) - see §
|
| 247 |
| P4 | Domain-dependent decay curve (code vs. prose separate) | **PASS** (code m90 1.8 vs. prose 1.2) |
|
| 248 |
| P5 | Targeted masking | **PASS** |
|
| 249 |
| P6 | Confidence escalation | **FAIL** |
|
|
@@ -257,7 +309,7 @@ published next to this card.
|
|
| 257 |
The failures are kept in the paper with their evidence. A pre-registered
|
| 258 |
protocol that only reports passes is not evidence.
|
| 259 |
|
| 260 |
-
##
|
| 261 |
|
| 262 |
> **Correction (2026-10-21).** The corpus behind this section included an
|
| 263 |
> ingested knowledge-base attachment: 17 of its 82 files were chunked
|
|
@@ -285,7 +337,7 @@ all landed on the same top-K curve - so it concluded the ceiling was
|
|
| 285 |
*data-structural* and that the remedy was "stop looking for a better
|
| 286 |
encoder".
|
| 287 |
|
| 288 |
-
|
| 289 |
conversations, 6,975 stored chunks, 4,056 duplicate pairs and 6,533
|
| 290 |
topic-change cases, labelled lexically so the label is independent of the
|
| 291 |
encoder being measured (`experiments/encoder_suite.py`). The result
|
|
@@ -303,7 +355,7 @@ not re-derived.
|
|
| 303 |
| drift at the shipped threshold (0.6) | precision 0.08, **recall 0.96** - the topic-change test fires on almost every turn |
|
| 304 |
| drift as a *ranking* signal | **withdrawn** - measured on the pre-correction corpus (79% an ingested attachment), see above |
|
| 305 |
|
| 306 |
-
###
|
| 307 |
|
| 308 |
The obvious conclusion - "move the thresholds to the encoder's optimum" -
|
| 309 |
is **falsified** by a deterministic replay of the corpus through the real
|
|
@@ -344,32 +396,20 @@ and the fact-preservation constraint fixes where on it the system sits.
|
|
| 344 |
The clean replay and the three failed fine-tunes are in `AB-RESULTS.md`
|
| 345 |
§2C-quater through §2C-septies.
|
| 346 |
|
| 347 |
-
###
|
| 348 |
-
|
| 349 |
-
The stability run - 2,000 turns of a growing conversation, the real drone,
|
| 350 |
-
no model (`experiments.stability_500.py`):
|
| 351 |
-
|
| 352 |
-
| turns | the store | the assembled context | the curation's cost (p50) | the peak RSS |
|
| 353 |
-
|---|---|---|---|---|
|
| 354 |
-
| 250 | 212 chunks | 322 tok | 22.8 ms | 519 MB |
|
| 355 |
-
| 500 | 412 chunks | 320 tok | 37.3 ms | 564 MB |
|
| 356 |
-
| 1,000 | 812 chunks | 335 tok | 89.7 ms | 594 MB |
|
| 357 |
-
| 2,000 | 1,612 chunks | 332 tok | 156.8 ms | 667 MB |
|
| 358 |
|
| 359 |
-
|
| 360 |
-
|
| 361 |
-
|
| 362 |
-
|
| 363 |
-
|
| 364 |
-
|
| 365 |
-
|
| 366 |
-
|
| 367 |
-
|
| 368 |
-
|
| 369 |
-
crossover - the point where curating costs more than the tokens it saves
|
| 370 |
-
- is a measurement the system does not yet make.
|
| 371 |
|
| 372 |
-
###
|
| 373 |
|
| 374 |
The paired A/B - the same model, the same turns, the curated context
|
| 375 |
against the last-4k-token window of the same history - is the memory
|
|
@@ -403,20 +443,33 @@ is 4k tokens of pure recency, and the two arms are equal *at that
|
|
| 403 |
budget*. The honest claim for a strong generator is the bounded context
|
| 404 |
and the cost, not the better answers.
|
| 405 |
|
| 406 |
-
###
|
| 407 |
|
| 408 |
-
|
| 409 |
-
|
| 410 |
-
|
| 411 |
-
|
| 412 |
-
|
| 413 |
-
|
| 414 |
-
|
| 415 |
-
|
| 416 |
-
|
| 417 |
-
`docs/ENCODER-NOTES.md` in [`splinter/`](https://github.com/sky-is-green/hive/tree/main/splinter).
|
| 418 |
|
| 419 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 420 |
|
| 421 |
- **Precision is the open question.** The context that reaches the model is
|
| 422 |
fatter than it needs to be. The recall evidence says the facts the model
|
|
@@ -426,7 +479,7 @@ and the cost, not the better answers.
|
|
| 426 |
circularity but does not remove it: the auditor is the same model family
|
| 427 |
that produced the answer.
|
| 428 |
- **Synthetic-corpus bias.** The long-horizon corpus is generated. The
|
| 429 |
-
measurement in §
|
| 430 |
- **Cross-conversation contamination and hedge-replies** are harness-level
|
| 431 |
failure modes that look like architectural failures if you do not
|
| 432 |
isolate conversations per store and filter refusals out of the store.
|
|
@@ -447,7 +500,26 @@ and the cost, not the better answers.
|
|
| 447 |
keeps what the VRAM cache cannot hold in ordinary RAM, and the storage
|
| 448 |
read on the second request falls from 1.4 GB to 146 MB.
|
| 449 |
|
| 450 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 451 |
|
| 452 |
```bash
|
| 453 |
# the offline suite (deterministic diagnostics, no model needed)
|
|
@@ -466,7 +538,7 @@ open http://127.0.0.1:8765/cascade
|
|
| 466 |
The repository ships the run bundles, the event logs and the label files, so
|
| 467 |
a verdict can be traced back to the bytes that produced it.
|
| 468 |
|
| 469 |
-
### The artifacts
|
| 470 |
|
| 471 |
| artifact | what it is |
|
| 472 |
|---|---|
|
|
@@ -479,7 +551,7 @@ a verdict can be traced back to the bytes that produced it.
|
|
| 479 |
The releases are private while the results are being refreshed; they open
|
| 480 |
when the refreshed numbers are in.
|
| 481 |
|
| 482 |
-
## License and
|
| 483 |
|
| 484 |
MIT. See `LICENSE` and `CITATION.cff` in this repository.
|
| 485 |
|
|
@@ -489,7 +561,7 @@ MIT. See `LICENSE` and `CITATION.cff` in this repository.
|
|
| 489 |
for Long-Horizon LLM Conversations},
|
| 490 |
author = {sky-is-green},
|
| 491 |
year = {2026},
|
| 492 |
-
url = {https://github.com/sky-is-green/hive
|
| 493 |
}
|
| 494 |
```
|
| 495 |
|
|
|
|
| 12 |
pipeline_tag: sentence-similarity
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# Splinter Memory: A Managed-Decay Context Curation Architecture for Long-Horizon LLM Conversations
|
| 16 |
+
|
| 17 |
+
**A managed-decay context curation layer for long-horizon LLM conversations, with falsifiable predictions and the failures kept.**
|
| 18 |
+
|
| 19 |
+
**sky-is-green** · Hugging Face edition · November 2026 \
|
| 20 |
+
Canonical paper: [hive/splinter/SPLINTER-WHITE-PAPER.md](https://github.com/sky-is-green/hive/blob/hive_test/splinter/SPLINTER-WHITE-PAPER.md) · Code: [github.com/sky-is-green/hive](https://github.com/sky-is-green/hive)
|
| 21 |
+
|
| 22 |
+
A model that has forgotten a fact is rarely uncertain. It answers from
|
| 23 |
+
whatever is still in the window with the same confidence as a model that
|
| 24 |
+
remembers. Every long conversation inherits this: the window fills, the
|
| 25 |
+
oldest decisions fall out, and the model keeps talking.
|
| 26 |
+
|
| 27 |
+
Splinter Memory is a context-curation layer outside the generative model. A
|
| 28 |
+
small fleet of bidirectional encoders scores the stored conversation; a
|
| 29 |
+
membrane deduplicates and detects topic drift; a retention pass applies
|
| 30 |
+
managed decay; and a focal assembler builds a bounded, relevance-ranked
|
| 31 |
+
window. The primary model sees only what matters, at a fixed budget, every
|
| 32 |
+
turn.
|
| 33 |
+
|
| 34 |
+
This is the Hugging Face edition of the paper. The canonical paper lives with
|
| 35 |
+
the code; this edition adds the encoder measurement taken on the harness's own
|
| 36 |
+
traffic (§8) and the operational notes a practitioner needs. The test corpora
|
| 37 |
+
ship with the repository; the live run bundles are recorded (R), not
|
| 38 |
+
published.
|
| 39 |
+
|
| 40 |
+
**Contents**
|
| 41 |
+
|
| 42 |
+
- [The claim in one page](#the-claim-in-one-page)
|
| 43 |
+
- [1. The problem](#1-the-problem)
|
| 44 |
+
- [2. What it does](#2-what-it-does)
|
| 45 |
+
- [3. The measured results](#3-the-measured-results)
|
| 46 |
+
- [4. Use it](#4-use-it)
|
| 47 |
+
- [5. How it works](#5-how-it-works)
|
| 48 |
+
- [6. The harness and pipeline](#6-the-harness-and-pipeline)
|
| 49 |
+
- [7. The evidence, hypothesis by hypothesis](#7-the-evidence-hypothesis-by-hypothesis)
|
| 50 |
+
- [8. The encoder: measured on live traffic](#8-the-encoder-measured-on-live-traffic)
|
| 51 |
+
- [9. Limits and what is not claimed](#9-limits-and-what-is-not-claimed)
|
| 52 |
+
- [10. Reproduce](#10-reproduce)
|
| 53 |
+
- [11. License and community invitation](#11-license-and-community-invitation)
|
| 54 |
+
|
| 55 |
+
## The claim in one page
|
| 56 |
+
|
| 57 |
+
| the claim | the measurement |
|
| 58 |
+
|---|---|
|
| 59 |
+
| Bounded context keeps throughput flat | decode 14.5 → 15.5 tps over 308+ turns (+6.7%, P1 PASS) |
|
| 60 |
+
| Curation beats FIFO | 85.1% of fact-retrievable turns on the 27B (P3 PASS); 88.4% at-least-as-good on the 35B-A3B, a strict-win tie |
|
| 61 |
+
| Recall is real; precision is the ceiling | 90.3% deterministic recall (P2 recall PASS), 10.7% precision (P2 precision FAIL, encoder ceiling) |
|
| 62 |
+
| The score | post-run PES 80.0 GREEN vs rolling 12.2 / FIFO 11.6 |
|
| 63 |
+
| The encoder thresholds are the trade | dedup at 0.92: precision 1.00, recall 0.03; a tuned encoder halves fact preservation (32.66% vs 60.51%) |
|
| 64 |
+
| The context is flat; the curation cost is not | 322 → 332 tokens while the store grows 7.6×; 22.8 ms at 212 chunks → 156.8 ms at 1,612 |
|
| 65 |
+
|
| 66 |
+
Three results carry the paper:
|
| 67 |
+
|
| 68 |
+
1. **Bounded context is the lever.** The window stays fixed while the store
|
| 69 |
+
grows, decode throughput stays flat (P1), and facts survive when history
|
| 70 |
+
contains them (P2 recall, P3).
|
| 71 |
+
2. **Selection is the product; precision is the ceiling.** The small encoder
|
| 72 |
+
fleet and deterministic membrane beat FIFO (P3) and keep the densest
|
| 73 |
+
duplicate (P9). P2 precision, P6 and P10 fail; the failures and the
|
| 74 |
+
threshold trade are kept (§8).
|
| 75 |
+
3. **The curation's answer advantage is generator-dependent.** On the 27B it
|
| 76 |
+
won 115 turns to 26; on the 35B-A3B it is a tie. What survives for a strong
|
| 77 |
+
generator is the bounded context, the fidelity and the cost (§8.3).
|
| 78 |
+
|
| 79 |
+
What is not claimed: that retrieval precision is solved, that curated context
|
| 80 |
+
always wins on answer quality, that P6/P10 work, or that the auditor is
|
| 81 |
+
independent. The evidence and caveats are in §9.
|
| 82 |
+
|
| 83 |
+
## 1. The problem
|
| 84 |
|
| 85 |
A long conversation is not a long document. Facts that mattered twenty turns
|
| 86 |
ago fall out of a rolling context window; the model answers from what
|
|
|
|
| 90 |
the prompt grows, the KV cache grows, the cost grows, and the model
|
| 91 |
"loses the middle" while the interesting fact sits somewhere in the middle.
|
| 92 |
|
| 93 |
+
## 2. What it does
|
| 94 |
|
| 95 |
Splinter Memory keeps a **managed** memory outside the model and hands the
|
| 96 |
model a **bounded, relevance-ranked** context every turn:
|
|
|
|
| 117 |
the hot path, while the 17M-parameter drone did the same job in
|
| 118 |
milliseconds.
|
| 119 |
|
| 120 |
+
## 3. The measured results
|
| 121 |
|
| 122 |
| Metric | Naive FIFO (baseline) | Target | Measured |
|
| 123 |
|---|---|---|---|
|
| 124 |
| Throughput over 500 turns | decays | flat ±10% | **flat** (14.5 → 15.5 tok/s over 308+ turns), 0 OOM events |
|
| 125 |
+
| Retrieval recall (facts the model actually stated) | n/a | ≥75% / ≥90% | **90.3%** live, 93.9% on the isolated-store replay |
|
| 126 |
+
| Retrieval precision (sentence proxy) | near-chance past the window | ≥70% / ≥85% | **10.7%** - *the encoder ceiling* (§8) |
|
| 127 |
| Context sufficiency vs. an equal-size FIFO window | — | ≥80% of turns | **85.1%** of the fact-retrievable turns (4.4:1) |
|
| 128 |
| Routing accuracy | — | ≥80% / ≥92% | **100%** |
|
| 129 |
+
| Added per-turn latency | 0 ms | <50 / <30 ms | **≈18 ms** on a small store (3.4 ms assembly + ~15 ms scoring) - **and it grows with the store**: 22.8 ms at 212 chunks, 156.8 ms at 1,612 (§8.4) |
|
| 130 |
| Peak RSS over 500 turns | — | small | **34.7 MB** |
|
| 131 |
| Pipeline Efficiency Score | ~30 | ≥65 / ≥80 | **80.0 (GREEN)** live, 73.1 on the 20-conversation evidence run |
|
| 132 |
|
| 133 |
Full protocol, per-hypothesis logic chains and falsification conditions:
|
| 134 |
§5 of the canonical paper.
|
| 135 |
|
| 136 |
+
### 3.1 The whole protocol, refreshed on the 35B-A3B
|
| 137 |
|
| 138 |
The P1-P11 protocol re-run live against the **35B-A3B ternary** (a
|
| 139 |
12-conversation fixture subset), to ask whether the verdict set is a
|
|
|
|
| 142 |
| | verdict | the evidence |
|
| 143 |
|---|---|---|
|
| 144 |
| P1 constant throughput | **PASS** | the throughput does not decay |
|
| 145 |
+
| P2 retrieval precision | **FAIL** | the precision clause (the known encoder ceiling, §8) |
|
| 146 |
| P3 context sufficiency | **PASS** | the curated context against an equal-size FIFO window |
|
| 147 |
| P4 domain-dependent decay | **PASS** | code m90 1.8 vs prose m90 1.2 - a gap of 0.6, beyond the 0.2 band |
|
| 148 |
| P5 targeted masking | SKIP | a training experiment |
|
|
|
|
| 153 |
| P10 drift reset | **FAIL** | the reset changes the recall by 0.0 points |
|
| 154 |
| P11 comb resurrection | **PASS** | the pressure regime: 100% topic-return recall against 20% without the comb |
|
| 155 |
|
| 156 |
+
**PASS 6/11, 2 skipped in the 35B re-run.** The 35B run reproduced
|
| 157 |
+
P1/P3/P4/P8/P9/P11 PASS and P2/P6/P10 FAIL; P5/P7 were not re-run there.
|
| 158 |
+
The canonical paper reports P5/P7 PASS from their separate measurements.
|
| 159 |
+
The reproduced subset is the point: the verdict set is a property of the
|
| 160 |
+
architecture, not of the generator.
|
| 161 |
|
| 162 |
**One of the failures now has a candidate mechanism.** P10 (the drift
|
| 163 |
reset) measures a detector whose threshold was calibrated for a different
|
| 164 |
+
encoder (§8): forced on, it fires on most turns, so "the reset has no
|
| 165 |
effect" may be measuring a detector that fires far too often rather than
|
| 166 |
a reset that does not work. (The exact rate was measured on the
|
| 167 |
+
pre-correction corpus and is withdrawn with the rest of §8.) The verdict
|
| 168 |
stands as measured; the mechanism is now in question, which is a
|
| 169 |
different claim.
|
| 170 |
|
| 171 |
**And the one result that does not carry over is P3's margin.** On the
|
| 172 |
27B the curated context won 115 turns outright to 26; on the 35B it is a
|
| 173 |
+
tie (§8.3). The *verdict* holds (the curated context is sufficient, and
|
| 174 |
at least as good on 88.4% of the turns); the *margin* does not. The
|
| 175 |
curation's accuracy advantage is a function of the generator - what
|
| 176 |
survives for a strong one is the bounded context and the cost.
|
| 177 |
|
| 178 |
+
## 4. Use it
|
| 179 |
|
| 180 |
Three ways, in increasing order of control:
|
| 181 |
|
|
|
|
| 206 |
diagnostics and the self-contained run bundles are in the repository, so the
|
| 207 |
claims can be re-measured on your own hardware and your own backend.
|
| 208 |
|
| 209 |
+
## 5. How it works
|
| 210 |
|
| 211 |
| Layer | Function | Core mechanisms |
|
| 212 |
|---|---|---|
|
|
|
|
| 221 |
and the remembrance pass → Focal assembles a bounded context → the backend
|
| 222 |
generates → the reply is observed back into the store.
|
| 223 |
|
| 224 |
+
### 5.1 The four postulates
|
| 225 |
|
| 226 |
1. **Context curation** - for a fixed token budget, a relevance-ranked
|
| 227 |
selection from the full history dominates a contiguous window of the same
|
|
|
|
| 236 |
the time, which is enough to drive parameter tuning without a human
|
| 237 |
labelling pass.
|
| 238 |
|
| 239 |
+
## 6. The harness and pipeline
|
| 240 |
|
| 241 |
The memory system is a library; the **HiveBench Studio** is the harness
|
| 242 |
that runs it, and it is where the management pipeline lives.
|
| 243 |
|
| 244 |
+
### 6.1 The management pipeline
|
| 245 |
|
| 246 |
A long-horizon memory has to be *managed*: every knob that decides what
|
| 247 |
the model sees used to live in three different menus, and a wrong setting
|
|
|
|
| 261 |
is indistinguishable from a setting which does not work**, and the only
|
| 262 |
way to know which one you have is to wire them and then measure.
|
| 263 |
|
| 264 |
+
### 6.2 The harness itself
|
| 265 |
|
| 266 |
| | |
|
| 267 |
|---|---|
|
|
|
|
| 271 |
| the cascade console | every runner knob in one form, with the Global tab's Quality block as its starting values |
|
| 272 |
| the harvest | every turn (the query, the reply, the curation that was used) written to a jsonl, off by default - this is the data the encoder is trained on |
|
| 273 |
|
| 274 |
+
### 6.3 Answer quality reporting and API escalation
|
| 275 |
|
| 276 |
The curation's effect is not only felt, it is reported. Every cascade run
|
| 277 |
writes a per-item report (the verdict, the detector findings, the repair
|
|
|
|
| 289 |
and the measured verdicts; the benchmark and the run reports are
|
| 290 |
published next to this card.
|
| 291 |
|
| 292 |
+
## 7. The evidence, hypothesis by hypothesis
|
| 293 |
|
| 294 |
| | Hypothesis | Verdict |
|
| 295 |
|---|---|---|
|
| 296 |
| P1 | Constant throughput over 500 turns | **PASS** |
|
| 297 |
| P2 | Retrieval precision ≥85% at ≥90% recall | **SPLIT** - recall PASS (90.3%), precision FAIL (the encoder ceiling) |
|
| 298 |
+
| P3 | Context sufficiency ≥80% of turns vs. an equal-size FIFO window | **PASS** (85.1% on the 27B); on the 35B-A3B the answer-level result is a tie (88.4% at-least-as-good, 13 strict wins to 17) - see §8.3 |
|
| 299 |
| P4 | Domain-dependent decay curve (code vs. prose separate) | **PASS** (code m90 1.8 vs. prose 1.2) |
|
| 300 |
| P5 | Targeted masking | **PASS** |
|
| 301 |
| P6 | Confidence escalation | **FAIL** |
|
|
|
|
| 309 |
The failures are kept in the paper with their evidence. A pre-registered
|
| 310 |
protocol that only reports passes is not evidence.
|
| 311 |
|
| 312 |
+
## 8. The encoder: measured on live traffic
|
| 313 |
|
| 314 |
> **Correction (2026-10-21).** The corpus behind this section included an
|
| 315 |
> ingested knowledge-base attachment: 17 of its 82 files were chunked
|
|
|
|
| 337 |
*data-structural* and that the remedy was "stop looking for a better
|
| 338 |
encoder".
|
| 339 |
|
| 340 |
+
I re-measured all three jobs on the harness's **own** traffic - 81
|
| 341 |
conversations, 6,975 stored chunks, 4,056 duplicate pairs and 6,533
|
| 342 |
topic-change cases, labelled lexically so the label is independent of the
|
| 343 |
encoder being measured (`experiments/encoder_suite.py`). The result
|
|
|
|
| 355 |
| drift at the shipped threshold (0.6) | precision 0.08, **recall 0.96** - the topic-change test fires on almost every turn |
|
| 356 |
| drift as a *ranking* signal | **withdrawn** - measured on the pre-correction corpus (79% an ingested attachment), see above |
|
| 357 |
|
| 358 |
+
### 8.1 The F1 is the wrong target, and the threshold is not the lever
|
| 359 |
|
| 360 |
The obvious conclusion - "move the thresholds to the encoder's optimum" -
|
| 361 |
is **falsified** by a deterministic replay of the corpus through the real
|
|
|
|
| 396 |
The clean replay and the three failed fine-tunes are in `AB-RESULTS.md`
|
| 397 |
§2C-quater through §2C-septies.
|
| 398 |
|
| 399 |
+
### 8.2 What did not work, and what shipped
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 400 |
|
| 401 |
+
- The shipped configuration stays: the stock encoder at the shipped
|
| 402 |
+
thresholds, because that is the configuration the published evidence was
|
| 403 |
+
produced with.
|
| 404 |
+
- The next fine-tune weights a false merge above a missed merge
|
| 405 |
+
(precision-weighted at the operating point) instead of optimising F1 -
|
| 406 |
+
the asymmetry makes F1 the wrong target - and is measured on a held-out
|
| 407 |
+
split, because the current one was trained and measured on the same
|
| 408 |
+
traffic (a fit, not a generalisation).
|
| 409 |
+
- The full sweep, the fine-tune runs and the replay are in
|
| 410 |
+
`docs/ENCODER-NOTES.md` in [`splinter/`](https://github.com/sky-is-green/hive/tree/hive_test/splinter).
|
|
|
|
|
|
|
| 411 |
|
| 412 |
+
### 8.3 The curation's advantage is generator-dependent
|
| 413 |
|
| 414 |
The paired A/B - the same model, the same turns, the curated context
|
| 415 |
against the last-4k-token window of the same history - is the memory
|
|
|
|
| 443 |
budget*. The honest claim for a strong generator is the bounded context
|
| 444 |
and the cost, not the better answers.
|
| 445 |
|
| 446 |
+
### 8.4 The curation's cost grows with the conversation it curates
|
| 447 |
|
| 448 |
+
The stability run - 2,000 turns of a growing conversation, the real drone,
|
| 449 |
+
no model (`experiments.stability_500.py`):
|
| 450 |
+
|
| 451 |
+
| turns | the store | the assembled context | the curation's cost (p50) | the peak RSS |
|
| 452 |
+
|---|---|---|---|---|
|
| 453 |
+
| 250 | 212 chunks | 322 tok | 22.8 ms | 519 MB |
|
| 454 |
+
| 500 | 412 chunks | 320 tok | 37.3 ms | 564 MB |
|
| 455 |
+
| 1,000 | 812 chunks | 335 tok | 89.7 ms | 594 MB |
|
| 456 |
+
| 2,000 | 1,612 chunks | 332 tok | 156.8 ms | 667 MB |
|
|
|
|
| 457 |
|
| 458 |
+
**The context is flat and the cost is not.** The assembled context moves
|
| 459 |
+
by 10 tokens across a 7.6x range of store - the curation's promise holds - and nothing is OOM-killed over 2,000 turns. But the curation's
|
| 460 |
+
per-turn cost is **linear in the store**: about 7x for 7.6x the chunks,
|
| 461 |
+
because every chunk is scored every turn.
|
| 462 |
+
|
| 463 |
+
So the memory's trade is explicit, and the published "≈18 ms added per
|
| 464 |
+
turn" is the *small-store* number: the prompt stays bounded (the
|
| 465 |
+
expensive part for the model) while the curation gets more expensive as
|
| 466 |
+
the conversation grows. The two costs trade against each other, and the
|
| 467 |
+
crossover - the point where curating costs more than the tokens it saves
|
| 468 |
+
- is a measurement the system does not yet make.
|
| 469 |
+
|
| 470 |
+
## 9. Limits and what is not claimed
|
| 471 |
+
|
| 472 |
+
### 9.1 Limits
|
| 473 |
|
| 474 |
- **Precision is the open question.** The context that reaches the model is
|
| 475 |
fatter than it needs to be. The recall evidence says the facts the model
|
|
|
|
| 479 |
circularity but does not remove it: the auditor is the same model family
|
| 480 |
that produced the answer.
|
| 481 |
- **Synthetic-corpus bias.** The long-horizon corpus is generated. The
|
| 482 |
+
measurement in §8 is on real traffic; the P1-P12 verdicts mostly are not.
|
| 483 |
- **Cross-conversation contamination and hedge-replies** are harness-level
|
| 484 |
failure modes that look like architectural failures if you do not
|
| 485 |
isolate conversations per store and filter refusals out of the store.
|
|
|
|
| 500 |
keeps what the VRAM cache cannot hold in ordinary RAM, and the storage
|
| 501 |
read on the second request falls from 1.4 GB to 146 MB.
|
| 502 |
|
| 503 |
+
### 9.2 What is not claimed
|
| 504 |
+
|
| 505 |
+
- **Not that retrieval precision is solved.** The 10.7% sentence-proxy
|
| 506 |
+
precision is the encoder ceiling, not a tuning miss.
|
| 507 |
+
- **Not that curated context always wins on answer quality.** On the 35B-A3B
|
| 508 |
+
the paired answers tie (88.4% at least as good; 13 strict wins to 17); what
|
| 509 |
+
survives is the bounded context, fidelity and cost.
|
| 510 |
+
- **Not that P6 or P10 work.** Both failed; the mechanisms and the
|
| 511 |
+
falsification records are kept.
|
| 512 |
+
- **Not that §8 is a generic ranking result.** It is what was measured on
|
| 513 |
+
this harness's own traffic; the canonical ranking ceiling still applies.
|
| 514 |
+
- **Not that the auditor is independent.** P7 is a single-rater, same-family
|
| 515 |
+
agreement measurement, not ground truth.
|
| 516 |
+
- **Not that P12 is measured.** It is a registered draft.
|
| 517 |
+
- **Not that the results generalise.** The P1-P12 verdicts are mostly
|
| 518 |
+
synthetic corpora; §8 is the real-traffic exception.
|
| 519 |
+
|
| 520 |
+
---
|
| 521 |
+
|
| 522 |
+
## 10. Reproduce
|
| 523 |
|
| 524 |
```bash
|
| 525 |
# the offline suite (deterministic diagnostics, no model needed)
|
|
|
|
| 538 |
The repository ships the run bundles, the event logs and the label files, so
|
| 539 |
a verdict can be traced back to the bytes that produced it.
|
| 540 |
|
| 541 |
+
### 10.1 The artifacts
|
| 542 |
|
| 543 |
| artifact | what it is |
|
| 544 |
|---|---|
|
|
|
|
| 551 |
The releases are private while the results are being refreshed; they open
|
| 552 |
when the refreshed numbers are in.
|
| 553 |
|
| 554 |
+
## 11. License and community invitation
|
| 555 |
|
| 556 |
MIT. See `LICENSE` and `CITATION.cff` in this repository.
|
| 557 |
|
|
|
|
| 561 |
for Long-Horizon LLM Conversations},
|
| 562 |
author = {sky-is-green},
|
| 563 |
year = {2026},
|
| 564 |
+
url = {https://github.com/sky-is-green/hive}
|
| 565 |
}
|
| 566 |
```
|
| 567 |
|