| --- |
| license: apache-2.0 |
| library_name: peft |
| base_model: Qwen/Qwen2.5-0.5B-Instruct |
| tags: |
| - memory |
| - long-context |
| - multi-session |
| - consolidation |
| - lora |
| --- |
| |
| # CLERK — Consolidated Ledger with Eviction and Rewrite Keys |
|
|
| **Fixed-budget, write-time memory consolidation for LLMs, trained (not |
| prompted).** CLERK maintains memory as a JSON ledger of atomic fact slots. |
| At every session boundary a trained policy reads (ledger, session |
| transcript) and emits a structured edit program — `ADD`, `UPDATE` |
| (supersede), `TOMBSTONE`, `EVICT` — which a deterministic reducer applies. |
| Reading then costs O(budget) tokens forever, regardless of history length, |
| and supersession is explicit rather than buried in stale text. |
|
|
| ## Why this is new |
|
|
| - All published write-time consolidation is **prompt-heuristic** (Mem0, |
| Zep, CUPMem); prior work shows LLM-consolidated memories corrupt over |
| repeated updates. CLERK **trains** the write policy on exact supervision |
| from programmatically generated gold ledger transitions. |
| - Learned memory policies (Memory-T1, MemAgent) act at **read time**; |
| CLERK acts at write time with a **fixed budget** and learned eviction. |
| - Prior fixed-budget parametric memories (RMT, Infini-attention) are |
| opaque token memories trained from scratch; CLERK is **interpretable, |
| reducer-checked, and LoRA-scale**. |
|
|
| ## Repository layout |
|
|
| ``` |
| clerk/ |
| common.py slot schema, edit-op reducer, serializers, salience oracle |
| prompts.py CONSOLIDATE / ANSWER instruction formats (single source of truth) |
| generator.py seeded synthetic evolving-session generator (gold ledger states) |
| build_sft.py timelines -> SFT messages dataset (CONSOLIDATE + ANSWER mixture) |
| train_sft.py LoRA SFT via TRL (verified against trl/examples/sft_qlora) |
| eval_synthetic.py held-out benchmark: clerk vs prompted-ledger/full/window/RAG |
| eval_locomo.py LoCoMo-MC10 zero-shot (Percena/locomo-mc10) |
| tests_smoke.py data-invariant checks (run before any GPU job) |
| run_all.sh end-to-end pipeline |
| requirements.txt pinned dependencies |
| paper/ the paper |
| ``` |
|
|
| ## Reproduce |
|
|
| ```bash |
| pip install -r requirements.txt |
| huggingface-cli login |
| bash run_all.sh # ~2-3 GPU-hours on one 16GB GPU (L4/A10G class) |
| ``` |
|
|
| Everything is seeded (generator seed 137/991/4242; training seed 137). |
| Smoke tests verify: gold op programs are executable by the reducer, |
| budgets are never exceeded, serializers round-trip, and QA supervision |
| matches the memory state it was answered against. |
|
|
| ## Results |
|
|
| See `paper/paper.md` and `results/` (populated by `run_all.sh`). |
|
|
| ## License |
|
|
| Apache-2.0. Author: Justin Wolcott, fallnai-research.org. |
