---
license: apache-2.0
library_name: peft
base_model: Qwen/Qwen2.5-0.5B-Instruct
tags:
  - memory
  - long-context
  - multi-session
  - consolidation
  - lora
---

# CLERK — Consolidated Ledger with Eviction and Rewrite Keys

**Fixed-budget, write-time memory consolidation for LLMs, trained (not
prompted).** CLERK maintains memory as a JSON ledger of atomic fact slots.
At every session boundary a trained policy reads (ledger, session
transcript) and emits a structured edit program — `ADD`, `UPDATE`
(supersede), `TOMBSTONE`, `EVICT` — which a deterministic reducer applies.
Reading then costs O(budget) tokens forever, regardless of history length,
and supersession is explicit rather than buried in stale text.

## Why this is new

- All published write-time consolidation is **prompt-heuristic** (Mem0,
  Zep, CUPMem); prior work shows LLM-consolidated memories corrupt over
  repeated updates. CLERK **trains** the write policy on exact supervision
  from programmatically generated gold ledger transitions.
- Learned memory policies (Memory-T1, MemAgent) act at **read time**;
  CLERK acts at write time with a **fixed budget** and learned eviction.
- Prior fixed-budget parametric memories (RMT, Infini-attention) are
  opaque token memories trained from scratch; CLERK is **interpretable,
  reducer-checked, and LoRA-scale**.

## Repository layout

```
clerk/
  common.py         slot schema, edit-op reducer, serializers, salience oracle
  prompts.py        CONSOLIDATE / ANSWER instruction formats (single source of truth)
  generator.py      seeded synthetic evolving-session generator (gold ledger states)
  build_sft.py      timelines -> SFT messages dataset (CONSOLIDATE + ANSWER mixture)
  train_sft.py      LoRA SFT via TRL (verified against trl/examples/sft_qlora)
  eval_synthetic.py held-out benchmark: clerk vs prompted-ledger/full/window/RAG
  eval_locomo.py    LoCoMo-MC10 zero-shot (Percena/locomo-mc10)
  tests_smoke.py    data-invariant checks (run before any GPU job)
run_all.sh          end-to-end pipeline
requirements.txt    pinned dependencies
paper/              the paper
```

## Reproduce

```bash
pip install -r requirements.txt
huggingface-cli login
bash run_all.sh          # ~2-3 GPU-hours on one 16GB GPU (L4/A10G class)
```

Everything is seeded (generator seed 137/991/4242; training seed 137).
Smoke tests verify: gold op programs are executable by the reducer,
budgets are never exceeded, serializers round-trip, and QA supervision
matches the memory state it was answered against.

## Results

See `paper/paper.md` and `results/` (populated by `run_all.sh`).

## License

Apache-2.0. Author: Justin Wolcott, fallnai-research.org.