Commit History
28 days ago
-
Create README.md
Verified signature10b4db8 -
Create LICENSE
Verified signature484ef15 -
Create README.md
Verified signature--- language: - en license: apache-2.0 library_name: custom pipeline_tag: text-generation tags: - content-addressable-memory - memory-augmented - working-memory - bottleneck - from-scratch - diy - micro-lm - numpy - transformer - retrieval - lookup datasets: - synthetic-lookup metrics: - accuracy - length-generalization pretty_name: content-address-v1 --- # content-address-v1 A pair of micro language models trained **from scratch** on a lookup task, demonstrating that content-addressable memory succeeds where attention-only retrieval fails. ## Model Description This model is a proof of concept for the **working-memory bottleneck hypothesis** described in Chapter 8 of *Craft Erasure*. The claim: an attention-only transformer must retrieve stored values by attending backward over the sequence, and this is bottlenecked by the effective attention window. A memory-augmented model that reads from a separate content-addressable store does not have this bottleneck, and its retrieval accura…54f5f51 -
Create content_address_v1.py
Verified signaturef934c8e -
initial commit
Verified signature9b30794