|
|
|
|
| <header> |
| <div class="kicker">Live proof · real weights · nothing canned</div> |
| <h1 class="serif">μLattice: the growing memory, running</h1> |
| <p class="dek serif">A one-million-parameter lattice mixer that stores associations while it |
| reads, overwrites them on sight, and answers from memory — at constant cost per token. |
| This page drives the actual trained checkpoint on the GPU.</p> |
| <div class="meta" id="meta">model: capacity8k.pt · loading…</div> |
| </header> |
| <main> |
|
|
| <section> |
| <div class="sn"><span class="n">01</span><h2 class="serif">Stream your own document</h2></div> |
| <p class="lead">Paste any text. Define your own facts as <em>name → value</em> pairs. Mark any |
| of them to be <strong>overwritten mid-stream</strong> with a new value. Then ask. The model reads |
| your words as the stream, stores your pairs as it passes them, and must return the |
| <em>latest</em> value — last write wins.</p> |
| <div class="fig"> |
| <div class="grid2"> |
| <div> |
| <label for="doc">Your text (becomes the stream)</label> |
| <textarea id="doc">The harbour was quiet that morning. Gulls circled the masts while the tide went out. A clerk stamped papers without looking up. Somewhere a bell rang twice, then stopped. The road east had been washed out again. Dust settled over the market stalls. Two children argued about a kite. Rain threatened but never came.</textarea> |
| </div> |
| <div> |
| <label>Your facts <span class="note">(name → value; fill “new value” to overwrite mid-stream)</span></label> |
| <div class="pairs" id="pairs"></div> |
| <button class="sec" id="addp" style="margin-top:8px;padding:6px 16px;font-size:13px">+ add fact</button> |
| </div> |
| </div> |
| <button id="runc">Read, store, overwrite — and answer</button> |
| <div class="stats" id="stc" style="display:none"></div> |
| <div id="cres" style="display:none"> |
| <table id="qrt"></table> |
| <div class="figcap" id="ccap"></div> |
| </div> |
| </div> |
| </section> |
|
|
| <section> |
| <div class="sn"><span class="n">02</span><h2 class="serif">The store at scale</h2></div> |
| <p class="lead">The same mechanism under load: up to 8,192 pairs in one 133,120-token stream, |
| half of them overwritten, answered after a single causal pass. The map shows the model’s |
| own learned addresses — distinct keys spread, exactly as the design intends.</p> |
| <div class="fig"> |
| <div class="row"> |
| <div class="rangewrap"><label style="margin:0">pairs</label> |
| <input id="np" type="range" min="64" max="8192" step="64" value="2048"><span class="rv" id="npv">2048</span></div> |
| <div class="rangewrap"><label style="margin:0">filler/pair</label> |
| <input id="fl" type="range" min="0" max="14" value="14"><span class="rv" id="flv">14</span></div> |
| <div class="rangewrap"><label style="margin:0">overwritten</label> |
| <input id="ow" type="range" min="0" max="50" value="50"><span class="rv" id="owv">50%</span></div> |
| <button class="sec" id="run" style="margin-top:0">stream & query</button> |
| </div> |
| <div class="stats" id="st" style="display:none"></div> |
| <div class="row" style="align-items:stretch" id="bigres" > |
| <canvas id="latt" height="280" style="flex:1.2;min-width:300px"></canvas> |
| <div style="flex:1;min-width:280px;max-height:300px;overflow:auto"> |
| <table id="qr"></table></div> |
| </div> |
| <div class="figcap">Left: learned addresses of 400 stored keys from this run (grid scale shown). |
| Right: 64 sampled queries; gold rows marked <span class="pill">overwritten</span> were |
| re-written later in the stream — the store must answer with the new value.</div> |
| </div> |
| </section> |
|
|
| <section> |
| <div class="sn"><span class="n">03</span><h2 class="serif">Constant cost while memory grows</h2></div> |
| <p class="lead">Each token touches exactly nine memory cells, so per-token work cannot grow with |
| context. Measure it: the curve below runs the model at increasing stream lengths, live.</p> |
| <div class="fig"> |
| <button class="sec" id="cost" style="margin-top:0">measure the cost curve</button> |
| <canvas id="cc" height="300" style="margin-top:14px"></canvas> |
| <div class="figcap">Blue: measured μs per token — flat is the guarantee. Green: lattice |
| cells, growing with stored content. Dashed red: attention’s O(T²) work, analytic reference.</div> |
| </div> |
| </section> |
|
|
| <section> |
| <div class="sn"><span class="n">04</span><h2 class="serif">Inside one cell</h2></div> |
| <p class="lead">Where does a value physically live, and where does the <em>new</em> value go on |
| overwrite? Pick one fact from section 01 (or any name/value). The model’s learned address |
| for that key, the 3×3 cells it touches, and the before/after answers — same cells, |
| value replaced in place by the delta rule.</p> |
| <div class="fig"> |
| <div class="grid2"> |
| <div> |
| <label>fact to inspect</label> |
| <div class="prow" style="grid-template-columns:1fr 1fr 1fr"> |
| <input id="cvn" placeholder="name" value="the vault password"> |
| <input id="cvv" placeholder="value" value="swordfish"> |
| <input id="cvnv" placeholder="new value" value="bluedragon"></div> |
| <button class="sec" id="cv">inspect the cell</button> |
| <div id="cvs" style="display:none;margin-top:14px"></div> |
| </div> |
| <div><canvas id="cellcv" height="300"></canvas> |
| <div class="figcap" id="cvcap"></div></div> |
| </div> |
| </div> |
| </section> |
|
|
| </main> |
| <footer>Every number on this page is produced by a live forward pass of |
| <code>capacity8k.pt</code> on request — timestamped per run. Filler for section 01 is your own |
| text, hashed deterministically into the model’s token space; facts occupy key–value |
| structural positions, exactly the regime the mixer was trained on.</footer> |
| <script> |
| const $=id=>document.getElementById(id); |
| const esc=s=>String(s).replace(/[&<>"]/g,c=>({"&":"&","<":"<",">":">",'"':"""}[c])); |
| async function api(p,body){const r=await fetch(p,{method:"POST", |
| headers:{"Content-Type":"application/json"},body:JSON.stringify(body)});return r.json()} |
| function stats(el,d){el.style.display="grid"; |
| const s=[["stream tokens",d.T.toLocaleString()],["time",d.wall_s+" s"], |
| ["μs/token",d.us_per_token],["cells",d.cells.toLocaleString()],["recall",d.acc]]; |
| el.innerHTML=s.map(x=>`<div class="stat"><b>${x[1]}</b><span>${x[0]}</span></div>`).join("")} |
| |
| const DEF=[["the vault password","swordfish","bluedragon"], |
| ["Elbaz's first name","Eric",""],["the car's color","red",""], |
| ["the meeting time","3pm","4pm"],["the favorite meal","stew",""]]; |
| function prow(n,v,nv){const d=document.createElement("div");d.className="prow"; |
| d.innerHTML=`<input placeholder="name" value="${esc(n)}"> |
| <input placeholder="value" value="${esc(v)}"> |
| <input placeholder="new value (optional)" value="${esc(nv)}"> |
| <button class="del" title="remove">×</button>`; |
| d.querySelector(".del").onclick=()=>d.remove();return d} |
| DEF.forEach(x=>$("pairs").appendChild(prow(...x))); |
| $("addp").onclick=()=>$("pairs").appendChild(prow("","","")); |
| $("runc").onclick=async()=>{$("runc").disabled=true;$("runc").textContent="reading your stream…"; |
| try{ |
| const pairs=[...document.querySelectorAll("#pairs .prow")].map(r=>{ |
| const i=r.querySelectorAll("input");return{name:i[0].value.trim(),value:i[1].value.trim(), |
| new_value:i[2].value.trim()}}).filter(p=>p.name&&p.value); |
| const d=await api("/api/custom",{text:$("doc").value,pairs,seed:(Math.random()*1e6)|0}); |
| if(d.error){alert(d.error);return} |
| $("meta").textContent=`capacity8k.pt · scale ${d.scale||96} · ${d.device} · run ${d.ts}`; |
| stats($("stc"),d); |
| $("qrt").innerHTML=`<tr><th>asked</th><th>the model answered</th><th>latest value</th><th></th></tr>`+ |
| d.rows.map(r=>`<tr><td><b>${esc(r.name)}</b></td> |
| <td class="${r.ok?"y":"n"}">${esc(r.asked)}</td> |
| <td class="${r.updated?"updv":""}">${esc(r.truth)}</td> |
| <td>${r.updated?`<span class="pill">was ${esc(r.old)}</span>`:""}${r.ok?"":" ✗"}</td></tr>`).join(""); |
| $("cres").style.display="block"; |
| $("ccap").innerHTML=`${d.n_pairs} of your facts stored in ${d.T.toLocaleString()} tokens of your text`+ |
| (d.n_upd?`, ${d.n_upd} overwritten mid-stream`:"")+` · recall <b>${d.acc}</b> · ${d.us_per_token} μs/token`+ |
| ` · key ids: `+Object.entries(d.keymap).slice(0,4).map(([n,k])=>`${esc(n)}=${k}`).join(", ")+`…`; |
| }catch(e){alert(e)}$("runc").disabled=false;$("runc").textContent="Read, store, overwrite — and answer"}; |
| |
| $("np").oninput=e=>$("npv").textContent=e.target.value; |
| $("fl").oninput=e=>$("flv").textContent=e.target.value; |
| $("ow").oninput=e=>$("owv").textContent=e.target.value+"%"; |
| function drawLatt(pts,sc){const c=$("latt"),x=c.getContext("2d"); |
| c.width=c.clientWidth*2;c.height=560;x.scale(2,2);const W=c.width/2,H=280; |
| x.fillStyle="#fff";x.fillRect(0,0,W,H); |
| x.strokeStyle="#eceef2";for(let i=0;i<=8;i++){const p=i/8; |
| x.beginPath();x.moveTo(p*W,0);x.lineTo(p*W,H);x.stroke(); |
| x.beginPath();x.moveTo(0,p*H);x.lineTo(W,p*H);x.stroke()} |
| x.fillStyle="#2547d0";for(const p of pts){const u=(p[0]/sc+1)/2,v=(p[1]/sc+1)/2; |
| x.beginPath();x.arc(u*W,v*H,1.7,0,7);x.fill()}} |
| $("run").onclick=async()=>{$("run").disabled=true;$("run").textContent="streaming…"; |
| try{const d=await api("/api/store",{n_pairs:+$("np").value,filler:+$("fl").value, |
| overwrite_frac:+$("ow").value/100,seed:(Math.random()*1e6)|0}); |
| $("meta").textContent=`capacity8k.pt · scale ${d.scale} · ${d.params.toLocaleString()} params · ${d.device} · run ${d.ts}`; |
| stats($("st"),d);drawLatt(d.addr_points,d.scale); |
| $("qr").innerHTML=`<tr><th>query key</th><th>answered</th><th>true</th><th></th></tr>`+ |
| d.rows.map(r=>`<tr><td>${r.key}</td><td class="${r.ok?"y":"n"}">${r.pred}</td>`+ |
| `<td class="${r.updated?"updv":""}">${r.gold}</td>`+ |
| `<td>${r.updated?'<span class="pill">overwritten</span>':""}${r.ok?"":" ✗"}</td></tr>`).join(""); |
| }catch(e){alert(e)}$("run").disabled=false;$("run").textContent="stream & query"}; |
| |
| function drawCC(data){const c=$("cc"),x=c.getContext("2d");c.width=c.clientWidth*2;c.height=600; |
| x.scale(2,2);const W=c.width/2,H=300,P=48; |
| const maxT=Math.max(...data.map(d=>d.T)),us=data.map(d=>d.us_per_token), |
| ce=data.map(d=>d.cells),maxU=Math.max(...us)*1.25,maxC=Math.max(...ce)*1.15, |
| maxA=Math.max(...data.map(d=>d.flops_attention_equiv)); |
| const X=t=>P+(Math.log(t)/Math.log(maxT))*(W-P-16), |
| Y=u=>H-30-(u/maxU)*(H-56),Y2=v=>H-30-(v/maxC)*(H-56),YA=v=>H-30-(v/maxA)*(H-56); |
| x.fillStyle="#fff";x.fillRect(0,0,W,H);x.strokeStyle="#eceef2";x.fillStyle="#6b7280"; |
| x.font="11px -apple-system,sans-serif"; |
| for(let i=0;i<=4;i++){const px=P+(i/4)*(W-P-16);x.beginPath();x.moveTo(px,20);x.lineTo(px,H-30);x.stroke()} |
| x.strokeStyle="#c2413b";x.setLineDash([6,5]);x.beginPath();let f1=true; |
| for(const d of data){const px=X(d.T),py=YA(d.flops_attention_equiv); |
| f1?x.moveTo(px,py):x.lineTo(px,py);f1=false}x.stroke();x.setLineDash([]); |
| x.strokeStyle="#2547d0";x.lineWidth=2.4;x.beginPath();f1=true; |
| for(const d of data){const px=X(d.T),py=Y(d.us_per_token);f1?x.moveTo(px,py):x.lineTo(px,py);f1=false} |
| x.stroke(); |
| x.strokeStyle="#0e8a5f";x.lineWidth=2.4;x.beginPath();f1=true; |
| for(const d of data){const px=X(d.T),py=Y2(d.cells);f1?x.moveTo(px,py):x.lineTo(px,py);f1=false} |
| x.stroke();x.lineWidth=1; |
| for(const d of data){x.fillStyle="#2547d0";x.beginPath();x.arc(X(d.T),Y(d.us_per_token),3.4,0,7);x.fill(); |
| x.fillStyle="#0e8a5f";x.beginPath();x.arc(X(d.T),Y2(d.cells),3.4,0,7);x.fill()} |
| x.fillStyle="#6b7280"; |
| for(const d of data)x.fillText(d.T.toLocaleString(),X(d.T)-16,H-14); |
| x.fillText("stream length (tokens, log scale)",W/2-80,H-2); |
| x.fillText("μs/token (blue, flat = constant) · cells (green) · attention O(T²) (dashed red)",P,16)} |
| $("cost").onclick=async()=>{$("cost").disabled=true;$("cost").textContent="measuring…"; |
| try{const sizes=[64,256,1024,2048,4096,8192].map(n=>({n_pairs:n,filler:14})); |
| const d=await api("/api/cost",{sizes});drawCC(d)}catch(e){alert(e)} |
| $("cost").disabled=false;$("cost").textContent="measure the cost curve"}; |
| |
| $("cv").onclick=async()=>{$("cv").disabled=true; |
| try{const d=await api("/api/cellview",{name:$("cvn").value.trim()||"key", |
| value:$("cvv").value.trim()||"v1",new_value:$("cvnv").value.trim(),seed:(Math.random()*1e6)|0}); |
| if(d.error){alert(d.error);return} |
| const inv={[d.vid]:d.value};inv[d.nid]=d.new_value||d.value; |
| $("cvs").innerHTML=` |
| <div class="stats" style="display:grid"> |
| <div class="stat"><b>(${d.addr[0]}, ${d.addr[1]})</b><span>learned address</span></div> |
| <div class="stat"><b>${esc(inv[d.answers.before] ?? "—")}</b><span>stored & retrieved before</span></div> |
| <div class="stat"><b class="${d.answers.after!==null&&d.answers.after!==d.answers.before?"updv":""}">${esc(d.answers.after!==null?inv[d.answers.after]:"—")}</b><span>after overwrite, same cells</span></div> |
| </div> |
| <p class="note" style="margin:8px 0 0">key id ${d.kid} → address on grid scale ${d.scale}; |
| the write lands as <code>W ← αW + β(v<sub>new</sub> − αWκ)κ<sup>T</sup>: |
| the old association is subtracted out, the new one takes its place — nothing is appended.</p>`; |
| drawCell(d)}catch(e){alert(e)}$("cv").disabled=false}; |
| function drawCell(d){const c=$("cellcv"),x=c.getContext("2d"); |
| c.width=c.clientWidth*2;c.height=600;x.scale(2,2);const W=c.width/2,H=300; |
| x.fillStyle="#fff";x.fillRect(0,0,W,H);const cw=W/3,ch=H/3; |
| for(let i=0;i<3;i++)for(let j=0;j<3;j++){ |
| const phi=d.cells[i][j].phi,z=d.cells[i][j].z; |
| const t=Math.min(1,phi/0.7); |
| x.fillStyle=`rgba(37,71,208,${(0.06+0.85*t).toFixed(3)})`; |
| x.fillRect(j*cw+6,i*ch+6,cw-12,ch-12); |
| x.fillStyle=t>0.45?"#fff":"#6b7280";x.font="12px -apple-system,sans-serif"; |
| x.fillText(`(${z[0]}, ${z[1]})`,j*cw+14,i*ch+24); |
| x.font="600 15px -apple-system,sans-serif"; |
| x.fillText(`phi ${phi.toFixed(3)}`,j*cw+14,i*ch+ch-18)} |
| $("cvcap").textContent="the 3x3 neighborhood of the key's address; darker = stronger write/read weight (the bump kernel)."} |
| window.addEventListener("resize",()=>{}); |
| </script> |
