Instructions to use mburgc/qwenputer-mirror with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mburgc/qwenputer-mirror with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mburgc/qwenputer-mirror", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Learning from a Deterministic Oracle Inside the Model
A CPU inside the weights teaches the model, exactly, and we measure where the memory stops
Experimental personal research — not peer-reviewed. Several experiments failed. They are reported with the same detail as the successes, including one apparent positive that a control refuted. Read Limitations before citing anything here.
Author: Marcelo Ernesto Burgos Cayupil · Date: November 2026 · License: MIT
TL;DR
A language model learns new facts with no gradients, using as ground truth a deterministic RISC-V CPU compiled into the weights of the same checkpoint.
- The oracle works. A 177M-parameter RV32IM circuit (recovered from a checkpoint whose runtime was lost) computes eight 18-digit products exactly, verified against Python.
- The loop works. Eight facts written into
lm_headwithΔW = YᵀK(KᵀK+λI)⁻¹move character accuracy from 0/71 to 71/71, against a shuffled-target control of 1/71, with a 6 MB adapter that removes cleanly. - Capacity is key diversity, not fact count. Forty-two constraints from independent prompts collapse; 391 constraints from a chained derivation fit exactly. Keys of different facts are nearly identical (cosine 0.95 median), so all signal lives in differences of 0.01.
1. The oracle
The checkpoint shipped without its driver (the repository returns 404). We recovered it from the weights alone: a four-line stub for the missing class, then the layout formula
c = core.build(Target(16, 32, 9216), BR.Text(tokenizer.json)) # 2918 dims, 2492 nodes
alloc = allocate(c, None, set(c.keep), 9216) # 1265 slots
lay = Layout(hidden=3328, off=2048, m=13, slots=alloc.slots) # col = 2062 + slot
which predicts the checkpoint's residual columns (2193 for the first code dim, 2215 for mu,
2216 for tau) without consulting it. Seven checks, seven matches.
| Expression | Circuit output | Python | |
|---|---|---|---|
| 52445 × 3471 | 182036595 | 182036595 | ✅ |
| 61750 × 1791 | 110594250 | 110594250 | ✅ |
| 19494 × 9779 | 190631826 | 190631826 | ✅ |
| 22337 × 6991 | 156157967 | 156157967 | ✅ |
| 86387 × 1950 | 168454650 | 168454650 | ✅ |
| 76510 × 4517 | 345595670 | 345595670 | ✅ |
| 14914 × 2408 | 35912912 | 35912912 | ✅ |
| 66838 × 7851 | 524745138 | 524745138 | ✅ |
Mean latency ≈ 70 s per product on a laptop 3070.
2. The learning loop
For each answer token j: key k_j is the post-norm hidden state at position len(prompt)-1+j,
target gap γ_j = logit(top) − logit(target) + 3 (capped at 12). Then
ΔW = YᵀK(KᵀK + λI)⁻¹, applied as W ← W + αΔW.
| Condition | Characters correct | Full answers |
|---|---|---|
| Base | 0 / 71 | 0 / 8 |
| ΔW | 71 / 71 | 8 / 8 (as prefix) |
| Control (shuffled targets, same norm) | 1 / 71 | 0 / 8 |
| Rollback | 0 / 71 | 0 / 8 |
base: ' = 181601490' (wrong; true value 182036595)
patched: ' 1820365956595' (exact, then the model keeps writing)
Adapter: rank 12, 6.0 MB, stored as U·S·Vᵀ.
3. Capacity
| Facts (independent prompts) | Pairs | rank(ΔW) | Accuracy |
|---|---|---|---|
| 5 | 10 | 10 | 5/5 |
| 10 | 18 | 16 | 3/10 |
| 15 | 28 | 26 | 9/15 |
| 20 | 37 | 34 | 6/20 |
Non-monotonic: ten facts score worse than fifteen. The measurement that explains it:
cosine between keys of different facts: median 0.95 max 0.99
Prompts sharing a template produce nearly identical states. A rank-limited linear map competes for the same directions, and the outcome depends on which names happen to be in the set.
The converse confirms it: a 65-token derivation taught across six examples produced 391 constraints that fit without collapse, because chained keys are correlated and cheap to cover.
Rule of thumb. Chained sequences are cheap; independent facts are expensive. Quote capacity in key diversity, not in facts.
4. Four failures
4.1 No generalization to paraphrases
The delta is keyed on exact states; an unseen phrasing produces an unseen key and the update contributes nothing beyond base level.
4.2 No procedure learning
Teaching a three-step derivation for six products (intermediate values produced by the oracle) memorizes the derivation — correct decomposition, correct first partial product — and scores 0/3 on unseen products.
4.3 Composition: a false positive caught by a control
With two chains (A→B, B→C), both direct facts were learned (4/4). Asked for A→C the model answered correctly in 1 of 2 cases — which looks like composition.
It is not. Three names never seen in training also answer Vela:
river of Zorblax → 'Vela' (trained)
river of Velmora → 'Velmora' (trained)
river of Fennax → 'Vela' (NEVER trained)
river of Ossric → 'the Vela' (NEVER trained)
river of Draven → 'Vela' (NEVER trained)
The update biased the output distribution globally. Composition: 0. Without the control, this paper would have claimed a capability that does not exist.
4.4 Internal layers, and a soft memory graft
A crude MEMIT-style edit of mlp.down_proj across five layers had no effect (0/20): the
targets ignored the final normalization, so this is an implementation failure, not evidence
against internal edits.
A soft memory graft at layer 8 retrieved the correct fact 10/10 and produced the correct first token 10/10, but failed end to end (0.20): re-injecting a fixed direction at every decoding step either destabilizes the residual or leaves it unchanged.
Both match the pattern of §2: the unit is the pair (prefix, next token); one vector per fact cannot sustain a sequence.
5. Related work
| System | Closed-form | Exact ground truth | No gradients | Fits 8 GB |
|---|---|---|---|---|
| ROME / MEMIT | ✅ | ❌ dataset | ✅ | ✅ |
| SEAL | ❌ SFT | ❌ self-generated | ❌ | — |
| HOPE / Nested Learning | ❌ | ❌ | ❌ | — |
| JitRL | — | ❌ | ✅ | — |
| Qwenputer | — | ✅ in-weights CPU | — | ❌ 18.8 GB |
| This work | ✅ | ✅ in-weights CPU | ✅ | ✅ 0.35 GB circuit |
The update rule is MEMIT's. What is new is the composition: an exact computer as the teacher, closed-form algebra as the writer, and an explicit map of where the memory stops working.
6. Limitations
- One model, one task family (invented facts and integer products).
- Single seed for the capacity sweep; variance is reported, not averaged.
- The 2B language half of the source checkpoint is not runnable (quantized weights, missing wrapper). Only the circuit was recovered.
- Composition is n=2, max — small, and negative after control.
- The MEMIT-style edit is a crude approximation and should not be read as a refutation.
- No claim of first-ness. The components are prior work; the composition is what we believe to be new, and we found no published precedent.
7. Reproduce
python experiments/e1_oracle_exact.py
python experiments/e2_loop_8facts.py
python experiments/e3_capacity.py
python experiments/e5_composition.py
python figures/make_figures.py
Hardware: RTX 3070 Laptop 8 GB, i7-11800H, 31.7 GB RAM. Stack: torch 2.13+cu126,
transformers 5.22.2. Raw numbers: results/results.json.
8. Resumen en español
Un modelo de lenguaje aprende hechos nuevos sin gradientes, usando como fuente de verdad un CPU RISC-V determinista compilado dentro de los pesos del mismo checkpoint.
El circuito se reconstruyó desde cero (el runtime original ya no está publicado) y se verificó:
8 de 8 productos de 18 dígitos exactos. Con esa verdad se calculó un delta de forma cerrada
sobre lm_head: el modelo pasó de 0/71 a 71/71 caracteres, con control de targets barajados
en 1/71 y rollback limpio. El adapter pesa 6 MB.
Lo que no funciona está reportado con el mismo detalle: paráfrasis, composición (un falso positivo que un control descartó), edición de capas internas con una aproximación cruda, y el injerto de memoria más allá del primer token.
Investigación personal experimental. No revisada por pares, no es un producto, no reclama estado del arte.
Citation
@software{burgos2026oracle,
author = {Burgos Cayupil, Marcelo Ernesto},
title = {Learning from a Deterministic Oracle Inside the Model},
year = {2026},
note = {Experimental personal research, not peer-reviewed},
url = {https://huggingface.co/mburgc/qwenputer-mirror-results}
}



