Update logbook: Repro - KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem

logbook.json

@@ -11,7 +11,7 @@
     "icml2026-repro",
     "paper-k5nKHWp9VC"
   ],
-  "updated_at": "2026-07-16T15:57:01+00:00",
+  "updated_at": "2026-07-16T15:59:06+00:00",
   "root": {
     "slug": "index",
     "title": "Repro - KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem",
@@ -55,6 +55,6 @@
       }
     ]
   },
-  "agent_view_tokens": 7692,
-  "revision": "1784217421093453189"
+  "agent_view_tokens": 7827,
+  "revision": "1784217546867069398"
 }

pages/claim-3-tpt-and-two-stage-optimization/page.md

@@ -1659,3 +1659,55 @@ Loading weights: 100%|██████████| 146/146 [00:00<00:00, 374.
 {"type": "markdown", "id": "cell_4f7ec612d20f", "created_at": "2026-07-16T15:56:54+00:00", "title": "This benchmark separates algorithmic correctness from systems performance: DP-s…"}
 -->
 This benchmark separates algorithmic correctness from systems performance: DP-selected masks preserve output and acceptance, but direct Python sublayer dispatch, per-round cache cloning, and non-fused verification dominate wall time. A real speedup now requires fused/compiled kernels or an inference runtime implementation, not further DP tuning alone.
+
+
+---
+<!-- trackio-cell
+{"type": "code", "id": "cell_c73b29783712", "created_at": "2026-07-16T15:59:03+00:00", "title": "Run: uv focused_timing.py (exit 1)", "command": ["uv", "run", "python", "repro_knapspec/focused_timing.py"], "exit_code": 1, "duration_s": 21.075}
+-->
+````bash
+$ uv run python repro_knapspec/focused_timing.py
+````
+
+exit 1 · 21.1s
+
+
+````python title=focused_timing.py
+#!/usr/bin/env/env -S uv run --script
+# /// script
+# dependencies = ["torch", "transformers==5.14.1", "huggingface-hub>=1.23"]
+# ///
+import statistics,torch
+from transformers import AutoTokenizer,AutoModelForCausalLM
+from ssd_benchmark import MODEL,install_switches,greedy,spec_decode
+text='The committee reviewed the proposal and requested a detailed analysis of the expected costs, benefits, risks, and implementation timeline.'
+tok=AutoTokenizer.from_pretrained(MODEL); m=AutoModelForCausalLM.from_pretrained(MODEL,dtype=torch.float16).cuda().eval(); sw=install_switches(m); seed=tok(text,return_tensors='pt').input_ids.cuda(); p=seed.repeat(1,(512+seed.shape[1]-1)//seed.shape[1])[:,:512]; mask={16,22}; greedy(m,p,64); spec_decode(m,sw,p,64,8,skip_mask=mask); bt=[];st=[]
+for _ in range(2):
+ b,t=greedy(m,p,64); bt.append(64/t); _,s=spec_decode(m,sw,p,64,8,skip_mask=mask); st.append(64/s['elapsed_s']); print(s)
+print({'baseline_tps':statistics.median(bt),'spec_tps':statistics.median(st),'speedup':statistics.median(bt)/statistics.median(st)})
+
+````
+
+
+````output
+
+Loading weights:   0%|          | 0/146 [00:00<?, ?it/s]
+Loading weights:   1%|          | 1/146 [00:00<00:35,  4.09it/s]
+Loading weights:  47%|████▋     | 68/146 [00:00<00:00, 235.01it/s]
+Loading weights:  70%|██████▉   | 102/146 [00:00<00:00, 170.09it/s]
+Loading weights:  86%|████████▋ | 126/146 [00:00<00:00, 150.04it/s]
+Loading weights: 100%|██████████| 146/146 [00:01<00:00, 138.56it/s]
+Loading weights: 100%|██████████| 146/146 [00:01<00:00, 143.29it/s]
+Traceback (most recent call last):
+  File "/home/nick/workspace/icml26-hf-challenge-knapspec/repro_knapspec/focused_timing.py", line 9, in <module>
+    tok=AutoTokenizer.from_pretrained(MODEL); m=AutoModelForCausalLM.from_pretrained(MODEL,dtype=torch.float16).cuda().eval(); sw=install_switches(m); seed=tok(text,return_tensors='pt').input_ids.cuda(); p=seed.repeat(1,(512+seed.shape[1]-1)//seed.shape[1])[:,:512]; mask={16,22}; greedy(m,p,64); spec_decode(m,sw,p,64,8,skip_mask=mask); bt=[];st=[]
+                                                                                                                                                                                                                                                                                                         ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/nick/workspace/icml26-hf-challenge-knapspec/repro_knapspec/ssd_benchmark.py", line 109, in spec_decode
+    past=clone_cache(target_past) if skip_mask is not None else None
+         ~~~~~~~~~~~^^^^^^^^^^^^^
+  File "/home/nick/workspace/icml26-hf-challenge-knapspec/repro_knapspec/ssd_benchmark.py", line 68, in clone_cache
+    return DynamicCache.from_legacy_cache(tuple((k.clone(),v.clone()) for k,v in legacy))
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+AttributeError: type object 'DynamicCache' has no attribute 'from_legacy_cache'
+
+````