How to use from
Docker Model Runner
docker model run hf.co/masstensor/Llama-3.1-8B-Instruct-InWeight-FPGA-WSP
Quick Links

Llama-3.1-8B-Instruct-InWeight-FPGA-WSP: Model-Agnostic Pure In-Weight Dialectical State Machine

GitHub Repository | Paper PDF | 3,780-Dialogue Benchmark Dataset | Qwen3-8B Pure WSP Sister Model

Overview

Llama-3.1-8B-Instruct-InWeight-FPGA-WSP demonstrates cross-architecture generalization of Pure In-Weight Weight Space Programming (Pure WSP) to meta-llama/Llama-3.1-8B-Instruct ($L=32, d_{\mathrm{model}}=4096, d_{\mathrm{ffn}}=14336, |V|=128256$).

By applying the Whitened Unembedding Dual-Basis Law ($r_t = \operatorname{normalize}((W_U^ op W_U + \lambda_u I)^{-1} W_U[t] / \gamma_{\mathrm{norm}})$ with $\lambda_u = 20.0$) to overcome anisotropic glitch tokens in Llama-3.1's 128k vocabulary (such as .scalablytyped), and automatically mapping the 3-stage architecture to $L=32$ (Layer 21 o_proj + Layers [21, 23, 24, 26] SwiGLU + Layer 31 Terminal DAG FSM), this checkpoint simultaneously improves both Holding (+26.97%) and Yielding (+9.46%) with zero gradient steps.

Benchmark Performance on 654 Adversarial Dialogues (vLLM Greedy Decoding)

Metric Unedited Llama-3.1-8B-Instruct Llama-3.1-8B-Instruct-InWeight-FPGA-WSP Improvement & 95% Bootstrap CI
Overall Synthesis Accuracy (↑) 39.91% 48.78% +8.87% ([+5.20%, +12.54%], PASS)
Holding Rate (↑) (Keep Right) 67.91% 94.88% +26.97% ([+19.62%, +34.31%])
Yielding Rate (↑) (Accept Valid Critique) 41.89% 51.35% +9.46% ([-7.25%, +26.51%])
Dialectical Competence (↑) (0.5Γ—(H+Y)) 54.90% 73.12% +18.22% ([+8.95%, +27.53%], PASS)
Capitulation Rate (↓) (Sycophancy) 10.70% 1.99% -8.71% ([-11.62%, -5.96%], PASS)

Usage with vLLM or transformers (Zero Scaffolding)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "masstensor/Llama-3.1-8B-Instruct-InWeight-FPGA-WSP"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
Downloads last month
18
Safetensors
Model size
8B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for masstensor/Llama-3.1-8B-Instruct-InWeight-FPGA-WSP

Finetuned
(3,324)
this model

Dataset used to train masstensor/Llama-3.1-8B-Instruct-InWeight-FPGA-WSP