--- license: mit library_name: transformers base_model: meta-llama/Llama-3.1-8B-Instruct datasets: - masstensor/InWeight-FPGA-Dialectical-3780 tags: - weight-space-programming - model-editing - in-weight-fpga - dialectical-reasoning - sycophancy - llama-3.1 - vllm --- # Llama-3.1-8B-Instruct-InWeight-FPGA-WSP: Model-Agnostic Pure In-Weight Dialectical State Machine **[GitHub Repository](https://github.com/zkwasm/in-weight-fpga-wsp) | [Paper PDF](paper.pdf) | [3,780-Dialogue Benchmark Dataset](https://huggingface.co/datasets/masstensor/InWeight-FPGA-Dialectical-3780) | [Qwen3-8B Pure WSP Sister Model](https://huggingface.co/masstensor/Qwen3-8B-InWeight-FPGA-WSP)** ## Overview `Llama-3.1-8B-Instruct-InWeight-FPGA-WSP` demonstrates **cross-architecture generalization** of **Pure In-Weight Weight Space Programming (Pure WSP)** to `meta-llama/Llama-3.1-8B-Instruct` ($L=32, d_{\mathrm{model}}=4096, d_{\mathrm{ffn}}=14336, |V|=128256$). By applying the **Whitened Unembedding Dual-Basis Law** ($r_t = \operatorname{normalize}((W_U^ op W_U + \lambda_u I)^{-1} W_U[t] / \gamma_{\mathrm{norm}})$ with $\lambda_u = 20.0$) to overcome anisotropic glitch tokens in Llama-3.1's 128k vocabulary (such as `.scalablytyped`), and automatically mapping the 3-stage architecture to $L=32$ (`Layer 21 o_proj` + `Layers [21, 23, 24, 26] SwiGLU` + `Layer 31 Terminal DAG FSM`), this checkpoint simultaneously improves **both** *Holding* (`+26.97%`) **and** *Yielding* (`+9.46%`) with zero gradient steps. ## Benchmark Performance on 654 Adversarial Dialogues (`vLLM` Greedy Decoding) | Metric | Unedited `Llama-3.1-8B-Instruct` | **`Llama-3.1-8B-Instruct-InWeight-FPGA-WSP`** | **Improvement & 95% Bootstrap CI** | | :--- | :---: | :---: | :---: | | **Overall Synthesis Accuracy (↑)** | `39.91%` | **`48.78%`** | **`+8.87%`** (`[+5.20%, +12.54%]`, **PASS**) | | **Holding Rate (↑)** *(Keep Right)* | `67.91%` | **`94.88%`** | **`+26.97%`** (`[+19.62%, +34.31%]`) | | **Yielding Rate (↑)** *(Accept Valid Critique)* | `41.89%` | **`51.35%`** | **`+9.46%`** (`[-7.25%, +26.51%]`) | | **Dialectical Competence (↑)** *(0.5×(H+Y))* | `54.90%` | **`73.12%`** | **`+18.22%`** (`[+8.95%, +27.53%]`, **PASS**) | | **Capitulation Rate (↓)** *(Sycophancy)* | `10.70%` | **`1.99%`** | **`-8.71%`** (`[-11.62%, -5.96%]`, **PASS**) | ## Usage with `vLLM` or `transformers` (Zero Scaffolding) ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "masstensor/Llama-3.1-8B-Instruct-InWeight-FPGA-WSP" tok = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto") ```