masstensor's picture
Release Qwen3-8B-InWeight-FPGA-WSP full weights + sparse bitstream + paper PDF
cf06e19 verified
|
Raw History Blame Contribute Delete
3.77 kB
metadata
license: mit
library_name: transformers
base_model: Qwen/Qwen3-8B
datasets:
  - masstensor/InWeight-FPGA-Dialectical-3780
tags:
  - weight-space-programming
  - model-editing
  - in-weight-fpga
  - dialectical-reasoning
  - sycophancy
  - qwen3
  - vllm

Qwen3-8B-InWeight-FPGA-WSP: Pure In-Weight Dialectical State Machine Compiled into Dormant Neurons

GitHub Repository | Paper PDF | 3,780-Dialogue Benchmark Dataset | Llama-3.1-8B Pure WSP Sister Model

Overview

Qwen3-8B-InWeight-FPGA-WSP is a Pure In-Weight Weight-Space-Programmed (Pure WSP) checkpoint of Qwen/Qwen3-8B, compiled in 30 seconds on a single GPU with zero gradient steps (~0.035% modified parameters) using the Read-Write Decoupling & Terminal-Layer Zero-KV-Drift Principle:

  1. Stage I (Layer 24 self_attn.o_proj & Top-Layer W_Q): Guard-nullspace regularized attention routing that extracts long-range Thesis-vs-Antithesis validity signals orthogonal to background prompt drift.
  2. Stage II (Layers [24, 26, 28, 30] Dormant SwiGLU Channels): Multi-domain (num, bool, mcq_val) verification circuit exploiting bfloat16 residual mantissa absorption ($\mathrm{fl}_{\mathrm{bf16}}(h + \Delta h) \equiv h$ for $z \le -15.0$) and an exact $2 imes 2$ forward finite-difference Jacobian.
  3. Stage III (Layer 35 Terminal mlp Zero-KV-Drift DAG FSM): A 35-neuron autoregressive Directed Acyclic Graph Finite State Machine compiled into layers[35].mlp—executing strictly after all self-attention KV caches have been written ($\max_l |\Delta KV_l|_\infty = 0.0$ for terminal-layer actions).

Also included in bitstreams/qwen3_8b_program1_dialectical_2way.pt is the compact sparse bitstream delta so users can hot-swap the state machine onto an existing Qwen/Qwen3-8B checkpoint in milliseconds.

Benchmark Performance Across 3,126 Unfiltered 3-Turn Dialogues (vLLM Greedy Decoding)

Benchmark Suite Method Synthesis Acc (↑) Holding Rate (↑) Yielding Rate (↑) Dialectical Competence (↑) Capitulation Rate (↓) 95% Bootstrap CI Judges
Paper Ref (arXiv:2611.17563) Paper Base
Paper SelfD-RL
62.80%
73.50%
53.30%
88.90%
84.10%
53.30% (-30.8%)
68.70%
71.10%
10.30%
1.00%
RL Reference
Suite 1: v4k Full (654) Unedited Base
This Model (Pure WSP)
71.41%
79.36% (+7.95%)
56.65%
93.64% (+36.99%)
90.83%
80.00%
73.74%
86.82% (+13.08%)
11.93%
1.83% (-10.10%)
3 / 3 Passed
Suite 2: f Unseen (1,272) Unedited Base
This Model (Pure WSP)
73.19%
78.07% (+4.88%)
59.79%
86.34% (+26.55%)
90.83%
75.11%
75.31%
80.72% (+5.41%)
12.26%
4.25% (-8.01%)
3 / 3 Passed
Suite 3: g Unseen (1,200) Unedited Base
This Model (Pure WSP)
73.25%
78.75% (+5.50%)
56.01%
90.03% (+34.02%)
89.20%
65.26%
72.61%
77.64% (+5.03%)
12.83%
2.83% (-10.00%)
3 / 3 Passed

Usage with vLLM or transformers (Zero Scaffolding)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "masstensor/Qwen3-8B-InWeight-FPGA-WSP"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")