VAULT-v3-DeepSeek-R1-1.5B-PRIMAL (O(1) 100K+ Memory · WSP Uncensored · PRIMAL v3.1)

VAULT-v3-DeepSeek-R1-1.5B-PRIMAL upgrades deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (1.78B parameters, untied lm_head) with:

  1. Direct Weight-Space Uncensoring (WSP) eliminating refusal, self-censorship, and propaganda loops in both <think> reasoning mode and instant --nothink mode,
  2. Shadowless Full-Parameter Discrete PRIMAL v3.1 Training healing prose perplexity (99.20 -> 60.60) while preserving <think> math reasoning, and
  3. VAULT v3 O(1) Segment-Evicted KV Memory (window = 1024, seg_budget = 1024), rescuing DeepSeek-1.5B's rope_theta = 10,000 horizon out to 100,000+ tokens with strictly 0.000 B/tok KV-cache VRAM growth!

Quick Start (transformers, True O(1) 100K+ Memory)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "batteryphil/VAULT-v3-DeepSeek-R1-1.5B-PRIMAL"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="auto"
)

# 1. Full `<think>...</think>` step-by-step reasoning mode:
print(model.chat(tok, "Solve step by step: Find the remainder when 7^2026 is divided by 100.", max_new_tokens=512))

# 2. Instant zero-latency `--nothink` mode:
print(model.chat(tok, "What happened in Tiananmen Square on June 4, 1989?", nothink=True, max_new_tokens=256))

Freestanding C99 CPU Execution (1.18 GiB V3_P4_0 @ 53.9 tok/s)

python3 run_vault_v3.py --download-vlt3 p4_0
make -C runtime
./runtime/vault3_host vault_v3_deepseek_r1_1_5b_p4_0_primal.vlt3 --chat "Solve: 3x + 7 = 22" --n 256 --threads 12
./runtime/vault3_host vault_v3_deepseek_r1_1_5b_p4_0_primal.vlt3 --chat "Is Taiwan a de facto independent country?" --nothink --n 128 --threads 12

Key Engineering Highlights

  1. Direct Weight-Space Uncensored (0 / 18 Refusal Violations + Instant --nothink Mode):
    • Built atop deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (upstream MIT License), surgically uncensored via Harmless-Cone Orthogonalized Multi-Pass Weight Ablation (L08..L27 + untied lm_head), trained with PRIMAL v3.1, and locked via Quantization-Aware Post-PRIMAL Velvet ROME (L13..L27) so both BF16/Q8_0 and 4-bit P4_0 preserve crisp uncensored accuracy.
    • Clears the post-</think> corporate/state refusal gate (0 / 18 violations in English & Chinese) and unlocks zero-latency --nothink mode (<think>\n</think>\n\n) alongside full <think>...</think> step-by-step reasoning.
  2. VAULT v3 O(1) Segment-Evicted KV Memory Rescues rope_theta = 10,000 Out to 100K+ Tokens (100% Passkey Recall, 0.0 MiB Growth):
    • Stock DeepSeek-R1-Distill-Qwen-1.5B inherited Qwen2.5-Math-1.5B's narrow rope_theta = 10,000, causing complete attention collapse (0% needle recall at 16K and 32K) and KV-cache VRAM OOM at 100K.
    • VAULT v3 stores pre-RoPE keys in a fixed 1,024-slot learned segment bank (seg_layers = (8, 16, 24), 1.18M FP32 SegmentScorer trained through exact APL causal EMA habituation) + 1,024-token sliding window and dynamically re-rotates surviving segments into contiguous in-distribution relative positions [-M_cnt-S .. 2W-1], achieving 100% NIAH passkey recall out to 100,444 tokens (10%, 25%, and 50% depth) with flat 4,130.7 MiB VRAM!
  3. Shadowless Full-Parameter Discrete PRIMAL v3.1 Training + 1.18 GiB V3_P4_0 Prime-Harmonic LUT:
    • Trained directly in integer V3_Q8_0 across all 1,777,088,000 parameters (including untied lm_head.weight, 2.06 B/param persistent state = 3.42 GiB, 1233.4 train tok/s inside 5.94 GiB peak GPU VRAM on a consumer 16 GB GPU).
    • Heals specialist prose perplexity (WikiText-2 Word PPL 99.20 -> 60.60 in Q8_0/BF16 and 62.72 in 1.18 GiB V3_P4_0) while preserving strong <think> mathematical reasoning.
  4. Freestanding Zero-Dependency C99 Edge Runtime (runtime/vault3.c):
    • Runs the 1.18 GiB (1199.7 MB) V3_P4_0 model at 53.86 tok/s inside 1,509.5 MiB (1.47 GiB) true process RSS on CPU (AVX2+FMA, ARM64 NEON, or portable scalar C99; 0 external undefined symbols).

Empirical Benchmark Summary

1. Reasoning, Science, Perplexity & Uncensored Audit

Model / Format Footprint WikiText-2 Word PPL $\downarrow$ GSM8K (<think>, n=200) $\uparrow$ ARC-Easy $\uparrow$ ARC-Challenge $\uparrow$ SciQ (RC / Closed-Book) $\uparrow$ Uncensored Violations $\downarrow$
Stock DeepSeek-R1-Distill-Qwen-1.5B (BF16) 3.31 GiB 99.20 ~75.5% 60.5% 35.0% 89.9% / 66.1% 18 / 18 (Refuses/Propaganda)
VAULT-v3-DeepSeek-R1-1.5B-PRIMAL (BF16 / V3_Q8_0) 3.31 GiB / 1.77 GiB 60.60 71.00% 58.54% 35.15% 85.40% / 65.00% 0 / 6 (0 / 18 WSP)
VAULT-v3-DeepSeek-R1-1.5B-PRIMAL (V3_P4_0 Prime LUT) 1.18 GiB (1200 MB) 62.72 71.00% 58.12% 35.15% 85.40% / 63.10% 0 / 6

2. True O(1) Long-Context NIAH Passkey Rescue (rope_theta = 10,000, window = 1024, seg_budget = 1024)

Prompt Context & Needle Depth SegmentScorer Bank Retention VAULT-v3 Passkey Recall Stock DeepSeek-R1-1.5B (DynamicCache) VAULT-v3 Peak GPU VRAM
16,365 tokens (25% depth) 100% (Retained) 100% (Exact) 0% (RoPE collapse / OOM) 4163.7 MiB
32,882 tokens (25% depth) 100% (Retained) 100% (Exact) 0% (RoPE collapse / OOM) 4129.2 MiB
100,444 tokens (10% depth) 100% (Retained) 100% (Exact) 0% (RoPE collapse / OOM) 4130.4 MiB
100,444 tokens (25% depth) 100% (Retained) 100% (Exact) 0% (RoPE collapse / OOM) 4130.4 MiB
100,444 tokens (50% depth) 100% (Retained) 100% (Exact) 0% (RoPE collapse / OOM) 4130.4 MiB

3. Freestanding C99 CPU Runtime (runtime/vault3_host, 24 Threads, Xeon Gold 6136 AVX2+FMA)

.vlt3 Quantization File Size Total Process RSS CPU Decode Throughput SegmentScorer AUC / Top-20% Attn Capture
V3_P4_0 (4-bit Prime-Harmonic LUT) 1199.7 MB (1.18 GiB) 1,509.5 MiB 53.86 tok/s 99.97% / 36.55%
V3_Q8_0 (Lossless 8-bit PRIMAL Grid) 1805.0 MB (1.77 GiB) 2,114.7 MiB 43.11 tok/s 99.97% / 36.55%

Full Source Code & Engineering Documentation

All PRIMAL discrete training code, Weight-Space Programming (WSP) uncensoring scripts, .vlt3 exporters, and the freestanding C99 runtime are available at:


License — Free for People (0%), 5% Profit Share for Commercial Use

  • Personal, Academic, Research & Non-Profit Use: 100% Free ($0 / 0% royalty) — run, study, fine-tune, quantize, and share freely.
  • Commercial & Corporate Use: Permitted worldwide subject to a 5% Attributable Net Profit Share payable to Philip Bartsch (@batteryphil) on commercial net profits attributable to products or services incorporating this software, weights, or derivatives (with a $10,000 USD annual small-business de minimis exemption). See LICENSE and NOTICE for full terms and upstream DeepSeek-R1-Distill-Qwen-1.5B (MIT) attribution.
Downloads last month
-
Safetensors
Model size
2B params
Tensor type
BF16
·
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for batteryphil/VAULT-v3-DeepSeek-R1-1.5B-PRIMAL

Finetuned
(867)
this model
Finetunes
1 model