VAULT-v3-3B-PRIMAL โ€” Lossless Shadowless Discrete 3B + True $O(1)$ 100K+ Memory

Built with Qwen. Derivative of Qwen/Qwen2.5-3B-Instruct, full-parameter fine-tuned on a single 16 GB GPU (AMD RX 9060 XT) via Lossless Error-Feedback Shadowless Discrete PRIMAL v3.1 (V3_Q8_0, 5.931 GiB persistent state โ€” 2.00ร— smaller than layer-hooked BF16 + AdamW8bit at 11.857 GiB and 7.75ร— smaller than standard FP32-master AdamW at 45.98 GiB) and equipped with VAULT v3 $O(1)$ Segment-Evicted KV Memory.

Important Licensing & trust_remote_code=True Notice (license: other):

  • Runtime Software (AGPL-3.0 when trust_remote_code=True): While modeling_vault_v3.py and configuration_vault_v3.py are Apache-2.0, calling AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True) dynamically imports vault_v3_core.py (GNU AGPL-3.0-or-later). Deploying trust_remote_code=True behind a network endpoint triggers AGPLv3 Section 13 for the VAULT runtime software unless covered by a separate VAULT Commercial Software License (or exported to stock Qwen2ForCausalLM via tools/export_qwen2_compat.py with trust_remote_code=False).
  • Model Weights (Qwen Research License, Non-Commercial): All .safetensors and .vlt3 weights are derived from Qwen/Qwen2.5-3B-Instruct and governed by Alibaba Cloud's Qwen RESEARCH LICENSE AGREEMENT. A VAULT Commercial Software License covers VAULT code only and grants zero commercial rights to Qwen2.5-3B-derived weights without separate commercial licensing from Alibaba Cloud.

Clean Split Repository Layout (5.765 GiB Download, Zero Bloat):

  • This repo (batteryphil/VAULT-v3-3B-PRIMAL): 5.765 GiB transformers BF16 safetensors + SegmentScorer (kept in exact FP32) + cross-platform C99 runtime/ source (x86_64 AVX2 + qemu-aarch64 execution-verified Apple Silicon / ARM64 NEON + scalar C11).
  • Companion .vlt3 binary repo (batteryphil/VAULT-v3-3B-PRIMAL-VLT3): standalone zero-Python .vlt3 weights (1.805 GiB / 1,848.5 MiB V3_P4_0 and 3.064 GiB / 3,137.5 MiB zero-PTQ V3_Q8_0; 4.870 GiB combined). Dequantizing V3_Q8_0 (int8 ร— fp16) into BF16 safetensors introduces minor 7-bit mantissa rounding (93.75% top-1 greedy token agreement vs. exact FP32, 0.16% mean absolute accuracy delta across 5 zero-shot benchmarks and -0.02 WikiText-2 PPL); load vault_v3_3b_q8_0_primal.vlt3 for exact bit-identical V3_Q8_0 weights.

Quick Start

1. Hugging Face transformers (trust_remote_code=True โ€” Full $O(1)$ 100K+ Memory)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "batteryphil/VAULT-v3-3B-PRIMAL"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16
).to("cuda").eval()

print(model.chat(tok, "Solve step by step: A train travels 180 km in 2.5 hours. What is its average speed?"))

2. Zero-Dependency Freestanding C99 (x86_64 AVX2 + Apple Silicon / ARM64 NEON, 2.55 GiB Peak RSS)

make -C runtime
python run_vault_v3.py --download-vlt3 p4_0
./runtime/vault3_host vault_v3_3b_p4_0_primal.vlt3 --chat --prompt "Explain O(1) KV eviction in 3 sentences."

3. Stock vLLM / Ollama / llama.cpp (trust_remote_code=False)

python tools/export_qwen2_compat.py
# Exports ./qwen2_compat (Shards 1 & 2 only; realpath + EXDEV copy fallback).
# WARNING: Stock Qwen2 engines do NOT run VAULT v3 O(1) Segment Memory (standard 32K O(L) KV cache applies).

Full-Split Unconfounded Benchmarks (vs. Both Qwen2.5-3B Base & Instruct, Plus Hooked 8-Bit AdamW Control)

All evaluations use full un-subsampled validation/test splits (GSM8K n=1,319 @ 1,024 tok, HellaSwag n=10,042, LAMBADA n=5,153, BoolQ n=3,270, ARC-E n=2,376, PIQA n=1,838, Winogrande n=1,267, ARC-C n=1,172, SciQ n=1,000). Table columns report exported BF16 safetensors inference (matching AutoModelForCausalLM.from_pretrained(..., dtype=torch.bfloat16)):

Metric / Full Split Qwen2.5-3B Base (BF16) Qwen2.5-3B-Instruct (BF16) Control: Hooked BF16+AdamW8bit (11.86 GiB) VAULT-v3-3B PRIMAL v3.1 (Q8_0 -> BF16) VAULT-v3-3B PRIMAL v3.1 (P4_0, 1.805 GiB)
Persistent Train State โ€” โ€” 11.857 GiB (b=1 req.) 5.931 GiB (2.00ร— smaller, B=2 ga=4) (exported from Q8_0)
GSM8K Full (n=1,319, 1024 tok) โ€” 85.52% 86.20% 85.60% 84.69%
8-Task MC Avg (Full Splits) 74.08% 75.00% 75.30% 77.20% 76.33%
9-Task Zero-Shot Avg (Full Splits) 73.27% 73.99% 74.27% 76.06% 75.37%
WikiText-2 Word PPL (L=2048) 13.14 14.21 14.15 13.58 14.09

(Control note: Hooked BF16+AdamW8bit above ran at lr_peak=1e-5 [400 b=1 micro-steps = 819K tok, BF16 ULP-damped] on the v3.0 off-policy stream because B=2 OOMs on 16 GB VRAM; raising AdamW8bit to lr_peak=2e-4 [246K tok] on that off-policy stream reaches 13.63 WT2 PPL and 72.63% GSM8K, whereas PRIMAL v3.1 trains on 410K tokens with verified on-policy Qwen2.5-3B-Instruct math traces.)

Transparent $O(1)$ Long-Context Architecture & Non-Isomorphic Benchmarks

VAULT v3's $O(1)$ long-context engine (window=1024, seg_budget=1024, max_sink=512) combines three explicit mechanisms:

  1. Learned SegmentScorer MLP (1.59M FP32 params over layers (10, 20, 30)) + APL Causal EMA Habituation (f - 0.65 * ema) for query-agnostic segment retention as chunks leave the active window.
  2. Pre-RoPE Contiguous Slot Re-Rotation ([-M_cnt-S .. 2W-1]): caches pre-RoPE keys (k_seg_raw, k_sink_raw) and dynamically re-rotates surviving segments into contiguous relative slots (<= 3,584 total positions), avoiding Qwen2.5-3B's 32K RoPE horizon limit at any stream length (100Kโ€“10M+ tokens) while trading away exact inter-segment gap distance.
  3. Sparse Head-Gated (2/16 Query Heads = 1 per GQA Group) 2-Tier FNV-1a Shingle / Hop-2 Resonance: applies lexical/2-hop attention logit biases onto only 1 query head per GQA group (2/16 heads), leaving 14/16 (87.5%) of heads completely undistorted for multi-value softmax aggregation (toggleable via enable_shingle_bias=False).
  • 82-Probe Structured/Entity Suite (8Kโ€“100.5K): 79/82 (96.34%) (Q8_0) and 78/82 (95.12%) (P4_0) with Sparse 2/16 Shingle/Hop-2 ON vs. 71/82 (86.59%) with Shingle OFF.
  • External Non-Isomorphic RULER Suite (4Kโ€“64K, natural-language prose + Multi-Key-4 + Multi-Value-4 + 5-Hop Variable Tracking): 83/90 (92.22%) with Sparse 2/16 Shingle ON and 88/90 (97.78%) with Shingle OFF vs. 8/90 (8.89%) with sliding-window + sinks only (seg_budget=0, 0.0% from 8Kโ€“64K).
  • Freestanding C99 8,457-Token 26-Needle Eviction Proof (86.2% eviction pressure): 26/26 (100.0%) with SEG ON vs. 0/26 (0.0%) with --noseg in 2617 MB RSS!
  • Known Scope Limits: (a) Query-agnostic eviction over 100% homogeneous high-entropy dumps (e.g., dense JSON ledgers with zero prose) scales with (2*window + seg_budget) / L; (b) global corpus frequency/word-counting tasks (e.g., RULER CWE/FWE) are out-of-scope because APL EMA (f - 0.65 * ema) intentionally suppresses repetitive tokens.

Split Licensing & Attribution

  • HF Wrapper & Export Tools (configuration_vault_v3.py, modeling_vault_v3.py, run_vault_v3.py, tools/export_qwen2_compat.py): Apache-2.0 (LICENSE-CODE-APACHE-2.0); note that trust_remote_code=True imports AGPL-3.0 vault_v3_core.py.
  • VAULT v3 Core & C99 Runtime (vault_v3_core.py, runtime/*): GNU AGPL-3.0-or-later OR VAULT Commercial Software License (LICENSE-AGPL-3.0, LICENSE-COMMERCIAL.md โ€” covers VAULT software only, not Qwen2.5-3B weights).
  • Model Weights (*.safetensors, *.vlt3): Qwen RESEARCH LICENSE AGREEMENT (LICENSE-QWEN-RESEARCH) โ€” Built with Qwen. Full upstream dataset attribution is provided in NOTICE.
Downloads last month
27
Safetensors
Model size
3B params
Tensor type
BF16
ยท
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for batteryphil/VAULT-v3-3B-PRIMAL

Base model

Qwen/Qwen2.5-3B
Finetuned
(1,697)
this model