Instructions to use batteryphil/VAULT-v3-3B-PRIMAL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use batteryphil/VAULT-v3-3B-PRIMAL with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="batteryphil/VAULT-v3-3B-PRIMAL", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("batteryphil/VAULT-v3-3B-PRIMAL", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use batteryphil/VAULT-v3-3B-PRIMAL with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "batteryphil/VAULT-v3-3B-PRIMAL" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "batteryphil/VAULT-v3-3B-PRIMAL", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/batteryphil/VAULT-v3-3B-PRIMAL
- SGLang
How to use batteryphil/VAULT-v3-3B-PRIMAL with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "batteryphil/VAULT-v3-3B-PRIMAL" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "batteryphil/VAULT-v3-3B-PRIMAL", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "batteryphil/VAULT-v3-3B-PRIMAL" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "batteryphil/VAULT-v3-3B-PRIMAL", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use batteryphil/VAULT-v3-3B-PRIMAL with Docker Model Runner:
docker model run hf.co/batteryphil/VAULT-v3-3B-PRIMAL
- VAULT-v3-3B-PRIMAL โ Lossless Shadowless Discrete 3B + True $O(1)$
100K+Memory
VAULT-v3-3B-PRIMAL โ Lossless Shadowless Discrete 3B + True $O(1)$ 100K+ Memory
Built with Qwen. Derivative of
Qwen/Qwen2.5-3B-Instruct, full-parameter fine-tuned on a single 16 GB GPU (AMD RX 9060 XT) via Lossless Error-Feedback Shadowless Discrete PRIMAL v3.1 (V3_Q8_0,5.931 GiBpersistent state โ2.00รsmaller than layer-hookedBF16 + AdamW8bitat11.857 GiBand7.75รsmaller than standard FP32-master AdamW at45.98 GiB) and equipped with VAULT v3 $O(1)$ Segment-Evicted KV Memory.
Important Licensing &
trust_remote_code=TrueNotice (license: other):
- Runtime Software (
AGPL-3.0whentrust_remote_code=True): Whilemodeling_vault_v3.pyandconfiguration_vault_v3.pyare Apache-2.0, callingAutoModelForCausalLM.from_pretrained(..., trust_remote_code=True)dynamically importsvault_v3_core.py(GNU AGPL-3.0-or-later). Deployingtrust_remote_code=Truebehind a network endpoint triggers AGPLv3 Section 13 for the VAULT runtime software unless covered by a separate VAULT Commercial Software License (or exported to stockQwen2ForCausalLMviatools/export_qwen2_compat.pywithtrust_remote_code=False).- Model Weights (
Qwen Research License, Non-Commercial): All.safetensorsand.vlt3weights are derived fromQwen/Qwen2.5-3B-Instructand governed by Alibaba Cloud's Qwen RESEARCH LICENSE AGREEMENT. A VAULT Commercial Software License covers VAULT code only and grants zero commercial rights toQwen2.5-3B-derived weights without separate commercial licensing from Alibaba Cloud.
Clean Split Repository Layout (
5.765 GiBDownload, Zero Bloat):
- This repo (
batteryphil/VAULT-v3-3B-PRIMAL):5.765 GiBtransformersBF16safetensors +SegmentScorer(kept in exactFP32) + cross-platform C99runtime/source (x86_64 AVX2+qemu-aarch64execution-verifiedApple Silicon / ARM64 NEON+ scalar C11).- Companion
.vlt3binary repo (batteryphil/VAULT-v3-3B-PRIMAL-VLT3): standalone zero-Python.vlt3weights (1.805 GiB/1,848.5 MiBV3_P4_0and3.064 GiB/3,137.5 MiBzero-PTQV3_Q8_0;4.870 GiBcombined). DequantizingV3_Q8_0(int8 ร fp16) intoBF16safetensors introduces minor 7-bit mantissa rounding (93.75%top-1 greedy token agreement vs. exactFP32,0.16%mean absolute accuracy delta across 5 zero-shot benchmarks and-0.02WikiText-2 PPL); loadvault_v3_3b_q8_0_primal.vlt3for exact bit-identicalV3_Q8_0weights.
Quick Start
1. Hugging Face transformers (trust_remote_code=True โ Full $O(1)$ 100K+ Memory)
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "batteryphil/VAULT-v3-3B-PRIMAL"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo, trust_remote_code=True, dtype=torch.bfloat16
).to("cuda").eval()
print(model.chat(tok, "Solve step by step: A train travels 180 km in 2.5 hours. What is its average speed?"))
2. Zero-Dependency Freestanding C99 (x86_64 AVX2 + Apple Silicon / ARM64 NEON, 2.55 GiB Peak RSS)
make -C runtime
python run_vault_v3.py --download-vlt3 p4_0
./runtime/vault3_host vault_v3_3b_p4_0_primal.vlt3 --chat --prompt "Explain O(1) KV eviction in 3 sentences."
3. Stock vLLM / Ollama / llama.cpp (trust_remote_code=False)
python tools/export_qwen2_compat.py
# Exports ./qwen2_compat (Shards 1 & 2 only; realpath + EXDEV copy fallback).
# WARNING: Stock Qwen2 engines do NOT run VAULT v3 O(1) Segment Memory (standard 32K O(L) KV cache applies).
Full-Split Unconfounded Benchmarks (vs. Both Qwen2.5-3B Base & Instruct, Plus Hooked 8-Bit AdamW Control)
All evaluations use full un-subsampled validation/test splits (GSM8K n=1,319 @ 1,024 tok, HellaSwag n=10,042, LAMBADA n=5,153, BoolQ n=3,270, ARC-E n=2,376, PIQA n=1,838, Winogrande n=1,267, ARC-C n=1,172, SciQ n=1,000). Table columns report exported BF16 safetensors inference (matching AutoModelForCausalLM.from_pretrained(..., dtype=torch.bfloat16)):
| Metric / Full Split | Qwen2.5-3B Base (BF16) |
Qwen2.5-3B-Instruct (BF16) |
Control: Hooked BF16+AdamW8bit (11.86 GiB) |
VAULT-v3-3B PRIMAL v3.1 (Q8_0 -> BF16) |
VAULT-v3-3B PRIMAL v3.1 (P4_0, 1.805 GiB) |
|---|---|---|---|---|---|
| Persistent Train State | โ | โ | 11.857 GiB (b=1 req.) |
5.931 GiB (2.00ร smaller, B=2 ga=4) |
(exported from Q8_0) |
GSM8K Full (n=1,319, 1024 tok) |
โ | 85.52% |
86.20% |
85.60% |
84.69% |
| 8-Task MC Avg (Full Splits) | 74.08% |
75.00% |
75.30% |
77.20% |
76.33% |
| 9-Task Zero-Shot Avg (Full Splits) | 73.27% |
73.99% |
74.27% |
76.06% |
75.37% |
WikiText-2 Word PPL (L=2048) |
13.14 |
14.21 |
14.15 |
13.58 |
14.09 |
(Control note: Hooked BF16+AdamW8bit above ran at lr_peak=1e-5 [400 b=1 micro-steps = 819K tok, BF16 ULP-damped] on the v3.0 off-policy stream because B=2 OOMs on 16 GB VRAM; raising AdamW8bit to lr_peak=2e-4 [246K tok] on that off-policy stream reaches 13.63 WT2 PPL and 72.63% GSM8K, whereas PRIMAL v3.1 trains on 410K tokens with verified on-policy Qwen2.5-3B-Instruct math traces.)
Transparent $O(1)$ Long-Context Architecture & Non-Isomorphic Benchmarks
VAULT v3's $O(1)$ long-context engine (window=1024, seg_budget=1024, max_sink=512) combines three explicit mechanisms:
- Learned
SegmentScorerMLP (1.59MFP32 params over layers(10, 20, 30)) + APL Causal EMA Habituation (f - 0.65 * ema) for query-agnostic segment retention as chunks leave the active window. - Pre-RoPE Contiguous Slot Re-Rotation (
[-M_cnt-S .. 2W-1]): caches pre-RoPE keys (k_seg_raw,k_sink_raw) and dynamically re-rotates surviving segments into contiguous relative slots (<= 3,584total positions), avoidingQwen2.5-3B's32KRoPE horizon limit at any stream length (100Kโ10M+tokens) while trading away exact inter-segment gap distance. - Sparse Head-Gated (
2/16Query Heads =1per GQA Group) 2-Tier FNV-1a Shingle / Hop-2 Resonance: applies lexical/2-hop attention logit biases onto only 1 query head per GQA group (2/16heads), leaving14/16(87.5%) of heads completely undistorted for multi-value softmax aggregation (toggleable viaenable_shingle_bias=False).
- 82-Probe Structured/Entity Suite (
8Kโ100.5K):79/82(96.34%) (Q8_0) and78/82(95.12%) (P4_0) with Sparse2/16Shingle/Hop-2 ON vs.71/82(86.59%) with Shingle OFF. - External Non-Isomorphic RULER Suite (
4Kโ64K, natural-language prose + Multi-Key-4 + Multi-Value-4 + 5-Hop Variable Tracking):83/90(92.22%) with Sparse2/16Shingle ON and88/90(97.78%) with Shingle OFF vs.8/90(8.89%) with sliding-window + sinks only (seg_budget=0,0.0%from8Kโ64K). - Freestanding C99
8,457-Token26-Needle Eviction Proof (86.2%eviction pressure):26/26(100.0%) withSEG ONvs.0/26(0.0%) with--nosegin2617 MBRSS! - Known Scope Limits: (a) Query-agnostic eviction over 100% homogeneous high-entropy dumps (e.g., dense JSON ledgers with zero prose) scales with
(2*window + seg_budget) / L; (b) global corpus frequency/word-counting tasks (e.g., RULER CWE/FWE) are out-of-scope because APL EMA (f - 0.65 * ema) intentionally suppresses repetitive tokens.
Split Licensing & Attribution
- HF Wrapper & Export Tools (
configuration_vault_v3.py,modeling_vault_v3.py,run_vault_v3.py,tools/export_qwen2_compat.py): Apache-2.0 (LICENSE-CODE-APACHE-2.0); note thattrust_remote_code=Trueimports AGPL-3.0vault_v3_core.py. - VAULT v3 Core & C99 Runtime (
vault_v3_core.py,runtime/*): GNU AGPL-3.0-or-later OR VAULT Commercial Software License (LICENSE-AGPL-3.0,LICENSE-COMMERCIAL.mdโ covers VAULT software only, not Qwen2.5-3B weights). - Model Weights (
*.safetensors,*.vlt3): Qwen RESEARCH LICENSE AGREEMENT (LICENSE-QWEN-RESEARCH) โ Built with Qwen. Full upstream dataset attribution is provided inNOTICE.
- Downloads last month
- 27