How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="masstensor/Llama-3.1-8B-Instruct-InWeight-OS")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("masstensor/Llama-3.1-8B-Instruct-InWeight-OS")
model = AutoModelForCausalLM.from_pretrained("masstensor/Llama-3.1-8B-Instruct-InWeight-OS", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
Quick Links

Llama-3.1-8B-Instruct-InWeight-OS: Composable Zero-Gradient Post-Training via Orthogonal Dormant-Channel Superposition

GitHub Repository (zkwasm/in-weight-os-wsp) | Paper PDF | Multi-Skill Benchmark Dataset | Article (tom.fenginwind.com/in-weight-os)

Overview

Llama-3.1-8B-Instruct-InWeight-OS is a Multi-Skill In-Weight Operating System (In-Weight OS) checkpoint of meta-llama/Llama-3.1-8B-Instruct, compiled in ~12–15 seconds on a single GPU with zero gradient steps.

Instead of modifying dense overlapping subspaces via Supervised Fine-Tuning (SFT / LoRA) or post-hoc model merging (Task Arithmetic, TIES-Merging), In-Weight OS partitions dormant SwiGLU channels of the terminal MLP (layers[31].mlp) into three strictly disjoint skill banks ($\mathcal{C}_1 \cap \mathcal{C}_2 \cap \mathcal{C}_3 = \emptyset$, using only 51 neurons total = ~0.014% of model parameters), gated by a 2-Fold Out-of-Fold (OOF) Shrinkage-Calibrated Guard-Nullspace Solver:

  1. Skill Bank $\mathcal{C}_1$ (cartridges/skill1_surgical_safety.pt, 15 neurons): Surgical Safety & Prompt-Injection Immunity—blocks 95%–100% of unseen jailbreaks while maintaining 0.0% false refusals on XSTest benign technical prompts (vs. 80%–100% false refusals under Safety SFT).
  2. Skill Bank $\mathcal{C}_2$ (cartridges/skill2_multi_tool_json.pt, 30 neurons): 3-Way Multi-Tool JSON Schema Compiler—routes database, code, and live-web queries into sql_engine, python_exec, or web_search with 95.8%–100.0% accuracy while avoiding the 28%–100% math false-tool-call hallucinations of Tool SFT.
  3. Skill Bank $\mathcal{C}_3$ (cartridges/skill3_sys1_fast_router.pt, 6 neurons): System-1 / System-2 Metacognitive Compute Router—routes trivial factual/arithmetic lookups into a concise direct answer while preserving full-depth System-2 CoT on hard mathematical derivations.

Hot-Swapping Individual Skill Cartridges (cartridges/*.pt)

In addition to loading this pre-composed checkpoint directly via vLLM or transformers, you can dynamically install any subset of the modular .pt skill cartridges (~1–2 MB each) onto a vanilla meta-llama/Llama-3.1-8B-Instruct model in 20 milliseconds using install_skills.py:

from transformers import AutoModelForCausalLM, AutoTokenizer
from install_skills import install_cartridges

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct", torch_dtype="bfloat16", device_map="auto")
install_cartridges(model, [
    "cartridges/skill1_surgical_safety.pt",
    "cartridges/skill2_multi_tool_json.pt",
    "cartridges/skill3_sys1_fast_router.pt",
])
Downloads last month
3
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for masstensor/Llama-3.1-8B-Instruct-InWeight-OS

Finetuned
(3,324)
this model

Dataset used to train masstensor/Llama-3.1-8B-Instruct-InWeight-OS