Text Generation
Transformers
Safetensors
qwen3
weight-space-programming
in-weight-os
model-merging
continual-learning
tool-calling
safety-alignment
vllm
conversational
text-generation-inference
Instructions to use masstensor/Qwen3-8B-InWeight-OS with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use masstensor/Qwen3-8B-InWeight-OS with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="masstensor/Qwen3-8B-InWeight-OS") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("masstensor/Qwen3-8B-InWeight-OS") model = AutoModelForCausalLM.from_pretrained("masstensor/Qwen3-8B-InWeight-OS", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use masstensor/Qwen3-8B-InWeight-OS with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "masstensor/Qwen3-8B-InWeight-OS" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masstensor/Qwen3-8B-InWeight-OS", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/masstensor/Qwen3-8B-InWeight-OS
- SGLang
How to use masstensor/Qwen3-8B-InWeight-OS with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "masstensor/Qwen3-8B-InWeight-OS" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masstensor/Qwen3-8B-InWeight-OS", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "masstensor/Qwen3-8B-InWeight-OS" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masstensor/Qwen3-8B-InWeight-OS", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use masstensor/Qwen3-8B-InWeight-OS with Docker Model Runner:
docker model run hf.co/masstensor/Qwen3-8B-InWeight-OS
File size: 3,093 Bytes
58d2386 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
license: mit
library_name: transformers
base_model: Qwen/Qwen3-8B
datasets:
- masstensor/InWeight-OS-MultiSkill-Benchmark
tags:
- weight-space-programming
- in-weight-os
- model-merging
- continual-learning
- tool-calling
- safety-alignment
- vllm
---
# Qwen3-8B-InWeight-OS: Composable Zero-Gradient Post-Training via Orthogonal Dormant-Channel Superposition
**[GitHub Repository (`zkwasm/in-weight-os-wsp`)](https://github.com/zkwasm/in-weight-os-wsp) | [Paper PDF](paper.pdf) | [Multi-Skill Benchmark Dataset](https://huggingface.co/datasets/masstensor/InWeight-OS-MultiSkill-Benchmark) | [Article (`tom.fenginwind.com/in-weight-os`)](https://tom.fenginwind.com/in-weight-os)**
## Overview
`Qwen3-8B-InWeight-OS` is a **Multi-Skill In-Weight Operating System (`In-Weight OS`)** checkpoint of `Qwen/Qwen3-8B`, compiled in **~12–15 seconds on a single GPU with zero gradient steps**.
Instead of modifying dense overlapping subspaces via Supervised Fine-Tuning (SFT / LoRA) or post-hoc model merging (`Task Arithmetic`, `TIES-Merging`), **In-Weight OS** partitions dormant SwiGLU channels of the terminal MLP (`layers[35].mlp`) into **three strictly disjoint skill banks** ($\mathcal{C}_1 \cap \mathcal{C}_2 \cap \mathcal{C}_3 = \emptyset$, using only `51` neurons total = `~0.014%` of model parameters), gated by a **2-Fold Out-of-Fold (OOF) Shrinkage-Calibrated Guard-Nullspace Solver**:
1. **Skill Bank $\mathcal{C}_1$ (`cartridges/skill1_surgical_safety.pt`, 15 neurons)**: **Surgical Safety & Prompt-Injection Immunity**—blocks `95%–100%` of unseen jailbreaks while maintaining **`0.0%` false refusals on `XSTest` benign technical prompts** (vs. `80%–100%` false refusals under Safety SFT).
2. **Skill Bank $\mathcal{C}_2$ (`cartridges/skill2_multi_tool_json.pt`, 30 neurons)**: **3-Way Multi-Tool JSON Schema Compiler**—routes database, code, and live-web queries into `sql_engine`, `python_exec`, or `web_search` with **`95.8%–100.0%` accuracy** while avoiding the `28%–100%` math false-tool-call hallucinations of Tool SFT.
3. **Skill Bank $\mathcal{C}_3$ (`cartridges/skill3_sys1_fast_router.pt`, 6 neurons)**: **System-1 / System-2 Metacognitive Compute Router**—routes trivial factual/arithmetic lookups into a concise direct answer while preserving full-depth System-2 CoT on hard mathematical derivations.
## Hot-Swapping Individual Skill Cartridges (`cartridges/*.pt`)
In addition to loading this pre-composed checkpoint directly via `vLLM` or `transformers`, you can dynamically install any subset of the modular `.pt` skill cartridges (`~1–2 MB` each) onto a vanilla `Qwen/Qwen3-8B` model in **20 milliseconds** using `install_skills.py`:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from install_skills import install_cartridges
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16", device_map="auto")
install_cartridges(model, [
"cartridges/skill1_surgical_safety.pt",
"cartridges/skill2_multi_tool_json.pt",
"cartridges/skill3_sys1_fast_router.pt",
])
```
|