Text Generation
MLX
Safetensors
English
Chinese
qwen3_5
mixed-precision
coding
tool-use
experimental
conversational
4-bit precision
Instructions to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download quantization_recipe.json from Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced: direct link, hf CLI and curl.
- Browser
- Download file 35.8 kB
-
https://huggingface.co/Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced/resolve/main/quantization_recipe.json
- Command line
-
hf download hf://Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced/quantization_recipe.json
-
curl -L -o quantization_recipe.json https://huggingface.co/Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced/resolve/main/quantization_recipe.json
35.8 kB
{
"module_precision": {
"language_model.lm_head": [
8,
64
],
"language_model.model.embed_tokens": [
8,
64
],
"language_model.model.layers.0.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.0.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.0.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.0.mlp.down_proj": [
4,
64
],
"language_model.model.layers.0.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.0.mlp.up_proj": [
4,
32
],
"language_model.model.layers.1.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.1.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.1.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.1.mlp.down_proj": [
4,
64
],
"language_model.model.layers.1.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.1.mlp.up_proj": [
4,
32
],
"language_model.model.layers.10.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.10.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.10.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.10.mlp.down_proj": [
4,
64
],
"language_model.model.layers.10.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.10.mlp.up_proj": [
4,
32
],
"language_model.model.layers.11.mlp.down_proj": [
4,
64
],
"language_model.model.layers.11.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.11.mlp.up_proj": [
4,
32
],
"language_model.model.layers.11.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.11.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.11.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.11.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.12.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.12.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.12.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.12.mlp.down_proj": [
4,
64
],
"language_model.model.layers.12.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.12.mlp.up_proj": [
4,
32
],
"language_model.model.layers.13.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.13.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.13.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.13.mlp.down_proj": [
4,
64
],
"language_model.model.layers.13.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.13.mlp.up_proj": [
4,
32
],
"language_model.model.layers.14.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.14.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.14.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.14.mlp.down_proj": [
4,
32
],
"language_model.model.layers.14.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.14.mlp.up_proj": [
4,
32
],
"language_model.model.layers.15.mlp.down_proj": [
4,
32
],
"language_model.model.layers.15.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.15.mlp.up_proj": [
4,
32
],
"language_model.model.layers.15.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.15.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.15.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.15.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.16.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.16.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.16.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.16.mlp.down_proj": [
4,
32
],
"language_model.model.layers.16.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.16.mlp.up_proj": [
4,
32
],
"language_model.model.layers.17.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.17.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.17.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.17.mlp.down_proj": [
4,
32
],
"language_model.model.layers.17.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.17.mlp.up_proj": [
4,
32
],
"language_model.model.layers.18.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.18.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.18.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.18.mlp.down_proj": [
4,
32
],
"language_model.model.layers.18.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.18.mlp.up_proj": [
4,
32
],
"language_model.model.layers.19.mlp.down_proj": [
4,
32
],
"language_model.model.layers.19.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.19.mlp.up_proj": [
4,
32
],
"language_model.model.layers.19.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.19.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.19.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.19.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.2.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.2.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.2.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.2.mlp.down_proj": [
4,
64
],
"language_model.model.layers.2.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.2.mlp.up_proj": [
4,
32
],
"language_model.model.layers.20.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.20.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.20.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.20.mlp.down_proj": [
4,
32
],
"language_model.model.layers.20.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.20.mlp.up_proj": [
4,
32
],
"language_model.model.layers.21.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.21.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.21.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.21.mlp.down_proj": [
4,
32
],
"language_model.model.layers.21.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.21.mlp.up_proj": [
4,
32
],
"language_model.model.layers.22.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.22.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.22.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.22.mlp.down_proj": [
4,
32
],
"language_model.model.layers.22.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.22.mlp.up_proj": [
4,
32
],
"language_model.model.layers.23.mlp.down_proj": [
4,
32
],
"language_model.model.layers.23.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.23.mlp.up_proj": [
4,
32
],
"language_model.model.layers.23.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.23.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.23.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.23.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.24.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.24.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.24.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.24.mlp.down_proj": [
4,
32
],
"language_model.model.layers.24.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.24.mlp.up_proj": [
4,
32
],
"language_model.model.layers.25.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.25.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.25.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.25.mlp.down_proj": [
4,
32
],
"language_model.model.layers.25.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.25.mlp.up_proj": [
4,
32
],
"language_model.model.layers.26.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.26.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.26.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.26.mlp.down_proj": [
4,
32
],
"language_model.model.layers.26.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.26.mlp.up_proj": [
4,
32
],
"language_model.model.layers.27.mlp.down_proj": [
4,
32
],
"language_model.model.layers.27.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.27.mlp.up_proj": [
4,
32
],
"language_model.model.layers.27.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.27.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.27.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.27.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.28.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.28.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.28.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.28.mlp.down_proj": [
4,
32
],
"language_model.model.layers.28.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.28.mlp.up_proj": [
4,
32
],
"language_model.model.layers.29.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.29.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.29.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.29.mlp.down_proj": [
4,
32
],
"language_model.model.layers.29.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.29.mlp.up_proj": [
4,
32
],
"language_model.model.layers.3.mlp.down_proj": [
4,
64
],
"language_model.model.layers.3.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.3.mlp.up_proj": [
4,
32
],
"language_model.model.layers.3.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.3.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.3.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.3.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.30.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.30.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.30.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.30.mlp.down_proj": [
4,
32
],
"language_model.model.layers.30.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.30.mlp.up_proj": [
4,
32
],
"language_model.model.layers.31.mlp.down_proj": [
4,
32
],
"language_model.model.layers.31.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.31.mlp.up_proj": [
4,
32
],
"language_model.model.layers.31.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.31.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.31.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.31.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.32.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.32.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.32.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.32.mlp.down_proj": [
4,
32
],
"language_model.model.layers.32.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.32.mlp.up_proj": [
4,
32
],
"language_model.model.layers.33.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.33.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.33.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.33.mlp.down_proj": [
4,
32
],
"language_model.model.layers.33.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.33.mlp.up_proj": [
4,
32
],
"language_model.model.layers.34.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.34.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.34.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.34.mlp.down_proj": [
4,
32
],
"language_model.model.layers.34.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.34.mlp.up_proj": [
4,
32
],
"language_model.model.layers.35.mlp.down_proj": [
4,
32
],
"language_model.model.layers.35.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.35.mlp.up_proj": [
4,
32
],
"language_model.model.layers.35.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.35.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.35.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.35.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.36.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.36.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.36.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.36.mlp.down_proj": [
4,
32
],
"language_model.model.layers.36.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.36.mlp.up_proj": [
4,
32
],
"language_model.model.layers.37.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.37.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.37.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.37.mlp.down_proj": [
4,
32
],
"language_model.model.layers.37.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.37.mlp.up_proj": [
4,
32
],
"language_model.model.layers.38.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.38.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.38.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.38.mlp.down_proj": [
4,
32
],
"language_model.model.layers.38.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.38.mlp.up_proj": [
4,
32
],
"language_model.model.layers.39.mlp.down_proj": [
4,
32
],
"language_model.model.layers.39.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.39.mlp.up_proj": [
4,
32
],
"language_model.model.layers.39.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.39.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.39.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.39.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.4.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.4.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.4.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.4.mlp.down_proj": [
4,
64
],
"language_model.model.layers.4.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.4.mlp.up_proj": [
4,
32
],
"language_model.model.layers.40.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.40.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.40.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.40.mlp.down_proj": [
4,
32
],
"language_model.model.layers.40.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.40.mlp.up_proj": [
4,
32
],
"language_model.model.layers.41.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.41.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.41.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.41.mlp.down_proj": [
4,
32
],
"language_model.model.layers.41.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.41.mlp.up_proj": [
4,
32
],
"language_model.model.layers.42.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.42.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.42.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.42.mlp.down_proj": [
4,
32
],
"language_model.model.layers.42.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.42.mlp.up_proj": [
4,
32
],
"language_model.model.layers.43.mlp.down_proj": [
4,
32
],
"language_model.model.layers.43.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.43.mlp.up_proj": [
4,
32
],
"language_model.model.layers.43.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.43.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.43.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.43.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.44.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.44.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.44.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.44.mlp.down_proj": [
4,
32
],
"language_model.model.layers.44.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.44.mlp.up_proj": [
4,
32
],
"language_model.model.layers.45.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.45.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.45.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.45.mlp.down_proj": [
4,
32
],
"language_model.model.layers.45.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.45.mlp.up_proj": [
4,
32
],
"language_model.model.layers.46.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.46.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.46.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.46.mlp.down_proj": [
4,
32
],
"language_model.model.layers.46.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.46.mlp.up_proj": [
4,
32
],
"language_model.model.layers.47.mlp.down_proj": [
4,
32
],
"language_model.model.layers.47.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.47.mlp.up_proj": [
4,
32
],
"language_model.model.layers.47.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.47.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.47.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.47.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.48.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.48.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.48.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.48.mlp.down_proj": [
4,
32
],
"language_model.model.layers.48.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.48.mlp.up_proj": [
4,
32
],
"language_model.model.layers.49.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.49.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.49.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.49.mlp.down_proj": [
4,
32
],
"language_model.model.layers.49.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.49.mlp.up_proj": [
4,
32
],
"language_model.model.layers.5.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.5.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.5.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.5.mlp.down_proj": [
4,
64
],
"language_model.model.layers.5.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.5.mlp.up_proj": [
4,
32
],
"language_model.model.layers.50.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.50.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.50.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.50.mlp.down_proj": [
4,
32
],
"language_model.model.layers.50.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.50.mlp.up_proj": [
4,
32
],
"language_model.model.layers.51.mlp.down_proj": [
4,
32
],
"language_model.model.layers.51.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.51.mlp.up_proj": [
4,
32
],
"language_model.model.layers.51.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.51.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.51.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.51.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.52.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.52.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.52.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.52.mlp.down_proj": [
4,
32
],
"language_model.model.layers.52.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.52.mlp.up_proj": [
4,
32
],
"language_model.model.layers.53.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.53.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.53.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.53.mlp.down_proj": [
4,
32
],
"language_model.model.layers.53.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.53.mlp.up_proj": [
4,
32
],
"language_model.model.layers.54.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.54.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.54.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.54.mlp.down_proj": [
4,
32
],
"language_model.model.layers.54.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.54.mlp.up_proj": [
4,
32
],
"language_model.model.layers.55.mlp.down_proj": [
4,
32
],
"language_model.model.layers.55.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.55.mlp.up_proj": [
4,
32
],
"language_model.model.layers.55.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.55.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.55.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.55.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.56.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.56.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.56.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.56.mlp.down_proj": [
8,
64
],
"language_model.model.layers.56.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.56.mlp.up_proj": [
8,
64
],
"language_model.model.layers.57.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.57.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.57.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.57.mlp.down_proj": [
8,
64
],
"language_model.model.layers.57.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.57.mlp.up_proj": [
8,
64
],
"language_model.model.layers.58.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.58.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.58.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.58.mlp.down_proj": [
8,
64
],
"language_model.model.layers.58.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.58.mlp.up_proj": [
8,
64
],
"language_model.model.layers.59.mlp.down_proj": [
8,
64
],
"language_model.model.layers.59.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.59.mlp.up_proj": [
8,
64
],
"language_model.model.layers.59.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.59.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.59.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.59.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.6.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.6.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.6.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.6.mlp.down_proj": [
4,
64
],
"language_model.model.layers.6.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.6.mlp.up_proj": [
4,
32
],
"language_model.model.layers.60.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.60.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.60.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.60.mlp.down_proj": [
8,
64
],
"language_model.model.layers.60.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.60.mlp.up_proj": [
8,
64
],
"language_model.model.layers.61.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.61.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.61.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.61.mlp.down_proj": [
8,
64
],
"language_model.model.layers.61.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.61.mlp.up_proj": [
8,
64
],
"language_model.model.layers.62.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.62.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.62.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.62.mlp.down_proj": [
8,
64
],
"language_model.model.layers.62.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.62.mlp.up_proj": [
8,
64
],
"language_model.model.layers.63.mlp.down_proj": [
8,
64
],
"language_model.model.layers.63.mlp.gate_proj": [
8,
64
],
"language_model.model.layers.63.mlp.up_proj": [
8,
64
],
"language_model.model.layers.63.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.63.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.63.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.63.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.7.mlp.down_proj": [
4,
64
],
"language_model.model.layers.7.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.7.mlp.up_proj": [
4,
32
],
"language_model.model.layers.7.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.7.self_attn.o_proj": [
8,
64
],
"language_model.model.layers.7.self_attn.q_proj": [
8,
64
],
"language_model.model.layers.7.self_attn.v_proj": [
8,
64
],
"language_model.model.layers.8.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.8.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.8.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.8.mlp.down_proj": [
4,
64
],
"language_model.model.layers.8.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.8.mlp.up_proj": [
4,
32
],
"language_model.model.layers.9.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.9.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.9.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.9.mlp.down_proj": [
4,
64
],
"language_model.model.layers.9.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.9.mlp.up_proj": [
4,
32
]
},
"module_counts": {
"8bit-g64": 234,
"4bit-g64": 14,
"4bit-g32": 154
},
"bf16_recurrence_projections": 96,
"calibration_tokens": 2469,
"calibration_method": "BF16 input second moments; activation-weighted MLP error ranking; gate/up paired",
"edit": {
"source_index_sha256": "77042094076611b69791a610065f28b7013b8c621795fa86ddccc8bac7d1b9df",
"direction_sha256": "22bcd1e29c9c24ce3580b595d9f6dcaed3f0d471d161df39a582afd4f580d4d3",
"strength": 0.5,
"layers": [
10,
11,
12,
13,
14,
15,
16,
17
],
"edits": [
{
"tensor": "model.language_model.layers.10.linear_attn.out_proj.weight",
"before_sha256": "ddc9f40eb88b16bbe7020dba9a91a441fc5c18b057ebbd6223b40f8d7120c550",
"after_sha256": "5bdc573273f4a1c41eca30115641606d919f8eb7a2a5569dbedc801703f46c2b"
},
{
"tensor": "model.language_model.layers.11.self_attn.o_proj.weight",
"before_sha256": "cdf7e56ea0f52aa9c0be1cfb1a60fa2578a4397ab9ad1a4ecbcb1ec262915418",
"after_sha256": "844848bc7e38c8cd65d04e8fb745ac92b5c27f836fbb85b299366fc76707f13a"
},
{
"tensor": "model.language_model.layers.12.linear_attn.out_proj.weight",
"before_sha256": "fa2ec7ea29de6bdc055077e54e9569ccea2aba5c4c9290203b6781b91a0a5898",
"after_sha256": "65a415198ca96a99785994c3825708f92ab894a2fa547c78dd8ca8e4ff0b03a5"
},
{
"tensor": "model.language_model.layers.13.linear_attn.out_proj.weight",
"before_sha256": "6bab3392b54bf4f3fa192864cdaa916542a0e39082018d78849b0b902fc57d6a",
"after_sha256": "570764e7ccbdce6b778eabfb8d931b24e3fa550308d9b0a0cb1cbab3bf36109d"
},
{
"tensor": "model.language_model.layers.14.linear_attn.out_proj.weight",
"before_sha256": "b66bcce1b7bbf3f2d257ac5b1907ffa19d913eeb434d197f7c193cd6d950263a",
"after_sha256": "1257e190de1d82cf9e19d49eb067ffdfa144ce8b31a4f34794f7e676b3f1a7d4"
},
{
"tensor": "model.language_model.layers.15.self_attn.o_proj.weight",
"before_sha256": "b481b2613da9131959be949713d4fbdfd251d87f6e88cd085baa7a43ba5acb70",
"after_sha256": "15d66542152b54091fb39c7a89d60e7f539168258390a819b8dd8587f2d68636"
},
{
"tensor": "model.language_model.layers.16.linear_attn.out_proj.weight",
"before_sha256": "1dc2a73adb9d251315a2ed1b59275644aace902ba6c0ddfbb73123e42b95818b",
"after_sha256": "1b2dcf008a192de27de214dc33fb197876ad01cddc5c1b77e8b9ebe74e98c537"
},
{
"tensor": "model.language_model.layers.17.linear_attn.out_proj.weight",
"before_sha256": "6af44817ebcc60d20155d238acf8a480b228491f59c5b7cbb01889fd46a30766",
"after_sha256": "c0fbb2d3d8d2d98ae341bbce03d4cf09dcbae45828f5247d0866142761ac0612"
}
]
}
}
{
"module_precision": {
"language_model.lm_head": [
8,
64
],
"language_model.model.embed_tokens": [
8,
64
],
"language_model.model.layers.0.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.0.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.0.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.0.mlp.down_proj": [
4,
64
],
"language_model.model.layers.0.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.0.mlp.up_proj": [
4,
32
],
"language_model.model.layers.1.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.1.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.1.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.1.mlp.down_proj": [
4,
64
],
"language_model.model.layers.1.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.1.mlp.up_proj": [
4,
32
],
"language_model.model.layers.10.linear_attn.in_proj_qkv": [
8,
64
],
"language_model.model.layers.10.linear_attn.in_proj_z": [
8,
64
],
"language_model.model.layers.10.linear_attn.out_proj": [
8,
64
],
"language_model.model.layers.10.mlp.down_proj": [
4,
64
],
"language_model.model.layers.10.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.10.mlp.up_proj": [
4,
32
],
"language_model.model.layers.11.mlp.down_proj": [
4,
64
],
"language_model.model.layers.11.mlp.gate_proj": [
4,
32
],
"language_model.model.layers.11.mlp.up_proj": [
4,
32
],
"language_model.model.layers.11.self_attn.k_proj": [
8,
64
],
"language_model.model.layers.11.self_attn.o_proj": [
8,