Instructions to use luxoxo1/nex-si-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use luxoxo1/nex-si-12B with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("luxoxo1/nex-si-12B") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use luxoxo1/nex-si-12B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf luxoxo1/nex-si-12B:Q4_0 # Run inference directly in the terminal: llama cli -hf luxoxo1/nex-si-12B:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf luxoxo1/nex-si-12B:Q4_0 # Run inference directly in the terminal: llama cli -hf luxoxo1/nex-si-12B:Q4_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf luxoxo1/nex-si-12B:Q4_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf luxoxo1/nex-si-12B:Q4_0
Use Docker
docker model run hf.co/luxoxo1/nex-si-12B:Q4_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use luxoxo1/nex-si-12B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "luxoxo1/nex-si-12B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "luxoxo1/nex-si-12B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/luxoxo1/nex-si-12B:Q4_0
- Ollama
How to use luxoxo1/nex-si-12B with Ollama:
ollama run hf.co/luxoxo1/nex-si-12B:Q4_0
- Unsloth Desktop
- Pi
How to use luxoxo1/nex-si-12B with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "luxoxo1/nex-si-12B"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "luxoxo1/nex-si-12B" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use luxoxo1/nex-si-12B with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "luxoxo1/nex-si-12B"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "luxoxo1/nex-si-12B" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "luxoxo1/nex-si-12B", "messages": [ {"role": "user", "content": "Hello"} ] }' - Docker Model Runner
How to use luxoxo1/nex-si-12B with Docker Model Runner:
docker model run hf.co/luxoxo1/nex-si-12B:Q4_0
- Lemonade
How to use luxoxo1/nex-si-12B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull luxoxo1/nex-si-12B:Q4_0
Run and chat with the model
lemonade run user.nex-si-12B-Q4_0
List all available models
lemonade list
- Hermes Agent
How to use luxoxo1/nex-si-12B with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "luxoxo1/nex-si-12B"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default luxoxo1/nex-si-12B
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use luxoxo1/nex-si-12B with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "luxoxo1/nex-si-12B"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "luxoxo1/nex-si-12B" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
nex-si 12B
nex-si is a reasoning and coding fine-tune of Gemma 4 12B (abliterated), trained on a single MacBook Pro using only answers a computer could verify. Same architecture, same size (7.0 GB as Q4_0), better at math, logic and code on held-out tests.
โ ๏ธ This model is abliterated. Its base had refusal behaviour removed by huihui-ai, and nex-si keeps that. It will follow instructions the official Gemma model would refuse. You are responsible for how you use it and for any safety filtering your application needs. Not for production or public-facing use without your own safeguards.
Non-commercial. One training dataset (KodCode) is CC-BY-NC-4.0, so this release is licensed CC-BY-NC-4.0. The base model is Apache-2.0; its license text and notice are included (see License).
Results
All numbers are from problems never used in training: one attempt per problem, identical settings for every version, graded automatically (math answer match, code run against tests in a sandbox, logic answer match). 95% confidence intervals are paired bootstrap (10,000 resamples). "Original" is the base model above; v1 was trained on reasoning only; v2 (this release) added a coding round.
| Test (MLX 4-bit) | n | Original | v1 | v2 | v2 โ original (95% CI) |
|---|---|---|---|---|---|
| All coding (6,000-token limit) | 278 | 40.3% | 34.2% | 50.4% | +10.1 (+5.4 to +14.7) |
| HumanEval+ | 163 | 64.4% | 56.4% | 74.2% | +9.8 (+3.1 to +16.6) |
| CodeContests (test) | 40 | 0.0% | 2.5% | 27.5% | +27.5 (+15.0 to +42.5) |
| LeetCode (test) | 75 | 9.3% | 2.7% | 10.7% | +1.3 (โ4.0 to +6.7) |
| All reasoning (5,000-token limit) | 304 | 50.0% | 56.9% | 64.5% | +14.5 (+9.5 to +19.4) |
| GSM8K | 60 | 68.3% | 81.7% | 83.3% | +15.0 (+5.0 to +25.0) |
| MATH-500 (20 per level) | 100 | 55.0% | 64.0% | 75.0% | +20.0 (+11.0 to +29.0) |
| Procedural logic | 80 | 21.2% | 31.2% | 40.0% | +18.8 (+10.0 to +27.5) |
| HumanEval | 64 | 60.9% | 54.7% | 60.9% | 0.0 (โ12.5 to +12.5) |
v2 vs v1: coding +16.2 (CI +11.5 to +21.2), reasoning +7.6 (CI +3.0 to +12.2).
Deployed file (GGUF Q4_0 through Ollama, GSM8K, 60 problems): original 66.7%, v1 70.0%, v2 76.7% (v2 โ original +10.0, CI +1.7 to +18.3; v2 โ v1 +6.7, CI โ1.7 to +16.7, not significant on its own).
Raw summaries are in evals/. The full write-up, including what went wrong, is in
report.pdf.
These are this project's own harness and benchmark subsets, not official leaderboard numbers. Not yet compared with Google's official (non-abliterated) Gemma 4 12B.
Files
| File | What it is |
|---|---|
nex-si-12B-Q4_0.gguf |
Merged model, GGUF Q4_0 (7.0 GB), for Ollama / llama.cpp |
Modelfile |
Ollama config (Gemma 4 renderer and parser, 32K context) |
adapter-mlx/ |
The LoRA adapter itself (262 MB), for MLX on top of the 4-bit base |
evals/ |
Result summaries with confidence intervals |
report.pdf |
7-page write-up: method, timeline, honest review |
Use
Ollama (recommended). Download the GGUF and the Modelfile into one folder, then:
ollama create nex-si -f Modelfile
ollama run nex-si
Or pull directly: ollama run hf.co/luxoxo1/nex-si-12B:Q4_0. Pulling directly uses Ollama's
built-in template detection, which was not tested for this release; the Modelfile route was.
Settings used in all tests: temperature 0.6, top_p 0.95, top_k 0 (off), repeat_penalty 1.0, thinking on. Speed on an Apple M5 Pro is about 32โ34 tokens/s for one stream, the same as the base model.
MLX. Convert the base to 4-bit MLX, then load the adapter:
mlx_lm.convert --hf-path huihui-ai/Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated -q --q-bits 4 --mlx-path base-mlx-4bit
mlx_lm.generate --model base-mlx-4bit --adapter-path adapter-mlx --prompt "..."
Results above were measured on this project's own 4-bit conversion; another conversion may differ slightly.
How it was made
The model attempts problems with automatic checkers; only verified attempts are kept, and it is trained on its own best work (rejection-sampling fine-tuning, the STaR family of methods).
- Problems: GSM8K, MATH (levels 2โ5), 8 families of procedurally generated logic puzzles with brute-force-verified unique answers, and coding problems from LeetCodeDataset, MBPP, KodCode, APPS, TACO, CodeContests and rStar-Coder. Test problems were set aside first and checked for overlap (exact match; 8-gram overlap for the coding sets).
- Generation: the model itself (MLX, batches of 10). A short "solve once, check once" guide prompt was used during generation and removed from the training examples.
- Selection: shortest verified-correct attempt per problem; loops and over-long traces dropped.
- Salvage: in the coding round, 92 of the first 123 attempts ran out of tokens and none of the finished answers was wrong: the model kept re-checking working code. Cut-off attempts were trimmed right after the first draft in the reasoning that passed the tests (89 examples).
- Teacher solutions: 406 short solutions from the public OpenCodeReasoning and rStar-Coder datasets, each re-run against tests.
- Replay: 135 of the base model's own answers to everyday prompts (Dolly-15k plus a few French prompts), to keep general chat stable.
- v2 training set: 1,058 examples: math and logic 255, everyday replay 135, own verified code 262, teacher code 406.
- Training: QLoRA with MLX on the frozen 4-bit base, MacBook Pro M5 Pro, 24 GB. LoRA rank 32, scale 2.0, dropout 0.05, on all attention and MLP projections of the top 24 of 48 layers (65.6M trainable parameters). Completion-only loss, max 6,144 tokens, effective batch 8, 2 epochs (2,062 iterations, 257 optimizer updates), AdamW, learning rate 1e-4 with cosine decay. 242 minutes, 16.4 GB peak memory.
- Export: adapter merged into the bf16 weights, converted with llama.cpp, quantized Q4_0. The base is Google's quantization-aware-trained checkpoint for Q4_0; in Ollama the unchanged base scored 66.7% on GSM8K as Q4_0 versus 48.3% as Q4_K_M.
Training targets. Loss is computed only on completions: the model's own verified answers and the open-dataset teacher solutions. No proprietary-model outputs were used as training targets.
Known limitations
- Overthinking remains. 130 of 278 coding answers still hit the 6,000-token limit; about 90% of LeetCode attempts never finish (the base model too). Some long answers end in repeated lines.
- No new knowledge. Facts and knowledge cutoff are the base model's. Use retrieval for current facts.
- Small tests. 40โ163 problems per coding set and 60โ100 per reasoning set, one sample each.
License
- This release (fine-tuned weights, adapter, files): CC-BY-NC-4.0, because the training data includes KodCode (CC-BY-NC-4.0). Non-commercial use only.
- Base model: Gemma 4 12B by Google, Apache-2.0
(license); abliteration by huihui-ai, Apache-2.0.
See
LICENSE-APACHE-2.0.txtandNOTICE. - Training data licenses: GSM8K (MIT), MATH (MIT), MBPP (CC-BY-4.0), LeetCodeDataset (Apache-2.0), KodCode (CC-BY-NC-4.0), APPS (MIT), TACO (Apache-2.0), CodeContests (CC-BY-4.0), rStar-Coder (CC-BY-4.0), OpenCodeReasoning (CC-BY-4.0), databricks-dolly-15k prompts (CC-BY-SA-3.0). Evaluation only: HumanEval (MIT), HumanEval+ (Apache-2.0). Licenses as listed on Hugging Face, checked November 6โ7, 2026.
"Gemma" is a trademark of Google. nex-si is not affiliated with or endorsed by Google or huihui-ai.
Lineage
google/gemma-4-12B-it โ google/gemma-4-12B-it-qat-q4_0-unquantized โ
huihui-ai/Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated (commit 5201f48) โ nex-si 12B
- Downloads last month
- 37
Model tree for luxoxo1/nex-si-12B
Base model
google/gemma-4-12B