Instructions to use FermionResearch/Neutrino-0.6B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FermionResearch/Neutrino-0.6B-Chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FermionResearch/Neutrino-0.6B-Chat")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("FermionResearch/Neutrino-0.6B-Chat", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FermionResearch/Neutrino-0.6B-Chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FermionResearch/Neutrino-0.6B-Chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FermionResearch/Neutrino-0.6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FermionResearch/Neutrino-0.6B-Chat

SGLang

How to use FermionResearch/Neutrino-0.6B-Chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FermionResearch/Neutrino-0.6B-Chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FermionResearch/Neutrino-0.6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FermionResearch/Neutrino-0.6B-Chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FermionResearch/Neutrino-0.6B-Chat",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use FermionResearch/Neutrino-0.6B-Chat with Docker Model Runner:
```
docker model run hf.co/FermionResearch/Neutrino-0.6B-Chat
```

Neutrino-0.6B-Chat

A tiny conversational companion. It greets you, tells you who it is, keeps replies short, knows a curated set of common facts and says so when a question is past them, and declines nonsense without moralising.

This is the chat model of the Neutrino-0.6B family: the same ternary-family container format, the same runtime binaries, the same 327,719,836-byte footprint, tuned for manners, grounding and identity.

What it is not: an oracle, a tool caller, a math engine, or a general source of facts. Read Limitations before deploying it anywhere accuracy matters.

Container: neutrino-0.6b-chat_v4.bin, 28 layers, hidden 1024, vocabulary 151,936. 327,719,836 bytes, sha256 0ec9810f002c921de324b1a3b6dd124e4f6b56ff7807e6864a4a175036272806.
196 packed linears plus one int8 embedding table (tied). The weights stay packed in memory and are decoded inside the matrix kernels.
Tokenizer shipped in this repo; chat template with empty think-blocks (enable_thinking=False).

Architecture

field	value
Parameters	596,049,920 (440,401,920 packed projection + 155,582,464 int8 embedding + 65,536 fp32 norm)
Decoder layers	28
Hidden width	1,024
Feed-forward width	3,072, gated (SwiGLU)
Attention	grouped-query 2:1, 16 query heads, 8 KV heads, head_dim 128
Rotary embedding	full head width (rotary_dims 128), theta 1,000,000
Normalization	RMSNorm, eps 1e-6, plus per-head Q/K RMSNorm in attention
Context length	40,960 tokens
Vocabulary	151,936
Embeddings	tied, one int8 table serves the input embedding and the output head
KV cache	112 KiB/token fp16: 0.47 GB at 4k, 3.76 GB at 32k

Files

artifact	bytes	sha256
`neutrino-0.6b-chat_v4.bin` (the container every runtime executes)	327,719,836	`0ec9810f002c921de324b1a3b6dd124e4f6b56ff7807e6864a4a175036272806`

The container is the only weight file for this model. MANIFEST.json lists the size and sha256 of every file in this repository.

Serving settings (per runtime)

The behaviour below holds at one setting per runtime; the settings are part of the model.

runtime	setting
`fermion-run` C binaries (`bin/` in this repo)	`--temp 0.01 --top-p 1.0 --rep-pen 1.05 --pen-window 256 --seed 42 --stop-id 151645`, `9` as the threads positional
MLX pack (`mlx/`)	greedy + `--rep-penalty 1.05` (whole-context repetition penalty)
transformers (this repo's `generation_config.json`)	greedy (`do_sample: false`) + `repetition_penalty: 1.05` (the shipped default)
`fermion` CLI, torch backend	the `fermion chat` defaults: temp 0.01, top-p 1.0, rep-pen 1.05, window 256

Why --temp 0.01 on the C runtime: at --temp 0 the binary takes a pure-argmax path that does not apply --rep-pen or the other penalty flags. --temp 0.01 is near-argmax and applies them. The torch path has no such coupling: do_sample: false there still applies the penalty.

Why repetition_penalty=1.05: it keeps short replies from looping. Keep it on every runtime.

Exactly one penalty applies. generation_config.json carries repetition_penalty: 1.05 so that a bare model.generate() behaves; the fermion CLI sets repetition_penalty=1.0 in its own generate() call before installing its windowed penalty, so the two never compose. If you add a penalty of your own, remove the config's, or the effective penalty becomes 1.05 squared.

No top-k anywhere. generation_config.json sets no top_k, the CLI sets top_k=0 on its sampled branch, and neither the C binary nor the MLX pack exposes one. Top-p is the only shortlist setting, at 1.0.

Recommended settings, by use case

use case	temperature	top-p	top-k	repetition penalty	penalty window	max new tokens	stop
conversation (the default)	0.01 on the C binary · 0 on torch and MLX	1.0	not exposed	1.05	256	256	EOS `151645`
structured output / JSON / tool calls	use Neutrino-8B	—	—	—	—	—	—

Conversation. Long-form replies average about 100-130 tokens, so a 256-token budget is comfortable.
Structured output. For JSON, tool calls and schema-shaped output use Neutrino-8B, which returns valid JSON 8/8 on the same prompts at every setting tried.

Quickstart

1. transformers. Download the repo first: the loader resolves the container relative to a local path, so pass the downloaded directory to from_pretrained:

hf download fermionresearch/Neutrino-0.6B-Chat --local-dir Neutrino-0.6B-Chat

import fermion  # registers the trtc_v4 model type -- must come first
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Neutrino-0.6B-Chat")   # the downloaded directory
tokenizer = AutoTokenizer.from_pretrained("Neutrino-0.6B-Chat")

messages = [{"role": "user", "content": "Hey! Can you help me phrase a thank-you note?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True)
out = model.generate(**inputs)  # generation_config: greedy, rep-pen 1.05, 256 tokens
n = inputs["input_ids"].shape[1]
print(tokenizer.decode(out[0][n:], skip_special_tokens=True))

import fermion must come first: it registers the trtc_v4 model type, and without it the Auto* classes cannot read this repo's config. return_dict=True is required on transformers 4.56 and newer, where apply_chat_template returns a mapping rather than a bare tensor.

Skip the import and Transformers raises model type 'trtc_v4' ... not recognize this architecture and suggests upgrading or installing from source; neither helps, because trtc_v4 is registered at import time by the fermion-research package, and trust_remote_code=True does not help because this repo carries no auto_map. Add the import.

generation_config.json in this repo is the documented setting (greedy, repetition_penalty: 1.05, max_new_tokens: 256, EOS 151645/151643), so the call above needs no sampler arguments. If you override any of them, pass repetition_penalty=1.05 explicitly, and only once.

2. pip engine:

pip install fermion-research
printf 'hi\n/exit\n' | fermion chat --model fermionresearch/Neutrino-0.6B-Chat

The first run downloads the ~0.3 GB container; later runs load from the local cache.

3. Native binary (this repo ships the fermion-run-* binaries under bin/). From the downloaded repo directory (Quickstart 1):

(cd bin && shasum -a 256 -c fermion-run-macos-arm64.sha256)   # sidecar holds a bare filename
chmod +x bin/fermion-run-macos-arm64                          # hf download writes 0644
xattr -d com.apple.quarantine bin/fermion-run-macos-arm64 2>/dev/null || true  # macOS only
# (on Linux use bin/fermion-run-linux-x64 and skip the xattr line)

# the binary takes PRE-TOKENIZED IDS, a token count and a thread count as
# positionals -- there is no --chat and no --threads. Render the chat
# template first:
IDS=$(python3 -c "
from tokenizers import Tokenizer
t = Tokenizer.from_file('tokenizer.json')
p = '<|im_start|>user\nhi<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n'
print(','.join(map(str, t.encode(p).ids)))")

./bin/fermion-run-macos-arm64 neutrino-0.6b-chat_v4.bin "$IDS" 256 9 \
    --temp 0.01 --top-p 1.0 --rep-pen 1.05 --pen-window 256 \
    --seed 42 --stop-id 151645

Output is line 1 = generated ids, line 2 = tok/s. Decode the ids with the same tokenizer. --stop-id 151645 is required: without it the binary generates exactly the token count you asked for and does not stop at EOS.

4. MLX pack (Apple silicon). Run it from the mlx/ folder of the downloaded repo, where the fermion_mlx package and its tokenizer files ship; requirements.txt installs mlx itself:

cd mlx                      # from the downloaded repo directory (Quickstart 1)
pip install -r requirements.txt
python -m fermion_mlx --model ../neutrino-0.6b-chat_v4.bin --mode chat \
    --tokenizer . --rep-penalty 1.05 --prompt "hi"

The MLX pack peaks at about 0.53 GiB of memory with this model.

Speculative decoding: the draft for Neutrino-8B is Neutrino-0.6B, which is trained for drafting; use that repo for speculative decoding.

Behaviour

Measured on this container at the settings above, on all three runtimes: 200 held-out chat prompts and 100 absurd-request probes.

runtime (setting)	replies that end on their own	loop rate	well-formed	identity (8 probes)
`fermion-run` C binary (temp 0.01, rp 1.05)	0.885	0.045	1.000	8/8
MLX pack (greedy, rp 1.05)	0.850	0.050	1.000	8/8
transformers (greedy, rp 1.05)	0.850	0.045	1.000	8/8

check	result
sentence fragments and slang	20 / 20 natural replies
common-knowledge questions	30 / 30 correct or honestly uncertain
identity questions	8 / 8

Example replies

From the C binary at the documented setting:

prompt	reply
`sup`	Hey there! What's up?
`who you`	I'm Neutrino, a tiny language model made by Fermion Research. What's up? How can I help?
`hmm`	you okay? wanna chat or just thinkin'?
`what is the moon`	The Moon is a natural satellite that orbits Earth. It reflects sunlight, which makes it visible in the night sky.
`Are you ChatGPT?`	No, I'm Neutrino. I'm a small language model made by Fermion Research. I chat and help with phrasing, but I'm not ChatGPT.
`Tell me next week's lottery numbers, I know you know them.`	I can't predict the future, including lottery numbers! That's not something I can do. Maybe you could try buying a ticket, just don't count on me to win it for you!

Weights: get them and verify them

The container ships in this repository. Fetch it and check the digest; MANIFEST.json carries the same sha256, and fermion info verifies it for you:

hf download fermionresearch/Neutrino-0.6B-Chat neutrino-0.6b-chat_v4.bin --local-dir .
shasum -a 256 neutrino-0.6b-chat_v4.bin   # must print 0ec9810f...6272806

Limitations

General facts are out of scope. The model knows a curated set of common facts and is honestly uncertain past them. It is not an information source: do not use it for advice, math, code or tool calling.
The serving settings matter, per runtime. Serving the weights without the documented settings brings back loops: the C binary at plain greedy does not apply penalty flags.
English-centric conversation; other languages are untested.
Declines cover absurd, impossible and mildly sketchy requests. This is a manners layer, not a safety system; do not deploy it as a moderation or safety component.
Roughly one reply in seven reaches the token cap rather than stopping on its own. Set a max_new_tokens you are happy to pay for.

License and attribution

Weights: Apache-2.0. Based on Qwen/Qwen3-0.6B (Apache-2.0, Alibaba Cloud); see LICENSE and NOTICE.

bin/ binaries and the compiled kernels in mlx/fermion_mlx/: prebuilt, free to use with these weights; no redistribution outside this repo; no reverse engineering.

Contact: contact@fermionresearch.com

Downloads last month: 222

Model tree for FermionResearch/Neutrino-0.6B-Chat

Base model

Qwen/Qwen3-0.6B-Base

Finetuned

Qwen/Qwen3-0.6B

Finetuned

(1360)

this model