Instructions to use Anomly/SmolLM2-135M-Instruct-bposit8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Anomly/SmolLM2-135M-Instruct-bposit8 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anomly/SmolLM2-135M-Instruct-bposit8
# Run inference directly in the terminal:
llama cli -hf Anomly/SmolLM2-135M-Instruct-bposit8

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anomly/SmolLM2-135M-Instruct-bposit8
# Run inference directly in the terminal:
llama cli -hf Anomly/SmolLM2-135M-Instruct-bposit8

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Anomly/SmolLM2-135M-Instruct-bposit8
# Run inference directly in the terminal:
./llama-cli -hf Anomly/SmolLM2-135M-Instruct-bposit8

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Anomly/SmolLM2-135M-Instruct-bposit8
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Anomly/SmolLM2-135M-Instruct-bposit8

Use Docker

docker model run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8

LM Studio
Jan

vLLM

How to use Anomly/SmolLM2-135M-Instruct-bposit8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Anomly/SmolLM2-135M-Instruct-bposit8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Anomly/SmolLM2-135M-Instruct-bposit8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8

Ollama
How to use Anomly/SmolLM2-135M-Instruct-bposit8 with Ollama:
```
ollama run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8
```
Unsloth Desktop
Docker Model Runner
How to use Anomly/SmolLM2-135M-Instruct-bposit8 with Docker Model Runner:
```
docker model run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8
```

Lemonade

How to use Anomly/SmolLM2-135M-Instruct-bposit8 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Anomly/SmolLM2-135M-Instruct-bposit8

Run and chat with the model

lemonade run user.SmolLM2-135M-Instruct-bposit8-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

SmolLM2-135M-Instruct-bposit8

HuggingFaceTB/SmolLM2-135M-Instruct converted to b-posit8 (32-code blocks with a power-of-two scale, 8-bit posit codes, es = 2) for the exact profile of INVAR: every matmul accumulates in a 256-bit quire with one rounding, so a deterministic runtime produces bit-identical activations and logits on x86, CUDA and aarch64, and independent reference implementations reproduce a served answer from these weights and the token ids.

File: SmolLM2-135M-Instruct-bposit8.gguf (0.14 GB), SHA-256 f09fe23eb1ec46dfedfd4c6603c2928b3e349190c13b39b7993be8251b52d6d0
Format: GGUF general.file_type 42 (b-posit8), llama-cpp-et fork (deterministic backend)
Spec: docs/EXACT-PROFILE-SPEC.md in the INVAR repo; conformance fixture and tokenizer vectors in go/crverify/testdata
Licence: this is a re-quantised copy of HuggingFaceTB/SmolLM2-135M-Instruct; the upstream licence (apache-2.0) applies unchanged, including any use restrictions and attribution requirements.

Run it: invar serve --model SmolLM2-135M-Instruct-bposit8.gguf --binary llama-cli --spot-check --spot-check-units, then invar verify worldline.jsonl --model SmolLM2-135M-Instruct-bposit8.gguf --binary llama-cli --spot-check --units --reexec.

Check the file before you trust it: sha256sum SmolLM2-135M-Instruct-bposit8.gguf must print the digest above.

Measured

Whole-graph dump hash 68edecdc52a5e19e is identical on x86 (gcc), an Apple M4 Pro CPU and GPU (Metal), an NVIDIA RTX 5090 (CUDA exact profile), a RISC-V build under QEMU and a Tenstorrent Blackhole re-execution. This is the model INVAR's conformance fixtures and tokenizer vectors reference.

What this type does NOT do on its own: in a stock runtime without the deterministic elementwise profile, b-posit8 weights diverge across machines like any other quantisation (we measured the type alone at a larger cross-machine divergence than Q8_0). Reproducibility is the pair, type plus profile, which is what the exact profile of INVAR implements and specifies.

Why the Hub says "not able to determine the quantization variants"

The Hub's GGUF viewer recognises the quantization types shipped in upstream llama.cpp (Q4_K_M, Q8_0, F16, MXFP4 and so on) from the file name. b-posit8 is a new tensor type (GGUF tensor type 43, file type 42) that is not in that list yet, so the viewer cannot label it; the file itself is a valid GGUF and the metadata viewer reads it fine. Storage is 8 bits per weight plus one signed 8-bit power-of-two scale per 32 weights (8.25 bits per weight), comparable to Q8_0. It runs only on the deterministic llama.cpp fork that INVAR uses, not on stock llama.cpp or Ollama; that is the point of the profile, and the spec is public so other runtimes can implement it.

Downloads last month: 20

GGUF

Model size

0.1B params

Architecture

llama

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for Anomly/SmolLM2-135M-Instruct-bposit8

Base model

HuggingFaceTB/SmolLM2-135M

Quantized

HuggingFaceTB/SmolLM2-135M-Instruct

Quantized

(144)

this model