Instructions to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8
# Run inference directly in the terminal:
llama cli -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8
# Run inference directly in the terminal:
llama cli -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8
# Run inference directly in the terminal:
./llama-cli -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Use Docker

docker model run hf.co/Anomly/Mistral-7B-Instruct-v0.3-bposit8

LM Studio
Jan

vLLM

How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Anomly/Mistral-7B-Instruct-v0.3-bposit8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Anomly/Mistral-7B-Instruct-v0.3-bposit8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Anomly/Mistral-7B-Instruct-v0.3-bposit8

Ollama
How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with Ollama:
```
ollama run hf.co/Anomly/Mistral-7B-Instruct-v0.3-bposit8
```
Unsloth Desktop

How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Anomly/Mistral-7B-Instruct-v0.3-bposit8"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with Docker Model Runner:
```
docker model run hf.co/Anomly/Mistral-7B-Instruct-v0.3-bposit8
```

Lemonade

How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Anomly/Mistral-7B-Instruct-v0.3-bposit8

Run and chat with the model

lemonade run user.Mistral-7B-Instruct-v0.3-bposit8-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Anomly/Mistral-7B-Instruct-v0.3-bposit8

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Anomly/Mistral-7B-Instruct-v0.3-bposit8 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Anomly/Mistral-7B-Instruct-v0.3-bposit8

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Anomly/Mistral-7B-Instruct-v0.3-bposit8" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Mistral-7B-Instruct-v0.3-bposit8

mistralai/Mistral-7B-Instruct-v0.3 converted to b-posit8 (32-code blocks with a power-of-two scale, 8-bit posit codes, es = 2) for the exact profile of INVAR: every matmul accumulates in a 256-bit quire with one rounding, so a deterministic runtime produces bit-identical activations and logits on x86, CUDA and aarch64, and independent reference implementations reproduce a served answer from these weights and the token ids.

File: Mistral-7B-Instruct-v0.3-bposit8.gguf (7.48 GB), SHA-256 55b790ce4514baee4aa6343b141d950fd3c5069eca7b8c2f113d61cbbd97d1a9
Format: GGUF general.file_type 42 (b-posit8), llama-cpp-et fork (deterministic backend)
Spec: docs/EXACT-PROFILE-SPEC.md in the INVAR repo; conformance fixture and tokenizer vectors in go/crverify/testdata
Licence: this is a re-quantised copy of mistralai/Mistral-7B-Instruct-v0.3; the upstream licence (apache-2.0) applies unchanged, including any use restrictions and attribution requirements.

Run it: invar serve --model Mistral-7B-Instruct-v0.3-bposit8.gguf --binary llama-cli --spot-check --spot-check-units, then invar verify worldline.jsonl --model Mistral-7B-Instruct-v0.3-bposit8.gguf --binary llama-cli --spot-check --units --reexec.

Check the file before you trust it: sha256sum Mistral-7B-Instruct-v0.3-bposit8.gguf must print the digest above.

Measured

Under the exact profile the whole graph of this family is bit-identical across x86 CPUs, NVIDIA CUDA and aarch64 (whole-graph dumps compared tensor for tensor); the measurement files are in the INVAR repository.

What this type does NOT do on its own: in a stock runtime without the deterministic elementwise profile, b-posit8 weights diverge across machines like any other quantisation (we measured the type alone at a larger cross-machine divergence than Q8_0). Reproducibility is the pair, type plus profile, which is what the exact profile of INVAR implements and specifies.

Why the Hub says "not able to determine the quantization variants"

The Hub's GGUF viewer recognises the quantization types shipped in upstream llama.cpp (Q4_K_M, Q8_0, F16, MXFP4 and so on) from the file name. b-posit8 is a new tensor type (GGUF tensor type 43, file type 42) that is not in that list yet, so the viewer cannot label it; the file itself is a valid GGUF and the metadata viewer reads it fine. Storage is 8 bits per weight plus one signed 8-bit power-of-two scale per 32 weights (8.25 bits per weight), comparable to Q8_0. It runs only on the deterministic llama.cpp fork that INVAR uses, not on stock llama.cpp or Ollama; that is the point of the profile, and the spec is public so other runtimes can implement it.

Downloads last month: 29

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for Anomly/Mistral-7B-Instruct-v0.3-bposit8

Base model

mistralai/Mistral-7B-v0.3

Finetuned

mistralai/Mistral-7B-Instruct-v0.3

Quantized

(308)

this model