Instructions to use Thox-ai/ThoxEdge-RustCoder-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Thox-ai/ThoxEdge-RustCoder-0.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Thox-ai/ThoxEdge-RustCoder-0.5B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Thox-ai/ThoxEdge-RustCoder-0.5B")
model = AutoModelForCausalLM.from_pretrained("Thox-ai/ThoxEdge-RustCoder-0.5B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Thox-ai/ThoxEdge-RustCoder-0.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Thox-ai/ThoxEdge-RustCoder-0.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxEdge-RustCoder-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Thox-ai/ThoxEdge-RustCoder-0.5B

SGLang

How to use Thox-ai/ThoxEdge-RustCoder-0.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Thox-ai/ThoxEdge-RustCoder-0.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxEdge-RustCoder-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Thox-ai/ThoxEdge-RustCoder-0.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxEdge-RustCoder-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Thox-ai/ThoxEdge-RustCoder-0.5B with Docker Model Runner:
```
docker model run hf.co/Thox-ai/ThoxEdge-RustCoder-0.5B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

ThoxEdge-RustCoder-0.5B

A Rust-leaning fine-tune of Qwen/Qwen2.5-Coder-0.5B-Instruct, trained by THOX.ai on the Apache-2.0 Fortytwo-Network/Strandset-Rust-v1 corpus.

Both the base and this model are Apache-2.0. Nothing here is derived from, initialised from, or distilled out of any Gemma-licensed Rust model — see the lane doc for why that mattered.

What it is, and what it is not

It is a 0.5B model. It is not a strong Rust coder, and no number below should be read as claiming otherwise. It exists to serve cheap Rust completion in ThoxRoute's lowest tier and to establish whether the recipe moves the needle at all before the same recipe is spent on a 1.5B.

It does not fit THOX's 512 MB devices. The GGUF ladder for this architecture spans only 260 MB — Q2_K still costs 415 MB — because vocab_size 151,936 against hidden_size 896 puts 27.6% of params in an embedding whose row length is not divisible by the 256-wide K-quant super-block, so it never quantizes down. This is a hosted / desktop model.

Evaluation

MultiPL-E humaneval-rs, 156 problems, executed: each generation is assembled into a program, compiled with rustc, and its asserts run. Greedy decoding, so the delta is not sampling noise.

Both models were scored in the same job, so the comparison cannot drift between runs.

model	completion mode	chat mode
`Qwen/Qwen2.5-Coder-0.5B-Instruct` (base)	21.15% (33/156)	25.64% (40/156)
`Thox-ai/ThoxEdge-RustCoder-0.5B`	25.64% (40/156)	28.21% (44/156)
delta	+4.49 pp	+2.57 pp

The fine-tune improves on its base in both prompting modes, and the best available configuration moves from 25.64% (base, chat) to 28.21% (tuned, chat) — a net +2.57 pp, or four more HumanEval-Rust problems solved.

Noise floor: about ±1 problem (±0.64 pp). An earlier run of the identical base on different hardware scored 20.51% in completion mode against 21.15% here, despite greedy decoding — GPU reductions are not bit-identical across flavors. The +4.49 pp (7 problems) and +2.57 pp (4 problems) deltas both clear that, but anything under about 1.5 pp from this harness should be treated as a tie.

Assembly is deliberately generous — a body-only answer and a whole-function answer are both attempted, and the problem passes if either works. These numbers are therefore not comparable to published MultiPL-E leaderboards. The protocol is identical for both models, so the delta between the rows is the measurement.

Training


Method	LoRA r=32, α=64, attention + MLP, merged into the base
Corpus	Strandset-Rust-v1, filtered to the six code-producing categories (74,606 of 191,008 rows)
Steps	1200 at batch 16, max_length 1024
LR	2e-4, cosine, 30 warmup
Hardware	1× L40S

The ~117k rows dropped emit English about Rust — docstrings, naming, summaries, review. Training them would spend the step budget on describing code rather than writing it.

Prompting it

Ask for a complete function body in plain terms — for example "Write the complete body of the Rust function below."

Do not prompt it with "Complete the Rust code. Output only the missing code." That phrasing was the code_completion instruction across 14,527 training rows whose targets are short fill-in-the-middle fragments, so the model will answer it with a fragment — sometimes a bare identifier. Measured: that exact wording scored 2.56% where the recommended wording scores 28.21%. The training instruction has since been reworded to name both sides of the gap, but this checkpoint predates that fix and retains the sensitivity.

Known limitations in this build

Trained with a renderer that placed the rationale / bug_description output fields inside the ```rust fence, affecting ~53% of rows across the refactoring, optimization and bug-detection categories. Fixed in tools/train_thox_rustcoder.py after this run. It did not cause any measured regression — that was the prompt collision above — but a rerun should still carry the fix.
It is a 0.5B. 28.21% pass@1 means it fails roughly seven of every ten HumanEval-Rust problems.

Attribution

Training data: Fortytwo-Network/Strandset-Rust-v1, Apache-2.0 — synthetic Rust generated from idioms mined across 2,300+ crates.io crates, consensus-filtered and 94.3% rustc-verified.

Files

File	Size	SHA-256
`model.safetensors`	988.10 MB (988,097,824 B)	`bbb25d81022f1243f48cdd143c83f249ef2ca18ec8eef301803786402c5fec47`

Use with ThoxOS

This repo ships safetensors only (no GGUF). Run it with 🤗 Transformers on a ThoxOS host (see the usage section above), or convert it with llama.cpp convert_hf_to_gguf.py if the architecture is supported.

hf download Thox-ai/ThoxEdge-RustCoder-0.5B --local-dir ./ThoxEdge-RustCoder-0.5B

Use with MeshStack

Serve it behind an OpenAI-compatible server (for example a Transformers/vLLM server on port 8080) and register http://<node-ip>:8080/v1 with your MeshStack coordinator.

Device fit (by file size only)

model.safetensors (988.10 MB): Medium (<2.5 GB): devices with ≥4 GB RAM — recent phones, laptops, Apple Silicon Macs, MeshStack edge hosts.

RAM needed ≈ file size + context (KV cache) + runtime overhead. These tiers are derived from file size alone; no throughput, TOPS or battery figures are claimed here.

Licence and attribution

Upstream Qwen/Qwen2.5-Coder-0.5B-Instruct is Apache-2.0 (© Alibaba Cloud / Qwen team; verified against upstream Hub metadata 2026-10-23). The Apache-2.0 licence and NOTICE carry forward.

Out of scope

Medical, legal, financial, safety-critical or other high-stakes decisions without human review.
Unattended control of physical devices or actuators without a confirmation step.
Any use prohibited by the upstream licence or acceptable-use policy.

THOX.ai LLC — Your AI. Your Data. Your Rules.™ · thox.ai · Contact: tommy@thox.ai

Downloads last month: 28

Safetensors

Model size

0.5B params

Tensor type

BF16

Model tree for Thox-ai/ThoxEdge-RustCoder-0.5B

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-Coder-0.5B

Finetuned

Qwen/Qwen2.5-Coder-0.5B-Instruct

Finetuned

(112)

this model

Quantizations

1 model

Thox-ai
/

ThoxEdge-RustCoder-0.5B

ThoxEdge-RustCoder-0.5B

What it is, and what it is not

Evaluation

Training

Prompting it

Known limitations in this build

Attribution

Files

Use with ThoxOS

Use with MeshStack

Device fit (by file size only)

Licence and attribution

Out of scope

Model tree for Thox-ai/ThoxEdge-RustCoder-0.5B

Dataset used to train Thox-ai/ThoxEdge-RustCoder-0.5B