Instructions to use labqoat/squad-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use labqoat/squad-0.8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="labqoat/squad-0.8b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("labqoat/squad-0.8b")
model = AutoModelForMultimodalLM.from_pretrained("labqoat/squad-0.8b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use labqoat/squad-0.8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "labqoat/squad-0.8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/squad-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/labqoat/squad-0.8b

SGLang

How to use labqoat/squad-0.8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "labqoat/squad-0.8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/squad-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "labqoat/squad-0.8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "labqoat/squad-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use labqoat/squad-0.8b with Docker Model Runner:
```
docker model run hf.co/labqoat/squad-0.8b
```

▶ Live demo · qev-0.8b

squad-0.8b

A team of models. On your device. In milliseconds.

squad: the qev model and its team of experts. Not related to the SQuAD question-answering dataset.

A 0.8B typed-decision model (System One API: a state plus typed questions in, a probability per allowed answer out, one forward pass, no text generated) built as a MoQLE, a Mixture of Quantized LoRA Experts: one trunk, a 1.1 MB router after layer 5, and 2-bit LoRA experts it calls in when a question needs them.

This revision (October 2026) has a new trunk and four experts: games, e2e (decisions of a test agent driving a web app), multilingual (10 languages) and documents.

Team: each expert is 2.7 MB; the four add about 11 MB to the trunk.
Fast: 3.7 ms median per decision on one L40S (single requests), about 110 decisions/s with 16 clients. The router and experts cost nothing measurable.
Better than the previous squad-0.8b on rule following (+10 points on Rules-7), consistency (+5.8 on JevBench coherence), game boards (+3.9 on JevSpatial) and web-app tests on apps it never saw (35% vs 21% of tests passed); slightly lower on Benchmark Heaven (−0.5) and games_bench (−0.9), both within run-to-run noise.
New e2e expert (2026-10-26): retrained on real web-app decisions (138 generated apps, 819 tests, 32k labelled decisions from the real tester-army executor, including the states the expert itself reached). It no longer re-taps a control that already worked, and presses Enter where an app expects it.

Results

All numbers measured with the same benchmark client; each model served by its own runtime (this repo's qev_fast engine, per-question requests). basal-omni-pico (Remek/basal-omni-pico) was served by basal 2.4.0, fast mode.

	squad-0.8b (previous)	squad-0.8b (this revision)	basal-omni-pico
Benchmark Heaven (public, 231)	77.1	76.6	75.8
JevExtra	71.3	71.7	64.6
JevSpatial	75.8	79.7	68.4
Rules-7	63.8	74.1	69.5
JevBench accuracy	84.9	86.4	84.0
JevBench coherence	70.6	76.4	74.2
KLEJ (Polish, 7 tasks, 500 test items each)	70.9 (trunk only)	73.4	75.3
games_bench (9 unseen games, closed loop; 0 = random, 100 = planner)	11.3	10.4	7.5
e2e, 14 unseen apps (82 tests, held-out test split)	–	29 / 82 (first release of this revision: 17 / 82)	–
e2e, 8 real web apps (61 tests × 3 runs, eval only)	10 / 183	31 / 183 (first release: 34 / 183)	–

Speed on one NVIDIA L40S (Benchmark Heaven requests, one question each, after warm-up):

	this revision	basal-omni-pico (fast)	basal-omni-pico (eager)
single request, median / p95	3.7 / 30.5 ms	6.0 / 31.5 ms	50.4 / 101.3 ms
16 requests at once	115 decisions/s	94 decisions/s	33 decisions/s
GPU memory	6.3 GB	–	–

Read these honestly:

Benchmark Heaven moves by about ±1 point between runs (one item is 0.43 points); its hard tier (111 items) is around 57% for every model here.
Rules-7: part of the gain comes from procedurally generated rule-application data whose layout (the policy inside the question) resembles Rules-7's. No benchmark text was used; labels are computed by code. Sub-scores vary across training seeds.
e2e: runs of one test are near-copies (decisions are argmax), so count tests, not runs. On the 14 held-out test apps the new expert passes 29 of 82 tests vs 17 for the first release of this revision (the checkpoint was chosen on 14 other validation apps: 36 vs 18). On the 8 eval apps it passes 31 / 183 vs 34 / 183: TodoMVC 0 → 12, helpdesk 0 → 5, but the billing app 18 → 0 (its invite dialog). Pass rates swing a lot between nearby checkpoints (a later one scored 11 / 82); treat e2e numbers as ±5 tests. The e2e expert is used only when the router gives it at least 35% (a serving setting chosen while looking at Benchmark Heaven and the real-app tests).
games_bench is our own closed-loop benchmark of games never trained on; all models are close to random when a game is shown as a raw grid, coordinates or prose.
Image input was not re-evaluated for this revision (the trunk changed; the vision tower is unchanged).
Benchmark scores were used for reporting; checkpoints were chosen on our own validation data, except the serving threshold above.

How it works

Trunk: a full fine-tune of Qwen3.5-0.8B-Base (24 layers, Gated DeltaNet + gated attention). It continues the previous qev-0.8b trunk on procedurally generated, exactly labelled data: consistency twins (the same decision asked in rephrased, reordered, merged or complementary forms, trained with a consistency loss), rule application with natural "up to / below" semantics, graded similarity, plus replay of the old data. Final weights are a weight average of three runs. No LLM-written text is in the trunk.
Experts: rank-64 LoRA on gate_proj, up_proj, down_proj of layers 12–23, y = W x + 2.0 · B A x, stored as 2-bit rows (specialists/slots_2bit.safetensors). Slot 1 games, 2 e2e, 3 multilingual, 4 documents. Trained by reinforcement learning only (closed-loop games, simulated web apps, decision tasks with a proper-scoring reward), trunk frozen. The multilingual and documents experts learned from data written and cross-checked by two LLMs.
Router: LayerNorm → Linear(1024, 256) → GELU → Linear(256, 9) on the mean RMS-normalised residual after layer 5 (specialists/router.safetensors), top-1 of {trunk, games, e2e, multilingual, documents}. Trained on measured outcomes (log-probability of the right answer) on validation rows no expert saw, so it only calls an expert where it helps. router.min_p_slots in the manifest holds per-expert confidence thresholds (e2e: 0.35).

Quickstart (NVIDIA GPU)

pip install -r requirements.txt -r requirements-cuda.txt

from qev import QEV

m = QEV.from_pretrained("labqoat/squad-0.8b", specialists=True)     # router + experts (fast CUDA engine)
m.decide(state, questions, per_question=True)                        # System One answers
m.last_routes                                                        # e.g. ["games"]

python qev_server.py --model labqoat/squad-0.8b --specialists     # POST http://127.0.0.1:8017/v1/systemone

Without specialists=True you get the trunk alone (not the old qev-0.8b: the trunk changed in this revision; pin the previous revision 2043fda for it). Experts need the fast CUDA engine (requirements-cuda.txt).

On a Mac or CPU

MLX (labqoat/squad-0.8b-MLX): router + 4 experts, about 50 ms per decision on an M4 Pro; same top answer as this runtime on 100% of 387 test questions (bf16).
GGUF (labqoat/squad-0.8b-GGUF): llama.cpp b11786 or newer (/v1/systemone), one LoRA file per expert, qev_gguf.py for routing; same route on 333/333 requests and the same top answer on 99.5% of questions (Q8_0, Metal).

Cache

All three runtimes keep a prefix cache across requests (on by default): a request that starts like an earlier one (same page, same goal and history) only computes the new tokens, and an exact repeat is answered from memory. Answers are the same with the cache on or off (same top answer and route; probabilities within bf16 noise). On a recorded tester-army session: CUDA model time 900 → 721 ms (exact repeats 0.08 ms), GGUF on an M4 Pro 25-30% less total time, MLX 22.0 → 12.9 s and memory capped (about 3 GB).

Files

model.safetensors (new trunk; embeddings, head and vision tower as before) and tokenizer / config files · specialists/ (manifest, router, 2-bit experts) · qev.py, qev_server.py, qev_fast/ (runner, server, CUDA engine with experts) · qev_image.py, qev_image_server.py (transformers image path, trunk only) · SHA256SUMS.

Limitations

Text decisions only were evaluated for this revision. The e2e expert is far from reliable on real apps (34 of 183 real-app test runs pass). Polish sentiment and QA (KLEJ) are behind basal-omni-pico. Probabilities are calibrated with the previous version's temperatures; check them on your own data before using them as error rates.

License: Apache-2.0.