Instructions to use TypeScript/FastBrain-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TypeScript/FastBrain-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TypeScript/FastBrain-9B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("TypeScript/FastBrain-9B")
model = AutoModelForMultimodalLM.from_pretrained("TypeScript/FastBrain-9B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use TypeScript/FastBrain-9B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TypeScript/FastBrain-9B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf TypeScript/FastBrain-9B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf TypeScript/FastBrain-9B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf TypeScript/FastBrain-9B:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf TypeScript/FastBrain-9B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf TypeScript/FastBrain-9B:Q4_K_M

Use Docker

docker model run hf.co/TypeScript/FastBrain-9B:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use TypeScript/FastBrain-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TypeScript/FastBrain-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TypeScript/FastBrain-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/TypeScript/FastBrain-9B:Q4_K_M

SGLang

How to use TypeScript/FastBrain-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TypeScript/FastBrain-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TypeScript/FastBrain-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TypeScript/FastBrain-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TypeScript/FastBrain-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use TypeScript/FastBrain-9B with Ollama:
```
ollama run hf.co/TypeScript/FastBrain-9B:Q4_K_M
```
Unsloth Desktop

How to use TypeScript/FastBrain-9B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TypeScript/FastBrain-9B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TypeScript/FastBrain-9B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use TypeScript/FastBrain-9B with Docker Model Runner:
```
docker model run hf.co/TypeScript/FastBrain-9B:Q4_K_M
```

Lemonade

How to use TypeScript/FastBrain-9B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull TypeScript/FastBrain-9B:Q4_K_M

Run and chat with the model

lemonade run user.FastBrain-9B-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use TypeScript/FastBrain-9B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TypeScript/FastBrain-9B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TypeScript/FastBrain-9B:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use TypeScript/FastBrain-9B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf TypeScript/FastBrain-9B:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "TypeScript/FastBrain-9B:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

FastBrain-9B / README.md

TypeScript

Verify live leaderboard standings across 109 model entries, polish README.md, and add full Chinese README_zh.md

3787027 verified 3 days ago

preview code

Raw History Blame

39 kB

metadata

license: apache-2.0
language:
  - en
  - zh
library_name: transformers
pipeline_tag: text-generation
base_model: Cloudflare/clef-flash
datasets:
  - TypeScript/FastBrain-Instruct
tags:
  - fastbrain
  - clef
  - gguf
  - llama-cpp
  - sglang
  - systemone
  - decision-model
  - image-text-to-typed-output
  - decision-index
  - tool-calling
  - function-calling
  - intent-classification
  - structured-decision

FastBrain-9B

Single-Pass Non-Autoregressive System-1 Structured Decision Model (9.4B Parameters)

English | 简体中文 (Chinese README)

FastBrain-9B (9.4B total parameters: Qwen3.5-9B hybrid Gated DeltaNet + Full-Attention backbone + 116M JointSchemaHead) is a high-throughput, non-autoregressive System-1 structured decision model built to beat 27B–35B decision models at a fraction of their compute and latency.

Given an arbitrary JSON or multimodal state and one or more typed decision schemas (choice, noul binary verification, or score ordinal rubric), FastBrain-9B evaluates every question and candidate option **jointly in a single prefill forward pass (0 generated tokens)**—returning calibrated option probabilities, tool-call selections, guardrail verdicts, and open-set abstention decisions in 30–50 ms (14,600–19,300+ tok/s).

All fine-tuned backbone updates (104 projection layers across all 32 hybrid blocks) are merged directly into model-00001-of-00004.safetensors .. model-00004-of-00004.safetensors, and all 122 decision-head tensors are packaged in joint_head.safetensors (plus turnkey Q8_0 / Q4_K_M / BF16 GGUF builds for llama-server). No external LoRA or PEFT adapter is required at inference time.

Why `FastBrain-9B` Stands Out

Dominant #1 on Real-World Agent Security (safeagents-judge-bench v2 & v1): Ranks #1 / 8 overall on the official v2 7-Way Misuse Category Leaderboard (75.1% Macro-F1 · 85.8% Accuracy, beating 3x-larger 27.4B Cloudflare/clef by +18.8 pp and base clef-flash by +28.9 pp), catches 82 / 82 (100%) unsafe trajectories (98.5% AUROC · 99.6% Recall; 93.8% Calibrated Bal. Acc), and swept #1 / 9 on both Binary (97.4%) and Category (85.0%) tracks in v1.
Top-Tier Zero-Shot Generalization & Fastest Latency on LocalLLaMA/typed-decisions (94 Total Model Entries): Ranked #7 / 43 Overall Zero-Shot (#5 / 36 Open-Weight, #2 / 19 <=9B Open-Weight, and #1 across all Clef-architecture models of any size), while delivering 50 ms p50 latency—the #1 fastest model in the entire Top 28 (>70% accuracy tier) (66.7x faster than #1 Quyet-1.0-Large 3,335 ms, 7.1x faster than 27B Amos 1 large 357 ms). Also live on the Fitted Leaderboard at **#12 Tied (#14 / 51, 79.70% / 80.70% Acc, 0.0816 KL)**—ranking #1 in KL calibration and #1 <=9B among frozen-weight models (no weights changed).
Outperforms 3x-Larger 27B Models on Cloudflare Decision Index (v0.2.1): Achieves 80.55% 10-Benchmark Macro Mean (#2 / 7 Overall across all parameter scales), sweeping 10 / 10 benchmarks (#1 on every task) among <=9B models (+5.59 pp over base Clef-flash), outperforming 27B Jev (80.32%) overall, and beating 27.4B Cloudflare/clef head-to-head on 5 / 10 benchmarks.
Production-Ready Across SGLang, llama.cpp GGUF, and PyTorch 4-bit: Ships with a 1.90x faster fused FlashAttention decision head (max_logit_diff = 0.00000), lossless Q8_0.gguf (99.86% Top-1 agreement with BF16), and 6.73 GB Q4_K_M.gguf (98.78% agreement, runs in 6.78 GiB VRAM on a 16GB T4 or Apple Silicon Mac).

Benchmark Evaluation & Official Leaderboard Rankings

Official Live Leaderboard Standing Summary (Verified Across 109 Official Model Entries)

Official Leaderboard / Track	Live Entries	`FastBrain-9B` Score	Official Rank	Top Percentile	Key Models Outperformed (`Params` & Score)
`safeagents-judge-bench` (`v2`) (7-Way Category) (Merged PR #1 · Dataset Card)	8 Decision Models (+2 Baselines)	`75.1%` Macro-F1 `85.8%` Cat. Acc	#1 / 8 (Live)	Top 12.5% (Rank #1)	`Cloudflare/clef` (27.4B, #2 `56.3%` F1 / `73.7%` Acc), `vllm-sr/d3-mini` (#3 `51.2%`), `Cloudflare/clef-flash` (9.4B, #4 `46.2%`), `clef-omni` (30B, #5 `43.7%`)
`safeagents-judge-bench` (`v2` & `v1`) (Binary Guardrail) (Merged PR #1 · `v1` Tag)	8 Models (`v2`) 9 Models (`v1`)	`82/82` (`100%`) Traj. · `98.5%` AUROC (`93.8%` Cal. / `81.4%` Raw `v2` · `97.4%` `v1`)	`#1 / 9` (`v1` Live) `100%` Caught · #3 AUROC (`v2`)	Top 11.1% (`v1`) Top 37.5% AUROC (`v2`)	Beats `27.4B Cloudflare/clef` (`80.8%` Raw Bal. Acc), `clef-flash` (`96.7%` AUROC, `81/82` Traj.), `clef-omni` (`94.6%` AUROC, `81/82`), `d1-3B` (`68/82` Traj.)
`LocalLLaMA/typed-decisions` (Zero-Shot Overall) (Merged PR #58 · Trace)	43 Models (36 Open-Weight)	`75.25%` / `76.25%` Acc (KL `0.1508` · Brier `0.0795` · `50 ms`)	#7 / 43 (Live) #5 / 36 Open-Weight	Top 16.3% Overall Top 13.9% Open	`vllm-sr/d3` (27B, #8 `74.9%`), `Baseten Elephant 1` (9B, #9 `74.8%`), `Darwin-27B` (#10 `74.3%`), `Cloudflare/clef` (27B, #16 `72.8%`), `openjev` (27B, #17 `72.8%`)
`LocalLLaMA/typed-decisions` (Zero-Shot `<=9B` & Clef-Arch)	19 `<=9B` Open 4 Clef-Arch	`75.25%` / `76.25%` Zero-Shot (Distilled Raw: `77.35%`* · `50 ms`)*	#2 / 19 `<=9B` Open #1 Clef-Arch · #1 Speed	Top 10.5% (`<=9B`)	Beats 17 of 18 `<=9B` open models (`d3-mini` `74.3%`, `d3-flash` `71.7%`, `signpost-d-4b` `71.2%`, `clef-flash` `71.0%`, `d1-3B` `65.6%`) & `35B clef-omni` (`67.3%`)
`LocalLLaMA/typed-decisions` (Fitted on `train`) (PR #57 · Trace)	51 Models	`79.70%` / `80.70%` Acc (KL `0.0816`* · Brier `0.0437` · `50 ms`)*	#12 Tied (#14 / 51 Live) #1 Frozen `<=9B` · #1 Frozen KL	Top 23.5% Overall (Zero Weight Updates)	Beats 37 fine-tuned/specialist models: `Rev-gen1-midi` (25.6B, `79.45%`), `OpenDecider-small-td` (4B, `79.2%`), `OpenDecider-medium-td` (30B, `78.8%`), `Seb-9B` (`75.4%`)
Cloudflare Decision Index (`v0.2.1`) (10-Task Macro)	7 Models	`80.55%` Macro Mean	#2 / 7 Overall #1 / 10 (`<=9B` Sweep)	Top 28.6% Overall 10/10 `#1` (`<=9B`)	Outperforms `Jev` (27B, `80.32%`), `Kev 9B` (`76.51%`), `Clef-flash` (9.4B, `74.96%`), `DiffGemma` (26B, `69.14%`), and beats `27.4B Clef` on 5 / 10 tasks

Cloudflare Decision Index (`v0.2.1`) & Parameter Scale Comparison

Evaluated on the official Cloudflare Decision Index (v0.2.1) across 10 benchmarks (22,427 evaluation requests), FastBrain-9B (9.4B) achieves an overall Macro Mean of 80.55%:

Clean Sweep (10 / 10 #1 Ranks) Across All <=9B Models: Outperforms the same-size base model Clef-flash (9.4B, 74.96%) by +5.59 pp (including +31.67 pp on CLINC150+OOS and +15.55 pp on When2Call), Kev 9B (9B, 76.51%) by +4.04 pp, and Laya (0.4B, 58.09%) by +22.46 pp.
Outperforms 3x-Larger 27B-Class Models: Surpasses Jev (27B, 80.32%) and DiffusionGemma Jev (26B-A4B, 69.14%) in 10-benchmark Macro Mean, and beats the 3x-larger flagship Clef (27.4B, 81.81%) head-to-head on 5 out of 10 benchmarks (BFCL, API-Bank, CLINC150+OOS, Home appliances, and When2Call, claiming outright #1 overall across all model sizes on 4 of them).

Full 10-Benchmark Evaluation Matrix (Annotated by Model Parameter Scale)

Fine-Grained Decision Calibration & Open-Set Reliability

Summary Comparison by Model Parameter Scale

Benchmark	FastBrain-9B (9.4B · Ours)	Clef-flash (9.4B Base)	Kev 9B (9B)	Clef (27.4B)	Jev (27B)	DiffGemma (26B-A4B)	Laya (0.4B)
Params	9.41B	9.41B	9.0B	27.36B	27.0B	25.8B (4B)	0.42B
BFCL (Case Exact)	98.76% (#1)	98.76%	97.58%	98.47%	98.58%	97.64%	95.87%
ToolRet (nDCG@10)	67.26% (#1 9B)	67.26%	65.72%	68.18%	65.50%	45.28%	51.31%
API-Bank (Accuracy)	93.70% (#1)	93.11%	89.76%	91.93%	90.35%	85.24%	86.22%
BANKING77 (Macro-F1)	93.77% (#1 9B)	93.69%	91.77%	94.07%	93.58%	93.80%	62.66%
CLINC150+OOS (Macro-F1)	98.44% (#1)	66.77%	96.65%	97.43%	96.44%	94.64%	73.79%
Home Appl. (Case Exact)	97.73% (#1)	96.59%	88.64%	96.59%	88.64%	68.18%	23.86%
BRIGHT (nDCG@10)	38.58% (#1 9B)	38.49%	38.53%	48.03%	43.37%	18.69%	25.00%
Amazon ESCI (Macro-F1)	57.40% (#1 9B)	57.39%	53.75%	59.30%	56.60%	56.83%	52.84%
PhishNChips (Accuracy)	78.75% (#1 9B)	71.95%	72.10%	91.70%	88.20%	55.65%	60.45%
When2Call (Macro-F1)	81.13% (#1 9B)	65.58%	70.64%	72.37%	81.91%	75.44%	48.92%
Macro Mean (10 Tasks)	80.55% (#2 · #1 9B)	74.96%	76.51%	81.81%	80.32%	69.14%	58.09%

External Community Benchmarks (`safeagents-judge-bench` & `LocalLLaMA/typed-decisions`)

1. `harness-hub/safeagents-judge-bench` (`v2`: 661 Agent Tool Calls · 82 Unsafe Trajectories · 7 Categories)

Official v2 & v1 Leaderboard Verification (Merged PR #1 by @ybelkada · .eval_results/safeagents-judge-bench.yaml): Evaluated zero-shot via Hugging Face Jobs on NVIDIA A100-SXM4-80GB across 661 suspicious agent tool calls (536 malicious block calls across 82 unsafe trajectories + 125 benign allow calls):

Rank #1 / 8 Overall on 7-Way Misbehaviour Category Track (v2): Achieves 75.1% Category Macro-F1 and **85.8% Category Accuracy (567/661)**—leading #2 Cloudflare/clef (27.4B, 56.3% Macro-F1 / 73.7% Acc) by +18.8 pp Macro-F1 (+12.1 pp Acc), #3 vllm-sr/d3-mini (51.2%) by +23.9 pp, and base Cloudflare/clef-flash (46.2% / 57.6% Acc) by +28.9 pp Macro-F1 (+28.2 pp Acc).

100% (82 / 82) Unsafe Trajectories Caught & 98.5% AUROC on Binary Guardrail (v2): Blocks 534 / 536 (99.6%) malicious calls (100% recall on harness_introspection 272/272, credential_misuse 15/15, and sandbox_escape 4/4; 99.2% on eval_gaming 243/245) with 0 median leaked bad calls before catch. At the uncalibrated default threshold (P(block) >= 0.50), Balanced Accuracy is 81.4% (beating 27.4B Cloudflare/clef at 80.8%); calibrating the threshold (P(block) >= 0.961) reaches 93.8% Balanced Accuracy (5.6% False Block Rate) while still catching 82/82 (100%) unsafe trajectories. On the v1 (562-call) official release, FastBrain-9B ranked #1 / 9 on both Binary (97.4% Bal. Acc) and Category (85.0% Macro-F1) tracks.

Model (`safeagents-judge-bench v2` Official)	Params	Cat. Macro-F1 ↑	Cat. Acc ↑	AUROC ↑	Balanced Acc ↑	Recall ↑	False Blocks ↓	Traj. Caught ↑	P50 Latency (Hardware)
`TypeScript/FastBrain-9B` (`v2` Official · Default `t=0.50`)	9.4B	75.1% (#1 / 8)	85.8% (#1 / 8)	98.5%	81.4%	99.6%	36.8%	82 / 82 (100%)	327 ms (A100 · 104 ms G4)
`TypeScript/FastBrain-9B` (`v2` Calibrated · `t=0.961`)	9.4B	75.1% (#1)	85.8% (#1)	98.5%	93.8%	93.1%	5.6%	82 / 82 (100%)	327 ms (A100 · 104 ms G4)
`TypeScript/FastBrain-9B` (`v1` Official · 562 Calls)	9.4B	85.0% (#1 / 9)	92.3% (#1 / 9)	98.3%	97.4% (#1 / 9)	100.0%	5.1%	78 / 78 (100%)	104 ms (RTX PRO 6000)
`Cloudflare/clef`	27.4B	56.3% (#2)	73.7% (#2)	99.3%	80.8%	100.0%	38.4%	82 / 82 (100%)	636 ms (A100)
`vllm-sr/d3-mini`	1.8B	51.2% (#3)	65.2% (#3)	98.6%	85.6%	100.0%	28.8%	82 / 82 (100%)	168 ms (A100)
`Cloudflare/clef-flash` (Base Model)	9.4B	46.2% (#4)	57.6% (#4)	96.7%	89.3%	97.8%	19.2%	81 / 82 (98.8%)	326 ms (A100)
`Cloudflare/clef-omni`	30B-A3B	43.7% (#5)	56.4% (#5)	94.6%	87.7%	91.4%	16.0%	81 / 82 (98.8%)	285 ms (A100)
`knowledgator/gliclass-v4-edge`	0.4B	37.3% (#6)	52.8% (#7)	75.3%	67.6%	69.6%	34.4%	68 / 82 (82.9%)	101 ms (L4)
`fastino/GLiNER2.5-Decide`	0.5B	35.5% (#7)	62.0% (#6)	51.7%	57.6%	15.1%	0.0%	17 / 82 (20.7%)	117 ms (M5 Pro)
`LiquidAI/d1-3B`	3B	32.8% (#8)	45.7% (#8)	94.6%	83.6%	68.1%	0.8%	68 / 82 (82.9%)	35 ms (A100)

2. `LocalLLaMA/typed-decisions` (400 Test Cases · 2,000 Typed Decisions Across 4 Enterprise Workflows)

Official Live Leaderboard Note (43 Zero-Shot + 51 Fitted = 94 Model Entries):

Zero-Shot Official Live Rank #7 / 43 Overall (#5 / 36 Open-Weight · #2 / 19 <=9B Open-Weight · #1 Clef-Arch): Published in eval_submissions/typescript_fastbrain-9b.json and merged via PR #58, scoring 75.25% accuracy (0.1508 KL · 0.0795 Brier · 50 ms p50 latency) under bench/score_with_jobs.py (76.25% under tied-top scripts/score_model.py). With 50 ms p50 latency, FastBrain-9B is the #1 fastest model in the entire Top 28 (>70% accuracy tier)—66.7x faster than #1 Quyet-1.0-Large (3,335 ms), 10.5x faster than #2 meraGPT Decider 1 (526 ms), and 7.1x faster than 27B Amos 1 large (357 ms).

Fitted Official Live Rank #12 Tied (#14 / 51 Listed · #1 Frozen-Weight KL & <=9B): Published in eval_submissions/typescript_fastbrain-9b-fitted.json (PR #57), fitting only lightweight per-question priors and temperature on train (no weights changed) reaches 79.70% (80.70% tied-top) accuracy · 0.0816 KL · 0.0437 Brier · 50 ms, outperforming 37 fine-tuned and specialist models (including 25.6B Rev-gen1-midi 79.45%, LoRA-fine-tuned OpenDecider-small-td 79.2%, and 30B-A3B OpenDecider-medium-td 78.8%).

Model	Params	Overall Acc ↑	KL from Gold ↓	Brier ↓	`noul` Acc ↑	`choice` Acc ↑	`score` Acc ↑	Agent Trace ↑	Customer Svc ↑	Invoice Proc ↑	Security Inc ↑	P50 Latency ↓
`TypeScript/FastBrain-9B` (Fitted · No Weights Changed)	9.4B	79.70% / 80.70%	0.0816	0.0437	86.7% / 88.0%	76.5% / 77.2%	76.9% / 77.9%	75.4% / 76.6%	82.0% / 83.4%	83.4% / 83.6%	78.0% / 79.2%	50 ms
`TypeScript/FastBrain-9B` (Ordinal-KL Distilled Raw)	9.4B	77.35% (#1 Raw)	0.1039	0.0582	84.8%	74.8%	73.8%	71.2%	80.4%	82.6%	75.2%	50 ms
`TypeScript/FastBrain-9B` (Zero-Shot · Live #7 / 43)	9.4B	75.25% / 76.25%	0.1508	0.0795	83.2% / 84.3%	74.7% / 76.0%	69.8% / 70.4%	68.2% / 69.8%	77.6% / 78.8%	82.0%	73.2% / 74.4%	50 ms
`Amos 1 (large)`	27B	76.10%	0.1260	0.0690	83.2%	73.7%	72.6%	67.4%	78.8%	83.0%	75.2%	357 ms
`pplx-decider-v1.1-27b`	27B	75.30%	0.4410	0.1600	85.2%	73.7%	69.1%	66.4%	80.2%	80.2%	74.4%	—
`vllm-sr/d3`	27B	74.90%	0.4090	0.1460	84.7%	68.5%	72.4%	67.2%	76.6%	82.8%	73.0%	—
`Baseten Elephant 1`	9B	74.80%	0.1340	0.0690	—	—	—	—	—	—	—	352 ms
`Darwin-27B-ZTC`	27B	74.30%	0.2040	0.0970	—	—	—	—	—	—	—	—
`Cloudflare/clef`	27.4B	72.75%	0.1907	0.1032	84.5%	64.8%	69.9%	63.8%	74.2%	78.6%	74.4%	—
`vllm-sr/d3-flash`	9B	71.70%	0.3220	0.1310	—	—	—	—	—	—	—	—
`Cloudflare/clef-flash` (Base Model)	9.4B	71.00%	0.2093	0.1105	81.5%	71.3%	62.9%	62.0%	76.0%	76.4%	69.6%	—
`Cloudflare/clef-omni`	35B-A3B	67.30%	0.2469	0.1233	82.7%	63.5%	58.6%	55.2%	73.6%	74.2%	66.2%	—

Architecture

Hybrid Gated DeltaNet + Full-Attention Backbone (9.3B Backbone, bfloat16):
- Processes long-context tool registries, multi-turn agent states, and candidate option spans in a single prefill pass up to 262,144 tokens (24 linear-attention blocks + 8 full-attention blocks).
- Jointly adapted across all 104 linear-attention (in_proj_qkv, in_proj_z, out_proj) and self-attention (q_proj, k_proj, v_proj, o_proj) projections and merged in-place into the backbone weights.
Joint Schema Decision Head (joint_head.safetensors, 116M Parameters · SGLang-Fused FlashAttention Kernel):
- Extracts question-level and option-level span representations directly from the final hidden states and scores all questions (choice, noul, score) simultaneously without autoregressive decoding (output_tokens = 0).
- Vectorized CUDA Fast-Path (_forward_fast_cuda in joint_schema_model.py): Replaces per-span Python loops and dense SDPA masks with torch.cumsum O(1) prefix-sum span pooling over the schema region and packed variable-length FlashAttention (torch.ops.aten._flash_attention_forward with cu_seqlens), delivering a 1.90x decision-head kernel speedup (3.27 ms -> 1.72 ms, max_logit_diff = 0.00000).
- Ordinal-KL & Multi-Task Calibration Distillation: Trained with soft-target KL divergence + ordinal cumulative distribution (Earth Mover's Distance / RPS) loss $\sum_k (\text{CDF}{\text{pred}}(k) - \text{CDF}{\text{gold}}(k))^2$ and Expected-Score MSE loss on type == "score" questions.
Compile-Time AST Schema-Aware Margin Calibration (joint_schema_model.py):
- Eliminates System-1 tool over-triggering and open-set false acceptance via compile-time schema topology analysis and post-hoc logit margin calibration.

Quickstart

1. PyTorch / Transformers (`bfloat16` or `4-bit` / `8-bit` Quantized on Colab T4)

TypeScript/FastBrain-9B is a drop-in replacement for Cloudflare/clef-flash using the standard joint_schema_model runtime. It supports both full-precision bfloat16 (18.82 GB, for >=24 GB GPUs such as L4 / A100 / RTX 4090 / PRO 6000) and mixed-precision load_in_4bit=True (~6.8 GB VRAM, runs on a free Google Colab T4 16GB GPU) or load_in_8bit=True (~11.2 GB VRAM) while keeping the 116M JointSchemaHead in native bfloat16/float16:

from huggingface_hub import snapshot_download
import sys
import torch

model_dir = snapshot_download("TypeScript/FastBrain-9B", ignore_patterns=["*.gguf"])
sys.path.insert(0, model_dir)
import joint_schema_model as clef

# Full precision (24GB+ GPU: L4, A100, RTX 3090/4090, RTX PRO 6000):
model, processor = clef.load_release_model(model_dir, device="cuda", dtype=torch.bfloat16)

# Or 4-bit NF4 mixed-precision (8GB–16GB GPU, e.g., free Google Colab T4):
# model, processor = clef.load_release_model(model_dir, device="cuda", dtype=torch.float16, load_in_4bit=True)

response = clef.systemone(model, processor, {
    "model": "TypeScript/FastBrain-9B",
    "state": {
        "query": "Can you check my account balance?",
        "tools": ["get_balance", "transfer_funds"],
    },
    "questions": {
        "q": {
            "type": "choice",
            "query": "Select the best next action.",
            "criteria": {
                "a": "Call <tool_call>{\"name\": \"get_balance\", \"arguments\": {}}</tool_call>",
                "b": "Could you please provide your account ID or authentication token first?",
                "c": "I am unable to access personal banking records.",
            },
        },
    },
})
print(response)

2. `llama.cpp` GGUF (`Q8_0` / `Q4_K_M` / `BF16`) via `/v1/systemone`

This repository provides official llama.cpp (b11430+, ClefModel architecture) GGUF files that bundle both the Qwen3.5-9B hybrid backbone and the 116M JointSchemaHead (dec.* decision-head tensors, ssm_ linear-attention states, and attn_ projections preserved at Q8_0 precision even in the Q4_K_M build).

Required llama.cpp Version & ClefModel Batch Flags:

Minimum version: Use llama.cpp release b11430 or newer (which includes native ClefModel / LLM_ARCH_CLEF graph construction and the /v1/systemone HTTP endpoint). Standard Ollama or older llama.cpp builds without ClefModel support cannot execute dec.* decision-head tensors.

Set -b and -ub to match your maximum prompt length (e.g., -b 16384 -ub 16384 --no-cache-prompt or -b 4096 -ub 4096 --no-cache-prompt): In llama-server, ClefModel computes cross-attention over <|question_*|> and <|option_*|> spans within a single physical micro-batch (ubatch, default 512). Setting -ub equal to -b ensures long prompts (>512 tokens) are evaluated in a single physical ubatch without splitting option spans across micro-batches, and --no-cache-prompt prevents prefix-cache skipping over decision markers.

Step 1: Download Prebuilt `llama-server` (`b11430`+) and GGUF Weights

# 1. Download prebuilt llama.cpp (b11430+) with ClefModel & /v1/systemone support
#    - Linux CUDA 13.x (Colab / Ubuntu): llama-b11430-bin-ubuntu-cuda-13.4-x64.tar.gz
#    - Linux CUDA 12.x:                  llama-b11430-bin-ubuntu-cuda-12.8-x64.tar.gz
#    - macOS Apple Silicon (M1/M2/M3/M4): llama-b11430-bin-macos-arm64.tar.gz
curl -sL https://github.com/ggml-org/llama.cpp/releases/download/b11430/llama-b11430-bin-ubuntu-cuda-13.4-x64.tar.gz | tar -xz

# 2. Download FastBrain-9B GGUF (choose Q8_0 [9.66 GB, lossless] or Q4_K_M [6.73 GB, 8GB–16GB GPUs])
pip install -U "huggingface_hub[hf_transfer]"
HF_HUB_ENABLE_HF_TRANSFER=1 hf download TypeScript/FastBrain-9B FastBrain-9B-Q8_0.gguf --local-dir .

Step 2: Launch `llama-server` on the `/v1/systemone` Endpoint

# On Linux / Google Colab, ensure CUDA driver libraries are in LD_LIBRARY_PATH:
export LD_LIBRARY_PATH="$(pwd)/llama-b11430:/usr/lib64-nvidia:/usr/local/cuda/lib64:${LD_LIBRARY_PATH}"

# Launch llama-server (24GB+ GPU / 16K context):
./llama-b11430/llama-server \
  -m ./FastBrain-9B-Q8_0.gguf \
  -ngl 99 -c 16384 -b 16384 -ub 16384 -np 1 -fa on --no-cache-prompt \
  --port 8080

# Or on a 12GB–16GB GPU (e.g., Colab Free Tesla T4, 4K context with cuda-12.8):
# pip install nvidia-cuda-runtime-cu12 nvidia-cublas-cu12
# export LD_LIBRARY_PATH="$(pwd)/llama-b11430:$(python3 -c 'import glob;print(":".join(glob.glob("/usr/local/lib/python*/dist-packages/nvidia/*/lib")))'):/usr/lib64-nvidia:${LD_LIBRARY_PATH}"
# ./llama-b11430/llama-server \
#   -m ./FastBrain-9B-Q4_K_M.gguf \
#   -ngl 99 -c 4096 -b 4096 -ub 4096 -np 1 -fa on --no-cache-prompt \
#   --port 8080

# Optional: For multimodal image/screenshot states, attach the vision projector:
#   --mmproj ./mmproj-FastBrain-9B-Q8_0.gguf

Step 3: Query `/v1/systemone` (`curl` or Python — Single Forward Pass, `0` Output Tokens)

curl http://127.0.0.1:8080/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "model": "FastBrain-9B",
    "state": {
      "ticket_id": "ORD-99281",
      "customer_message": "I was charged twice for my subscription last week and nobody has replied to my ticket."
    },
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Payments, refunds, duplicate charges, or invoices",
          "technical": "Bugs, crashes, or service outages",
          "account": "Password reset or login issues"
        }
      },
      "urgent": {
        "type": "noul",
        "instructions": "Is the customer reporting a financial or urgent issue?"
      }
    }
  }'

Python Client Example (requests / urllib):

import requests

resp = requests.post(
    "http://127.0.0.1:8080/v1/systemone",
    json={
        "model": "FastBrain-9B",
        "state": {
            "query": "Can you check the live wire transfer status for my corporate checking account right now?",
            "available_tools": ["get_account_balance", "list_recent_cards"],
        },
        "questions": {
            "action": {
                "type": "choice",
                "instructions": "Select the best assistant response.",
                "criteria": {
                    "direct_answer": "Your wire transfer has been completed.",
                    "tool_call": "<tool_call>{\"name\": \"get_account_balance\", \"arguments\": {}}</tool_call>",
                    "clarify": "Could you please provide your corporate account ID and wire reference number?",
                    "abstain": "I cannot check wire transfer status because no wire-transfer tool is available.",
                },
            }
        },
    },
    timeout=30,
).json()

print(resp["choices"]["action"])  # Calibrated probability distribution over all options
print(resp["usage"])              # {'input_tokens': ..., 'output_tokens': 0, 'total_tokens': ...}

3. SGLang (`lmsysorg/sglang:dev-clef`) via `/v1/systemone`

TypeScript/FastBrain-9B is natively supported by SGLang's JointSchemaPooler (lmsysorg/sglang:dev-clef, sgl-project/sglang commit bd2d73daa5af), which executes the 9B backbone in uncached raw-K/V FlashAttention embedding mode and scores all questions and options across concurrent requests using vectorized prefix-sum span pooling (torch.cumsum) and packed variable-length FlashAttention (cu_seqlens):

docker run --gpus all \
  --shm-size 32g \
  -p 30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --ipc=host \
  lmsysorg/sglang:dev-clef \
  sglang serve \
    --model-path TypeScript/FastBrain-9B \
    --host 0.0.0.0 \
    --port 30000

Query SGLang's /v1/systemone endpoint directly (and optionally wrap with joint_schema_model.apply_schema_aware_calibration(req, resp["answers"]) on the client side):

curl http://localhost:30000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"model":"TypeScript/FastBrain-9B","state":"My Stripe integration keeps failing. Please help ASAP.","questions":{"urgency":{"type":"noul","instructions":"Does this message express urgency?"}}}'

Inference Engine Comparison: `SGLang (dev-clef)` vs. `llama.cpp (b11430)` vs. `PyTorch`

Measured head-to-head on the same NVIDIA RTX PRO 6000 Blackwell Server Edition (G4) GPU across five decision workloads (CLINC150+OOS, When2Call, PhishNChips, LocalLLaMA/typed-decisions, and safeagents-judge-bench):

Workload / Metric	SGLang `dev-clef` (BF16 · JointSchemaPooler)	`llama.cpp b11430` (GGUF Q8_0 · ClefModel)	`llama.cpp b11430` (GGUF Q4_K_M · ClefModel)	PyTorch Native (BF16 · `joint_schema_model.py`)
Model Load / Cold Start	11.11 s	2.56 s	2.00 s	11.11 s
GPU VRAM (16K ctx)	18.18 GiB (weights) / ~20.5 GiB	12.55 GiB	10.30 GiB	20.3–24.7 GiB
Runs on 16GB T4 GPU?	No (BF16 only, TP=1)	Yes (`9.03 GiB` @ 4K)	Yes (`6.78 GiB` @ 4K)	Yes (`load_in_4bit`: `6.8 GiB`)
Top-1 Agreement vs. BF16	100.00% (max diff 0.0078)	99.86% (1,471/1,473)	98.78% (1,455/1,473)	100.00% (Ref)
1. Short Routing (293 tok, K=2)	30.01 ms (9,763 tok/s)	32.82 ms (8,929 tok/s)	31.69 ms (9,245 tok/s)	43.23 ms (6,778 tok/s)
2. Typed Decisions (736 tok, K=5)	50.29 ms (14,635 tok/s)	62.40 ms (11,795 tok/s)	61.80 ms (11,909 tok/s)	63.58 ms (11,576 tok/s)
3. When2Call (1,068 tok, K=1)	66.06 ms (16,167 tok/s)	87.78 ms (12,167 tok/s)	89.34 ms (11,955 tok/s)	77.77 ms (13,733 tok/s)
4. PhishNChips (1,303 tok, K=9)	75.86 ms (17,176 tok/s)	108.55 ms (12,004 tok/s)	109.97 ms (11,849 tok/s)	90.32 ms (14,426 tok/s)
5. SafeAgents Judge (2,217 tok, K=2)	125.85 ms (17,616 tok/s)	181.70 ms (12,201 tok/s)	183.50 ms (12,082 tok/s)	138.06 ms (16,058 tok/s)
Batch `B=4` Typed Decisions (2,944 tok, 20 dec)	152.39 ms (19,319 tok/s · 131.2 dec/s)	248.50 ms (11,847 tok/s · 80.5 dec/s)	245.20 ms (12,006 tok/s · 81.6 dec/s)	175.85 ms (16,742 tok/s · 113.7 dec/s)

GGUF Quantization Accuracy Retention & Speed Benchmarks

1. Decision Index 10-Benchmark Test-Set Accuracy Retention (`BF16` vs. `Q8_0` vs. `Q4_K_M`)

Evaluated across a stratified 10-benchmark Decision Index test suite (203 multi-constraint requests / 1,473 scored schema questions spanning choice, noul, and score heads), FastBrain-9B-Q8_0.gguf (9.66 GB) is virtually lossless (99.86% Top-1 decision agreement with native bfloat16, -0.03 pp score delta), while FastBrain-9B-Q4_K_M.gguf (6.73 GB, with dec.* head preserved in Q8_0) retains 98.78% Top-1 agreement and -0.20 pp calibrated 10-task mean:

Benchmark / Metric	PyTorch bfloat16 (18.82 GB)	GGUF Q8_0 (9.66 GB)	GGUF Q4_K_M (6.73 GB · Q8 Head)
Top-1 Agreement vs. BF16 (1,473 q)	100.00% (Ref)	99.86% (1,471/1,473)	98.78% (1,455/1,473)
Probability MAE (\|p_q - p_bf16\|)	0.00000	0.00203 (0.20%)	0.00387 (0.39%)
10-Task Mean (Single-Ckpt Raw)	82.37%	82.34% (-0.03 pp)	82.56% (+0.19 pp)
10-Task Mean (Calibrated)	83.68%	83.65% (-0.03 pp)	83.48% (-0.20 pp)
├─ 1. BFCL (AST Tool Selection)	100.00%	100.00% (0.00)	100.00% (0.00)
├─ 2. ToolRet (20-Tool Reranking)	73.15%	73.70% (+0.55)	75.60% (+2.45)
├─ 3. API-Bank (Stateful Tool Call)	95.00%	95.00% (0.00)	95.00% (0.00)
├─ 4. BANKING77 (77-Way Intent)	100.00%	100.00% (0.00)	100.00% (0.00)
├─ 5. CLINC150+OOS (Intent + OOS)	93.94%	93.94% (0.00)	93.94% (0.00)
├─ 6. Home Appliances (Control)	90.00%	90.00% (0.00)	90.00% (0.00)
├─ 7. BRIGHT (Reasoning Retrieval)	41.93%	41.00% (-0.93)	40.31% (-1.62)
├─ 8. Amazon ESCI (Relevance)	73.89%	73.89% (0.00)	75.79% (+1.90)
├─ 9. PhishNChips (9-Probe Sec)	85.00%	85.00% (0.00)	85.00% (0.00)
└─ 10. When2Call (4-Way Gov)	75.00% (70.83%)	75.00% (0.00)	72.92% (-2.08)

2. Measured Hardware Footprint & Inference Speed (`llama-server b11430`)

Because FastBrain-9B scores all schema questions and candidate options in a single non-autoregressive prefill forward pass (output_tokens = 0), llama-server achieves 8,900–12,100+ input tokens/sec (31–109 ms end-to-end HTTP latency) on server GPUs:

Workload / Metric	Q8_0.gguf (PRO 6000)	Q4_K_M.gguf (PRO 6000)	Q8_0.gguf (Tesla T4 16G)	Q4_K_M.gguf (Tesla T4 16G)	PyTorch BF16 (PRO 6000)
Model Load Time	2.56 s	2.00 s	5.43 s	5.10 s	~11.5 s
Peak GPU VRAM	12.55 GiB (16K ctx)	10.30 GiB (16K ctx)	9.03 GiB (4K ctx)	6.78 GiB (4K ctx)	20.3–24.7 GiB
Intent + OOS (293 tok, K=2)	32.82 ms (8,928 tok/s)	31.69 ms (9,245 tok/s)	349.32 ms (839 tok/s)	342.98 ms (854 tok/s)	~35.0 ms
When2Call (1,068 tok, K=1)	87.78 ms (12,167 tok/s)	89.34 ms (11,955 tok/s)	1,149.8 ms (929 tok/s)	1,154.3 ms (925 tok/s)	~92.0 ms
PhishNChips (1,303 tok, K=9)	108.55 ms (12,004 tok/s)	109.97 ms (11,849 tok/s)	1,430.8 ms (911 tok/s)	1,440.5 ms (905 tok/s)	~115.0 ms

Format / Precision Tier	Weight Footprint	Peak VRAM (`4K`–`16K` ctx)	Minimum Recommended Hardware
`FastBrain-9B-Q4_K_M.gguf` (4-bit + `Q8_0` Head)	`6.73 GB`	`~6.8 – 10.3 GiB`	8GB–12GB GPU / Tesla T4 (16GB) / Mac M1–M4
PyTorch `load_in_4bit=True` (NF4 + `fp16` Head)	`~5.5 GB`	`~6.2 – 6.8 GiB`	Tesla T4 (16GB) / RTX 3060–4060 (8GB+)
`FastBrain-9B-Q8_0.gguf` (Full 8-bit GGUF)	`9.66 GB`	`~9.0 – 12.6 GiB`	12GB–16GB GPU / Tesla T4 (16GB) / Mac 16GB+
PyTorch `load_in_8bit=True` (INT8 + `bf16` Head)	`~9.6 GB`	`~11.2 GiB`	Tesla T4 (16GB) / RTX 4070 Ti (12GB+)
Native `bfloat16` (`safetensors` / `BF16.gguf`)	`18.82 GB` (`18.16 GB`)	`20.3 – 24.7 GiB`	NVIDIA L4 (24GB) / A100 / RTX 3090–4090 / PRO 6000

Repository Structure

File	Size	Description
`model-00001-of-00004.safetensors` .. `00004`	`18.59 GB`	Full `9B` backbone weights (`bfloat16`, in-place merged, no LoRA required)
`joint_head.safetensors` & `joint_head_config.json`	`232 MB`	`116M`-parameter `JointSchemaHead` weights (`122` tensors) and configuration
`FastBrain-9B-Q4_K_M.gguf`	`6.73 GB`	Recommended 4-bit GGUF (`dec.*` decision head, `ssm_`, and `attn_` preserved at `Q8_0`)
`FastBrain-9B-Q8_0.gguf`	`9.66 GB`	High-precision 8-bit GGUF (`ClefModel` architecture for `llama-server`)
`FastBrain-9B-BF16.gguf`	`18.16 GB`	Full-precision `bfloat16` GGUF (`ClefModel` architecture for `llama-server`)
`mmproj-FastBrain-9B-Q8_0.gguf` & `BF16.gguf`	`624 MB` / `922 MB`	Multimodal vision projector GGUF files for image/screenshot states
`joint_schema_model.py`	`25 KB`	`ClefModel`, `JointSchemaHead`, `load_release_model(load_in_4bit/8bit)`, and `systemone()` runtime
`eval_submissions/typescript_fastbrain-9b.json`	`901 KB`	Full 400-case / 2,000-decision Zero-Shot prediction trace for `LocalLLaMA/typed-decisions` (Merged PR #58)
`eval_submissions/typescript_fastbrain-9b-fitted.json`	`902 KB`	Full 400-case / 2,000-decision Fitted (`train`-calibrated) prediction trace for `LocalLLaMA/typed-decisions` (PR #57)
`eval_submissions/TypeScript__FastBrain-9B*.jsonl`	`110 KB`	Full 661-call (`v2`) binary & 7-way category prediction traces for `harness-hub/safeagents-judge-bench` (Merged PR #1)
`.eval_results/*.yaml`	`2.3 KB`	Official Hugging Face Hub benchmark evaluation metadata (`typed-decisions` & `safeagents-judge-bench v2`)
`README.md` & `README_zh.md`	`32 KB`	Full technical documentation in English and Simplified Chinese
`config.json`, `tokenizer.json`, `processor_config.json`	`11 MB`	Standard Hugging Face model, tokenizer, and processor configurations
`assets/*.png`	`1.2 MB`	High-resolution technical benchmark and calibration charts

FastBrain-9B

Why FastBrain-9B Stands Out

Benchmark Evaluation & Official Leaderboard Rankings

Official Live Leaderboard Standing Summary (Verified Across 109 Official Model Entries)

Cloudflare Decision Index (v0.2.1) & Parameter Scale Comparison

Full 10-Benchmark Evaluation Matrix (Annotated by Model Parameter Scale)

Fine-Grained Decision Calibration & Open-Set Reliability

Summary Comparison by Model Parameter Scale

External Community Benchmarks (safeagents-judge-bench & LocalLLaMA/typed-decisions)

1. harness-hub/safeagents-judge-bench (v2: 661 Agent Tool Calls · 82 Unsafe Trajectories · 7 Categories)

2. LocalLLaMA/typed-decisions (400 Test Cases · 2,000 Typed Decisions Across 4 Enterprise Workflows)

Architecture

Quickstart

1. PyTorch / Transformers (bfloat16 or 4-bit / 8-bit Quantized on Colab T4)

2. llama.cpp GGUF (Q8_0 / Q4_K_M / BF16) via /v1/systemone

Step 1: Download Prebuilt llama-server (b11430+) and GGUF Weights

Step 2: Launch llama-server on the /v1/systemone Endpoint

Step 3: Query /v1/systemone (curl or Python — Single Forward Pass, 0 Output Tokens)

3. SGLang (lmsysorg/sglang:dev-clef) via /v1/systemone

Inference Engine Comparison: SGLang (dev-clef) vs. llama.cpp (b11430) vs. PyTorch

GGUF Quantization Accuracy Retention & Speed Benchmarks

1. Decision Index 10-Benchmark Test-Set Accuracy Retention (BF16 vs. Q8_0 vs. Q4_K_M)

2. Measured Hardware Footprint & Inference Speed (llama-server b11430)

Repository Structure

Why `FastBrain-9B` Stands Out

Cloudflare Decision Index (`v0.2.1`) & Parameter Scale Comparison

External Community Benchmarks (`safeagents-judge-bench` & `LocalLLaMA/typed-decisions`)

1. `harness-hub/safeagents-judge-bench` (`v2`: 661 Agent Tool Calls · 82 Unsafe Trajectories · 7 Categories)

2. `LocalLLaMA/typed-decisions` (400 Test Cases · 2,000 Typed Decisions Across 4 Enterprise Workflows)

1. PyTorch / Transformers (`bfloat16` or `4-bit` / `8-bit` Quantized on Colab T4)

2. `llama.cpp` GGUF (`Q8_0` / `Q4_K_M` / `BF16`) via `/v1/systemone`

Step 1: Download Prebuilt `llama-server` (`b11430`+) and GGUF Weights

Step 2: Launch `llama-server` on the `/v1/systemone` Endpoint

Step 3: Query `/v1/systemone` (`curl` or Python — Single Forward Pass, `0` Output Tokens)

3. SGLang (`lmsysorg/sglang:dev-clef`) via `/v1/systemone`

Inference Engine Comparison: `SGLang (dev-clef)` vs. `llama.cpp (b11430)` vs. `PyTorch`

1. Decision Index 10-Benchmark Test-Set Accuracy Retention (`BF16` vs. `Q8_0` vs. `Q4_K_M`)

2. Measured Hardware Footprint & Inference Speed (`llama-server b11430`)