d3-GGUF

d3 is a 26.09B-parameter multimodal decision model from the vLLM Semantic Router team, the largest of the Decision 3.0 family, including a 0.46B vision encoder. It is built on Qwen3.8-27B and released under Apache-2.0. It takes text or JSON input, optionally with several images and videos, and answers multiple Choice, Yes/No and Score questions about it in one call. It returns a probability for every option and generates no text. On the official Jev Decision Index 0.3.2 text board it scores 66.3, first of all 149 entries and 10.4 points above Decision 2.0's 27B model, ahead of Decider 31B (64.6) and Perplexity Decider v1.1 (62.8). It also ranks first of 33 entries on the vision board at 71.3 and scores 77.4 on the Perception Test video benchmark. On one AMD Instinct MI325X, median latency is 54.4 ms for a text request, 290 ms with an image and 500 ms with a 10-second video. vLLM Semantic Router's Engine mode with batching serves up to 27 one-question text requests per second, 1.5× AutoModel. Unlike the smaller models, d3 is also supported natively by vLLM itself (nightly builds, via vllm serve and the /v1/systemone endpoint), in addition to transformers (AutoModel or a pipeline) and vllm-sr serve -e vllm-sr/d3.

This is a Decision Model GGUF that supports the llama.cpp v1/systemone API endpoint.

Model Files

File Name	Quant Type	File Size	File Link	Description
d3.BF16.gguf	BF16	53.8 GB	Link	Full BF16 weights. Highest quality, largest file size.
d3.Q3_K_L.gguf	Q3_K_L	14.7 GB	Link	Lower quality but usable, good for low RAM availability.
d3.Q4_K_M.gguf	Q4_K_M	16.7 GB	Link	Good quality, default size for most use cases, recommended.
d3.Q5_K_M.gguf	Q5_K_M	19.3 GB	Link	High quality, recommended.
d3.Q6_K.gguf	Q6_K	22.1 GB	Link	Very high quality, near perfect, recommended.
d3.mmproj-bf16.gguf	mmproj-bf16	931 MB	Link	Multimodal projection file in BF16 format. Used for vision/language models.

Run with llama.cpp (SystemOne API)

1. Start the Server

Launch llama-server with the GGUF model and wait for the health check endpoint to report ready.

import subprocess, time, requests

MODEL_PATH = "/data/llama.cpp/gguf-new/d3.BF16.gguf"
BASE_URL = "http://127.0.0.1:8080"

server = subprocess.Popen(
    [
        "/data/llama.cpp/build/bin/llama-server",
        "-m", MODEL_PATH,
        "--host", "127.0.0.1",
        "--port", "8080",
        "-ngl", "99",
    ],
    stdout=open("/data/llama-server.log", "w"),
    stderr=subprocess.STDOUT,
)

for _ in range(120):
    try:
        if requests.get(f"{BASE_URL}/health", timeout=2).status_code == 200:
            print("server ready")
            break
    except requests.exceptions.RequestException:
        pass
    time.sleep(2)
else:
    print("server not ready, check /data/llama-server.log")

2. Query the /v1/systemone Endpoint

Send a POST request with a state context and structured questions (choice, noul, or score).

curl -s http://127.0.0.1:8080/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Customer message: I was charged twice for my order last week and nobody has replied.",
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "payments, charges, refunds, invoices",
          "shipping": "delivery, tracking, lost or late parcels",
          "technical": "bugs, errors, login problems"
        }
      },
      "angry": {
        "type": "noul",
        "instructions": "Is the customer angry?"
      },
      "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["can wait", "this week", "today", "right now"]
      }
    }
  }' | python3 -m json.tool

3. Example Response

{
    "model": "/data/llama.cpp/gguf-new/d3.BF16.gguf",
    "answers": {
        "route": {
            "type": "choice",
            "choice": "billing",
            "probabilities": {
                "billing": 0.9996618974570523,
                "shipping": 0.00013507157686214213,
                "technical": 0.0002030309660855959
            },
            "confidence": 0.9970256171728359
        },
        "angry": {
            "type": "noul",
            "noul": 0.7505827517439388
        },
        "urgency": {
            "type": "score",
            "score": 2.1734206644237215,
            "legend": {
                "0": "can wait",
                "1": "this week",
                "2": "today",
                "3": "right now"
            },
            "probabilities": {
                "0": 0.007391267944072573,
                "1": 0.1251241500290827,
                "2": 0.5541572316858958,
                "3": 0.31332735034094905
            },
            "confidence": 0.28797389363668036
        }
    },
    "usage": {
        "input_tokens": 350,
        "output_tokens": 0
    }
}

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Quantized using llama.cpp by prithivMLmods.