prithivMLmods commited on
Commit
cbffb3d
·
verified ·
1 Parent(s): b99aa80

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +28 -1
README.md CHANGED
@@ -1,8 +1,29 @@
1
  ---
2
  base_model:
3
  - vllm-sr/d3-mini
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  ---
5
 
 
 
 
 
 
 
 
6
  ## Run with llama.cpp (SystemOne API)
7
 
8
  ### 1. Start the Server
@@ -114,4 +135,10 @@ curl -s http://127.0.0.1:8080/v1/systemone \
114
  "output_tokens": 0
115
  }
116
  }
117
- ```
 
 
 
 
 
 
 
1
  ---
2
  base_model:
3
  - vllm-sr/d3-mini
4
+ license: apache-2.0
5
+ language:
6
+ - en
7
+ library_name: transformers
8
+ tags:
9
+ - text-generation-inference
10
+ - llama-cpp
11
+ - zero-shot-classification
12
+ - decision-model
13
+ - classification
14
+ - system-one
15
+ - multimodal
16
+ - vision
17
+ pipeline_tag: image-text-to-text
18
  ---
19
 
20
+ # **d3-mini-GGUF**
21
+
22
+ > **d3-mini** is a 4.54B-parameter multimodal decision model from the vLLM Semantic Router team, including a 0.33B vision encoder. It is built on Qwen3.5-4B and released under Apache-2.0. It takes text or JSON input, optionally with several images and videos, and answers multiple Choice, Yes/No and Score questions about it in one call. It returns a probability for every option and generates no text. On the official Jev Decision Index 0.3.2 text board it scores 52.6, which is 7.7 points above Decision 2.0's 4B model. It ranks first among similarly sized entries and sits on the Pareto frontier. It scores 63.2 on the vision board, first among the seven entries of similar size, and 70.3 on the Perception Test video benchmark. On one AMD Instinct MI325X, median latency is 18.1 ms for a text request, 112 ms with an image and 336 ms with a 10-second video. vLLM Semantic Router's Engine mode with batching serves up to 151 one-question text requests per second. It can be used through `transformers` (`AutoModel` or a pipeline) or served with `vllm-sr serve -e vllm-sr/d3-mini`.
23
+
24
+ > [!NOTE]
25
+ > This is a Decision Model GGUF that supports the llama.cpp `v1/systemone` API endpoint.
26
+
27
  ## Run with llama.cpp (SystemOne API)
28
 
29
  ### 1. Start the Server
 
135
  "output_tokens": 0
136
  }
137
  }
138
+ ```
139
+
140
+ ## llama.cpp
141
+
142
+ LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
143
+
144
+ Quantized using llama.cpp by [prithivMLmods](https://huggingface.co/prithivMLmods).