Link required vLLM PRs (#52184, #55133) in model card
Browse files
README.md
CHANGED
|
@@ -71,11 +71,16 @@ vllm serve thinkingmachines/Inkling-Small-NVFP4 \
|
|
| 71 |
--trust-remote-code
|
| 72 |
```
|
| 73 |
|
| 74 |
-
> **
|
| 75 |
-
>
|
| 76 |
-
>
|
| 77 |
-
>
|
| 78 |
-
>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
|
| 80 |
## Training
|
| 81 |
|
|
|
|
| 71 |
--trust-remote-code
|
| 72 |
```
|
| 73 |
|
| 74 |
+
> **Required vLLM patches** (both open at time of writing):
|
| 75 |
+
> - [vllm-project/vllm#52184](https://github.com/vllm-project/vllm/pull/52184) —
|
| 76 |
+
> adds Eagle3 hidden-state extraction support for Inkling (the verifier must
|
| 77 |
+
> expose the aux hidden states this drafter consumes).
|
| 78 |
+
> - [vllm-project/vllm#55133](https://github.com/vllm-project/vllm/pull/55133) —
|
| 79 |
+
> treats a full/padded-vocab draft (draft `vocab_size` 201024 vs. the verifier's
|
| 80 |
+
> logical 200058) as an identity map, so no `d2t` mapping is required to load it.
|
| 81 |
+
>
|
| 82 |
+
> Also note: Inkling currently requires `--enforce-eager` (CUDA-graph capture of
|
| 83 |
+
> the custom ops crashes).
|
| 84 |
|
| 85 |
## Training
|
| 86 |
|