orestis-z commited on
Commit
08955f0
·
verified ·
1 Parent(s): e5c28fa

Link required vLLM PRs (#52184, #55133) in model card

Browse files
Files changed (1) hide show
  1. README.md +10 -5
README.md CHANGED
@@ -71,11 +71,16 @@ vllm serve thinkingmachines/Inkling-Small-NVFP4 \
71
  --trust-remote-code
72
  ```
73
 
74
- > **Note:** Inkling currently requires `--enforce-eager` (CUDA-graph capture of
75
- > the custom ops crashes). The drafter uses the full padded vocab (draft
76
- > `vocab_size` 201024 vs. the verifier's logical 200058); loading it in vLLM
77
- > needs [vllm-project/vllm#55133](https://github.com/vllm-project/vllm/pull/55133),
78
- > which treats a padded/full-vocab draft as an identity map (no `d2t` required).
 
 
 
 
 
79
 
80
  ## Training
81
 
 
71
  --trust-remote-code
72
  ```
73
 
74
+ > **Required vLLM patches** (both open at time of writing):
75
+ > - [vllm-project/vllm#52184](https://github.com/vllm-project/vllm/pull/52184) —
76
+ > adds Eagle3 hidden-state extraction support for Inkling (the verifier must
77
+ > expose the aux hidden states this drafter consumes).
78
+ > - [vllm-project/vllm#55133](https://github.com/vllm-project/vllm/pull/55133) —
79
+ > treats a full/padded-vocab draft (draft `vocab_size` 201024 vs. the verifier's
80
+ > logical 200058) as an identity map, so no `d2t` mapping is required to load it.
81
+ >
82
+ > Also note: Inkling currently requires `--enforce-eager` (CUDA-graph capture of
83
+ > the custom ops crashes).
84
 
85
  ## Training
86