Explicit `position_ids` passed to `generate()` crash on the first decoding step under Transformers 4.57.6. The generation loop keeps the prompt-length positions, while `prepare_inputs_for_generation()` indexes the next cache position beyond their end.
Extend explicit positions after generation updates only when the parent implementation leaves them unchanged. Preserve each row's final position and advance by the actual number of accepted tokens. Transformers 5.22.1 already extends positions, so it retains its existing behavior.
Validation:
- Seven new tests pass under both Transformers 4.57.6 and 5.22.1. All seven fail under 4.57.6 before this fix.
- Coverage includes cached and uncached decoding, left padding, custom per-row positions, beam search, multi-token updates, and caller-input preservation. Generation is checked against independent full-forward decoding.
- Existing 65-case suites introduce no additional failures: 4.57.6 has 62 passes, 2 known unrelated failures and 1 skip; …
- Fix generate() and attention masking under Transformers 5.x (b9e4394ccce54f7206da4358cd7987cb57efd893)
- Add DFlash2 inference fixes (warm-cache verify mask, logits_to_keep, chunked MLP) (c4bb0c1fad49ddce9de8cf9849e2d7ffa3b660f0)
- Rename MLP chunk env var to RIPPLE_MLP_CHUNK (c178f879bfc6a52f93f9adb467478193ec3689b7)
- Fix StaticCache causal mask and labels+logits_to_keep loss (8662efad14a490205c62dc795ecfd70c46807aff)
- Fix StaticCache causal mask and labels+logits_to_keep loss (2c3501bf7bf0d4f4eed89a5b417524f138d83091)
- fix(ricursive): mask from actual cache positions; keep labeled loss with logits_to_keep (0d2fa09828767ed003d41a8df08ad377d7b0ea17)
- Align the causal mask to the keys the layer holds and forward cache positions (ee974ea44fc20eb18134822bbd80e9ea1a0989ca)
Co-authored-by: Qubitium <Qubitium@users.noreply.huggingface.co>
Set Ripple preview tokenizer context to 1Mi tokens (#7)
- Set Ripple preview context metadata to 1Mi tokens (e3b388ad5c1d40f52db47297893158a1315283f8)
- Preserve the original 2Mi model positional capacity (0f78007cf2bf7eb5792580925b479bf904e2d056)
Apply the server changes from ripple-inference PR #18 (08fd9f4383). Default target context to 1048576, retain draft window 16384, and enable the stock SGLang positional-limit override required for the draft. Update the server README.
https://github.com/ricursive/ripple-inference/pull/18
Fix dropped parallel tool calls in speculative streams
Apply ripple-inference PR #17 (d68cca40e1). Reconcile streamed tool calls at finish so speculative deltas cannot drop trailing calls or argument suffixes. Validated with CPU parser and chat-format regression tests on stock SGLang 0.5.19.
https://github.com/ricursive/ripple-inference/pull/17
