Follow-up: the exact-capacity static-mask case is now covered. When the mask width equals `past_length + input_length`, its final slot distinguishes a full prompt with unused cache capacity from a new-only token block. A mask with unused capacity at the end keeps the full-prompt path.
[GPT-QModel #3574](https://github.com/ModelCloud/GPTQModel/pull/3574) contains regression tests for both exact-capacity and longer padded masks, plus existing new-only block coverage. It builds on merged [GPT-QModel #3571](https://github.com/ModelCloud/GPTQModel/pull/3571). The Ripple suite passed **35 tests** against this upstream source (`tests/test_ricursive_support.py`).
Handle capacity-padded masks for cached full prompts
Follow-up to the initial PR: merged [GPT-QModel #3571](https://github.com/ModelCloud/GPTQModel/pull/3571) exposed another `cache_position=None` case. An attention mask can be padded to a static cache's capacity, so `mask_length > input_length` alone does not prove that `input_ids` contains only new tokens. The updated code treats only `mask_length == past_length + input_length` as evidence for a new-only block. A regression test and the exact before/after reproduction are in [GPT-QModel #3574](https://github.com/ModelCloud/GPTQModel/pull/3574).
Validation against this updated upstream source: **34 passed** in the tiny Ripple suite (`tests/test_ricursive_support.py`); the extracted upstream generation method also passed all four prompt/block cases.
Fix warm-cache new token blocks in generation preparation
## Problem
`RicursiveForCausalLM.prepare_inputs_for_generation` assumes every multi-token
`input_ids` input includes the cached prefix when `cache_position` is absent.
If the cache already holds 10 tokens and a caller supplies only two new tokens,
the calculated new length is -8 and `torch.arange` raises
`RuntimeError: upper bound and lower bound inconsistent with step sign`.
## Fix
Distinguish new-only token blocks from full prompts containing cached tokens.
An input block no longer than the cache is new; a longer input with an attention
mask spanning the cache is also new. Keep existing full-prompt slicing when
neither clue is present.
## Validation
- The three regression cases pass against this patched upstream function:
short new block, longer new block with a spanning mask, and full prompt.
- The tiny-model Ripple suite passed against this source:
`GPTQMODEL_RIPPLE_REF=<patched Ripple checkout> python -m pytest -q -o log_cli=false tests/test_ricursive_support.py`
(**33 …
2 days ago
Fix explicit position_ids generation on Transformers 4 (#8)
- Fix explicit position_ids generation on Transformers 4 (9f61852322e86b4f09b10003445642c03dc5975f)
- Keep the generation regression harness outside the model repository (48f6e3cca18a1e4937f830a2dbbc60906e5ca1ca)
- Fix generate() and attention masking under Transformers 5.x (b9e4394ccce54f7206da4358cd7987cb57efd893)
- Add DFlash2 inference fixes (warm-cache verify mask, logits_to_keep, chunked MLP) (c4bb0c1fad49ddce9de8cf9849e2d7ffa3b660f0)
- Rename MLP chunk env var to RIPPLE_MLP_CHUNK (c178f879bfc6a52f93f9adb467478193ec3689b7)
- Fix StaticCache causal mask and labels+logits_to_keep loss (8662efad14a490205c62dc795ecfd70c46807aff)
- Fix StaticCache causal mask and labels+logits_to_keep loss (2c3501bf7bf0d4f4eed89a5b417524f138d83091)
- fix(ricursive): mask from actual cache positions; keep labeled loss with logits_to_keep (0d2fa09828767ed003d41a8df08ad377d7b0ea17)
- Align the causal mask to the keys the layer holds and forward cache positions (ee974ea44fc20eb18134822bbd80e9ea1a0989ca)
Co-authored-by: Qubitium <Qubitium@users.noreply.huggingface.co>
Set Ripple preview tokenizer context to 1Mi tokens (#7)
- Set Ripple preview context metadata to 1Mi tokens (e3b388ad5c1d40f52db47297893158a1315283f8)
- Preserve the original 2Mi model positional capacity (0f78007cf2bf7eb5792580925b479bf904e2d056)
Apply the server changes from ripple-inference PR #18 (08fd9f4383). Default target context to 1048576, retain draft window 16384, and enable the stock SGLang positional-limit override required for the draft. Update the server README.
https://github.com/ricursive/ripple-inference/pull/18
Fix dropped parallel tool calls in speculative streams
Apply ripple-inference PR #17 (d68cca40e1). Reconcile streamed tool calls at finish so speculative deltas cannot drop trailing calls or argument suffixes. Validated with CPU parser and chat-format regression tests on stock SGLang 0.5.19.
https://github.com/ricursive/ripple-inference/pull/17
