Follow-up: the exact-capacity static-mask case is now covered. When the mask width equals `past_length + input_length`, its final slot distinguishes a full prompt with unused cache capacity from a new-only token block. A mask with unused capacity at the end keeps the full-prompt path.
[GPT-QModel #3574](https://github.com/ModelCloud/GPTQModel/pull/3574) contains regression tests for both exact-capacity and longer padded masks, plus existing new-only block coverage. It builds on merged [GPT-QModel #3571](https://github.com/ModelCloud/GPTQModel/pull/3571). The Ripple suite passed **35 tests** against this upstream source (`tests/test_ricursive_support.py`).
Handle capacity-padded masks for cached full prompts
Follow-up to the initial PR: merged [GPT-QModel #3571](https://github.com/ModelCloud/GPTQModel/pull/3571) exposed another `cache_position=None` case. An attention mask can be padded to a static cache's capacity, so `mask_length > input_length` alone does not prove that `input_ids` contains only new tokens. The updated code treats only `mask_length == past_length + input_length` as evidence for a new-only block. A regression test and the exact before/after reproduction are in [GPT-QModel #3574](https://github.com/ModelCloud/GPTQModel/pull/3574).
Validation against this updated upstream source: **34 passed** in the tiny Ripple suite (`tests/test_ricursive_support.py`); the extracted upstream generation method also passed all four prompt/block cases.
Fix warm-cache new token blocks in generation preparation
## Problem
`RicursiveForCausalLM.prepare_inputs_for_generation` assumes every multi-token
`input_ids` input includes the cached prefix when `cache_position` is absent.
If the cache already holds 10 tokens and a caller supplies only two new tokens,
the calculated new length is -8 and `torch.arange` raises
`RuntimeError: upper bound and lower bound inconsistent with step sign`.
## Fix
Distinguish new-only token blocks from full prompts containing cached tokens.
An input block no longer than the cache is new; a longer input with an attention
mask spanning the cache is also new. Keep existing full-prompt slicing when
neither clue is present.
## Validation
- The three regression cases pass against this patched upstream function:
short new block, longer new block with a spanning mask, and full prompt.
- The tiny-model Ripple suite passed against this source:
`GPTQMODEL_RIPPLE_REF=<patched Ripple checkout> python -m pytest -q -o log_cli=false tests/test_ricursive_support.py`
(**33 …
2 days ago
Fix explicit position_ids generation on Transformers 4 (#8)
- Fix explicit position_ids generation on Transformers 4 (9f61852322e86b4f09b10003445642c03dc5975f)
- Keep the generation regression harness outside the model repository (48f6e3cca18a1e4937f830a2dbbc60906e5ca1ca)
- Fix generate() and attention masking under Transformers 5.x (b9e4394ccce54f7206da4358cd7987cb57efd893)
- Add DFlash2 inference fixes (warm-cache verify mask, logits_to_keep, chunked MLP) (c4bb0c1fad49ddce9de8cf9849e2d7ffa3b660f0)
- Rename MLP chunk env var to RIPPLE_MLP_CHUNK (c178f879bfc6a52f93f9adb467478193ec3689b7)
- Fix StaticCache causal mask and labels+logits_to_keep loss (8662efad14a490205c62dc795ecfd70c46807aff)
- Fix StaticCache causal mask and labels+logits_to_keep loss (2c3501bf7bf0d4f4eed89a5b417524f138d83091)
- fix(ricursive): mask from actual cache positions; keep labeled loss with logits_to_keep (0d2fa09828767ed003d41a8df08ad377d7b0ea17)
- Align the causal mask to the keys the layer holds and forward cache positions (ee974ea44fc20eb18134822bbd80e9ea1a0989ca)
Co-authored-by: Qubitium <Qubitium@users.noreply.huggingface.co>
