ricursiveAI

/

Ripple-65B-Preview

Like 397

Follow

Ricursive Intelligence 263

Problem

RicursiveForCausalLM.prepare_inputs_for_generation assumes every multi-token
input_ids input includes the cached prefix when cache_position is absent.
If the cache already holds 10 tokens and a caller supplies only two new tokens,
the calculated new length is -8 and torch.arange raises
RuntimeError: upper bound and lower bound inconsistent with step sign.

Fix

Distinguish new-only token blocks from full prompts containing cached tokens.
An input block no longer than the cache is new; a longer input with an attention
mask spanning the cache is also new. Keep existing full-prompt slicing when
neither clue is present.

Validation

• The three regression cases pass against this patched upstream function:
  short new block, longer new block with a spanning mask, and full prompt.
• The tiny-model Ripple suite passed against this source:
  GPTQMODEL_RIPPLE_REF=<patched Ripple checkout> python -m pytest -q -o log_cli=false tests/test_ricursive_support.py
  (33 passed).
• python -m py_compile modeling_ricursive.py passed.
• Unit test coverage and the corresponding GPT-QModel compatibility fix are in
  merged ModelCloud/GPTQModel#3571,
  especially test_generate_patch_distinguishes_new_blocks_from_full_prompts.

The current BF16 checkpoint marks its Transformers 5 support as native, so
GPT-QModel intentionally skips that compatibility patch; this repository needs
the fix directly.

Follow-up to the initial PR: merged GPT-QModel #3571 exposed another cache_position=None case. An attention mask can be padded to a static cache's capacity, so mask_length > input_length alone does not prove that input_ids contains only new tokens. The updated code treats only mask_length == past_length + input_length as evidence for a new-only block. A regression test and the exact before/after reproduction are in GPT-QModel #3574.

Validation against this updated upstream source: 34 passed in the tiny Ripple suite (tests/test_ricursive_support.py); the extracted upstream generation method also passed all four prompt/block cases.

Follow-up: the exact-capacity static-mask case is now covered. When the mask width equals past_length + input_length, its final slot distinguishes a full prompt with unused cache capacity from a new-only token block. A mask with unused capacity at the end keeps the full-prompt path.

GPT-QModel #3574 contains regression tests for both exact-capacity and longer padded masks, plus existing new-only block coverage. It builds on merged GPT-QModel #3571. The Ripple suite passed 35 tests against this upstream source (tests/test_ricursive_support.py).

Mostly looks good, and thank you for the fix, just couple of changes:

1. A capacity-sized mask can duplicate the cached prefix. With four cached tokens, a six-token full prompt, and an all-visible mask of width ten, the PR processes all six tokens again. The cache grows to ten instead of six, and logits change. This passed on the base code. The final mask bit cannot reliably distinguish the input layouts. Affected code.
2. The new short-block path can produce incorrect positions. Ten cached tokens plus two new tokens, with a sixteen-slot padded mask, produces positions [0, 0] instead of [10, 11]. The old exception disappears, but logits are wrong. Derive positions using cache_position, rather than taking the mask’s final entries. Affected code.