Ripple-65B-Preview-NVFP4

Model Overview

• Model Architecture: RicursiveForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Weight quantization: FP4
  • Activation quantization: FP4
• Release Date: 2026-11-06
• Version: 1.0
• Model Developers: RedHatAI

This model is a quantized version of ricursiveAI/Ripple-65B-Preview. It was evaluated on GPQA Diamond to assess its quality in comparison to the unquantized model.

Model Optimizations

This model was obtained by quantizing the weights and activations of ricursiveAI/Ripple-65B-Preview to FP4 data type, ready for inference with vLLM.

NVFP4 stores weights and activations in 4-bit floating point with per-16-element block scaling and FP8 scale factors, using GPTQ with activation-importance weighting to preserve accuracy.

This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 75%.

Only the weights and activations of the linear operators within transformer blocks are quantized using LLM Compressor.

Deployment

vLLM Serving

vllm serve RedHatAI/Ripple-65B-Preview-NVFP4 \
  --tensor-parallel-size 1 \
  --reasoning-parser ripple \
  --enable-auto-tool-choice \
  --tool-call-parser ripple

Creation

This model was created by applying LLM Compressor with calibration samples from the perfectblend dataset, as presented in the code snippet below. The model is exported in compressed-tensors format and can be loaded directly by vLLM.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

from llmcompressor import oneshot
from llmcompressor.modifiers.gptq import GPTQModifier

MODEL_ID = "ricursiveAI/Ripple-65B-Preview"
SAVE_DIR = "./Ripple-65B-Preview-NVFP4"

NUM_CALIBRATION_SAMPLES = 512
MAX_SEQUENCE_LENGTH = 2048

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map=None,
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)

recipe = GPTQModifier(
    targets="Linear",
    scheme="NVFP4",
    ignore=["lm_head"],
    weight_observer="nvfp4_expanded_imatrix",
    actorder="static",
    dampening_frac=0.01,
    block_size=128,
)

oneshot(
    model=model,
    tokenizer=tokenizer,
    dataset="perfectblend",
    splits="train[:512]",
    recipe=recipe,
    max_seq_length=MAX_SEQUENCE_LENGTH,
    num_calibration_samples=NUM_CALIBRATION_SAMPLES,
    trust_remote_code_model=True,
)

model.save_pretrained(SAVE_DIR, save_compressed=True)
tokenizer.save_pretrained(SAVE_DIR)

Evaluation

This model was evaluated on GPQA Diamond using Inspect with the inspect_evals/gpqa_diamond task, served with vLLM (OpenAI-compatible API). Evaluations were run on 1x NVIDIA B200 per model.

Accuracy

Category	Benchmark	ricursiveAI/Ripple-65B-Preview	RedHatAI/Ripple-65B-Preview-NVFP4	Recovery
Reasoning	GPQA Diamond (0-shot, CoT, avg@3)	81.82	79.12	96.70%

Standard error: ±2.48 (BF16), ±2.41 (NVFP4). Scores use the inspect_evals/gpqa_diamond protocol and are not directly comparable to the GPQA Diamond result reported in the base model card.

Reproduction

The results were obtained using the following commands:

GPQA Diamond (Inspect, 0-shot CoT, 3 epochs)

Start the vLLM server:

vllm serve RedHatAI/Ripple-65B-Preview-NVFP4 \
  --tensor-parallel-size 1 \
  --max-model-len 65536 \
  --reasoning-parser ripple \
  --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 64}' \
  --served-model-name RedHatAI/Ripple-65B-Preview-NVFP4

Run the evaluation:

export VLLM_BASE_URL=http://localhost:8000/v1
export VLLM_API_KEY=local

inspect eval inspect_evals/gpqa_diamond \
  --model vllm/RedHatAI/Ripple-65B-Preview-NVFP4 \
  --epochs 3 --epochs-reducer mean \
  --temperature 1.0 --top-p 0.95 --top-k 64 \
  --max-tokens 32768 \
  --max-connections 16 \
  -M client_timeout=7200 \
  --log-dir logs

Inspect's vLLM provider does not send top_k, so it is set server-side with --override-generation-config. The three epochs are independent unseeded draws, averaged per question (avg@3). The BF16 baseline was evaluated with the same protocol using ricursiveAI/Ripple-65B-Preview.