KU-AGI/FiRe-2500Step

🔥 FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation

🧩 Checkpoint: FiRe-2500Step (FiRe-GRPO, 2500 RL steps)

Official Checkpoint of FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation

This repository hosts the FiRe-GRPO checkpoint after 2500 reinforcement-learning steps, fine-tuned from FiRe-SFT using step-level GRPO with additional training beyond the paper's reported FiRe-300Step checkpoint. For the SFT base and the checkpoint used in our NeurIPS 2026 paper, see FiRe-SFT and FiRe-300Step.

FiRe Inference

FiRe-GRPO

📌 Paper Overview

Unified MLLMs can both understand and generate images, but their reasoning ability is rarely used to improve generation itself. Existing reasoning-based text-to-image methods rely on prompt augmentation or holistic image–text judgments, so they often miss fine-grained details such as attributes, counts, and spatial relations.

FiRe breaks the prompt into verifiable visual requirements, checks each one against the generated image, and corrects only the parts that are wrong. Concretely, it summarizes the prompt into verifiable visual details, decomposes them into atomic semantic tuples (objects, attributes, counts, spatial relations), verifies each tuple against the image with tuple-level VQA, and turns any unsatisfied tuple into an explicit correction instruction — which is then applied through localized image editing that fixes only the mismatched regions while preserving everything already correct.

We also propose FiRe-GRPO, a step-level reinforcement learning method that gives each reasoning step its own reward. Standard GRPO assigns a single, trajectory-level reward to the whole reasoning-and-generation rollout, so every step — whether it was the tuple decomposition, the VQA verification, or the final edit — gets the same credit regardless of which step actually caused the outcome, making it hard to tell which reasoning step to reinforce and which to discourage. FiRe-GRPO instead assigns step-specific rewards and estimates the advantage of each step separately within the same trajectory, then optimizes the policy with GRPO — enabling precise, step-level credit assignment and yielding better fine-grained image-prompt alignment.

📊 Results

Quantitative Results

Qualitative Results

🤗 Acknowledgment

We sincerely thank the authors of Janus-Series and Janus-Pro-R1 for making their models and code publicly available.

📝 Citation

@article{kim2026fire,
  title={FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation},
  author={Kim, Yongjin and Oh, Yoonjin and Kim, Yerin and Kim, Hyomin and Yun, Jeeyoung and Heo, Yujung and Kim, Minjun and Kim, Sungwoong},
  journal={arXiv preprint arXiv:2604.13491},
  year={2026}
}