RAP-Qwen3.8-27B-RL

The paper-to-poster model of RAP (Reconstruct A Poster). Starting from Qwen3.8-27B, the model is post-trained on RAP-25K paper–poster–markup triplets (paper) in two stages, both full-parameter (language model and vision–language aligner updated, vision encoder frozen; no LoRA, no quantization):

1. Supervised fine-tuning on paper → poster-HTML pairs derendered from human-crafted posters.
2. Reinforcement learning with GRPO, rewarding rendered posters with verifiable rule-based checks and a VLM judge.

Given the paper text, its figure/table/formula assets and a design instruction, the model writes the poster as a single self-contained HTML page. Weights are BF16 (Qwen3_5ForConditionalGeneration, 16K context).

Training

Supervised fine-tuning — training/validation loss, next-token accuracy and learning-rate schedule.

GRPO reinforcement learning — total reward, its rule-based and VLM-judge components, and completion length.

Usage

The demo Space and the code repository contain the full pipeline (PDF parsing, asset extraction, prompt construction and HTML rendering). The weights load with Transformers like any Qwen3.5-family vision-language model:

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "VincentHancoder/RAP-Qwen3.8-27B-RL"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda")

messages = [{"role": "user", "content": [
    {"type": "image", "image": "figure_1.png"},                  # extracted paper figures/tables, referenced by id in the prompt
    {"type": "text", "text": "<paper text, asset list and poster instruction>"},
]}]
inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,
                                       enable_thinking=False, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, do_sample=False, use_cache=True)
html = processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)

License

Apache-2.0, following the base model. The training data is released separately as RAP-25K.

Citation

If you find RAP useful or interesting for your research, please consider liking this model, starring the code repository and citing our paper.

@misc{han2026rapdatasetpapertopostergeneration,
      title={RAP: A Dataset for Paper-to-Poster Generation via Declarative Markup from Human-Crafted Exemplars},
      author={Haonan Han and Yukang Lin and Minliang Lin and Haoxian Tan and Jun-Yan He and Rui Yang and Zhi-Qi Cheng and Jie Hu and Guanglu Wan and Xiu Li},
      year={2026},
      eprint={2610.23230},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.23230},
}