| --- |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| base_model: Qwen/Qwen2.5-VL-3B-Instruct |
| tags: |
| - remote-sensing |
| - earth-observation |
| - geospatial |
| - qwen2_5_vl |
| - grpo |
| --- |
| |
| # GeoVLM_R1_Image_caption |
| |
| > **Disclaimer** |
| > |
| > THIS MODEL IS PROVIDED “AS IS,” WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY, OR NON-INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR THEIR AFFILIATES BE LIABLE FOR ANY DAMAGES ARISING FROM ACCESS TO OR USE OF THIS MODEL. |
| |
| This model is released **without a license**. It is not open-source software. |
| |
| ## Model description |
| |
| `GeoVLM_R1_Image_caption` is the **Image captioning** checkpoint of **GeoVLM-R1**, from the paper |
| [GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoning](https://arxiv.org/abs/2509.25026). |
|
|
| - **Architecture:** `Qwen2_5_VLForConditionalGeneration` (3B parameters, bfloat16) |
| - **Training:** Qwen2.5-VL-3B-Instruct, then supervised fine-tuning on EarthDial-Instruct, then |
| GRPO reinforcement fine-tuning with task-aware rewards on `GRPO_RSICD-Captions_instructions_train.jsonl` |
| - **Task:** Image captioning |
| - **Benchmarks:** NWPU-Captions, RSICD, RSITMD |
|
|
| GeoVLM-R1 has one checkpoint per task. All 9 are in |
| [ibm-research/GeoVLM-R1](https://huggingface.co/ibm-research/GeoVLM-R1). |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor |
| from qwen_vl_utils import process_vision_info |
| |
| model_path = "path/to/GeoVLM_R1_Image_caption" |
| model = Qwen2_5_VLForConditionalGeneration.from_pretrained( |
| model_path, torch_dtype=torch.bfloat16, device_map="auto").eval() |
| processor = AutoProcessor.from_pretrained(model_path) |
| |
| messages = [{ |
| "role": "user", |
| "content": [ |
| {"type": "image", "image": "example.png", "resized_height": 448, "resized_width": 448}, |
| {"type": "text", "text": "Describe this image."}, |
| ], |
| }] |
| text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| images, videos = process_vision_info(messages) |
| inputs = processor(text=[text], images=images, videos=videos, |
| padding=True, return_tensors="pt").to(model.device) |
| out = model.generate(**inputs, max_new_tokens=128) |
| print(processor.batch_decode([o[len(i):] for i, o in zip(inputs.input_ids, out)], |
| skip_special_tokens=True)[0]) |
| ``` |
|
|
| To run the benchmark, use `GeoVLM_inference/generate_inference_Image_Captions.py` from the |
| [GeoVLM-R1 Toolkit](https://github.com/IBM/GeoVLM-R1-Toolkit) with `--model_path path/to/GeoVLM_R1_Image_caption`. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{fiaz2025geovlmr1, |
| title = {GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoning}, |
| author = {Fiaz, Mustansar and Debary, Hiyam and Fraccaro, Paolo and Paudel, Danda and |
| Van Gool, Luc and Khan, Fahad and Khan, Salman}, |
| journal = {arXiv preprint arXiv:2509.25026}, |
| year = {2025} |
| } |
| ``` |
|
|
| ## Contact |
|
|
| Mustansar Fiaz <mustansar.fiaz@ibm.com> |
|
|
