README.md

---
library_name: transformers
tags:
  - smolvlm
  - vlm
  - dpo
  - hallucination-reduction
  - accessibility
  - qlora
  - rlaif
license: apache-2.0
language:
  - en
pipeline_tag: image-text-to-text
base_model: HuggingFaceTB/SmolVLM2-500M-Video-Instruct
datasets:
  - HuggingFaceH4/rlaif-v_formatted
---

![Model Logo](thumbnail.png)

# Solari: Hallucination-Reduced Vision Language Model

Solari is a 500M parameter vision-language model fine-tuned for **reduced hallucination** on real-world images. Built on [SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct), Solari uses **QLoRA + Direct Preference Optimization (DPO)** on the [RLAIF-V](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) dataset to align the model toward more faithful visual descriptions.

## Model Details

### Model Description

Solari targets **hallucination reduction** in vision-language tasks, with a focus on improving reliability for **accessibility applications** (e.g., assisting visually impaired users). The model was trained using parameter-efficient fine-tuning (QLoRA) with DPO to learn preferences between accurate and hallucinated image descriptions, achieving improved hallucination benchmarks while preserving general VLM capabilities.

- **Developed by:** Cubex11
- **Model type:** Vision-Language Model (Image-Text-to-Text)
- **Language(s):** English
- **License:** Apache-2.0
- **Finetuned from:** [HuggingFaceTB/SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct)

### Model Sources

- **Base Model:** [SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct)
- **Training Dataset:** [RLAIF-V (Formatted)](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) — 72K AI-generated preference pairs for hallucination reduction

## Uses

### Direct Use

Solari can be used for image understanding tasks where **factual accuracy** is critical:

- Describing real-world scenes for visually impaired users
- Visual question answering with reduced hallucination
- Image captioning with improved object recognition reliability

### Out-of-Scope Use

- Tasks requiring strong mathematical reasoning or code understanding (degraded from base model)
- Non-English language tasks
- Medical or safety-critical applications without additional validation

## How to Get Started with the Model

```python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image
import requests

model_id = "Cubex11/Solari"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

# Load an image (replace with your own image path or URL)
image = Image.open("your_image.jpg").convert("RGB")

# Create prompt
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Describe this image in detail."}
        ]
    }
]

text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=text, images=[[image]], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
trimmed = output[0][len(inputs.input_ids[0]):]
print(processor.decode(trimmed, skip_special_tokens=True))
```

## Training Details

### Training Data

[RLAIF-V (Formatted)](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) — a large-scale multimodal preference dataset containing ~72K preference pairs. Each sample includes an image, a prompt, a **chosen** response (more accurate), and a **rejected** response (more hallucinated). Preferences are generated by open-source AI models following the RLAIF-V methodology.

### Training Procedure

**Method:** QLoRA + Direct Preference Optimization (DPO)

The base model was quantized to 4-bit (NF4) and fine-tuned using Low-Rank Adaptation (LoRA) with DPO to learn preferences between accurate and hallucinated responses.

#### Training Hyperparameters

| Parameter | Value |
|-----------|-------|
| **Training regime** | bf16 mixed precision |
| **Quantization** | 4-bit NF4 (double quantization) |
| **LoRA rank (r)** | 16 |
| **LoRA alpha** | 16 |
| **LoRA dropout** | 0.1 |
| **DoRA** | Enabled |
| **Target modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| **Trainable params** | ~1.9% of total |
| **Learning rate** | 5e-5 |
| **DPO beta** | 0.1 |
| **Batch size** | 8 (per device) |
| **Gradient accumulation** | 4 (effective batch = 32) |
| **Epochs** | 2 (best checkpoint at ~1 epoch / step 2500) |
| **Warmup ratio** | 0.1 |
| **Optimizer** | AdamW |

#### Speeds, Sizes, Times

- **Training time:** ~9 hours on NVIDIA L4 (24GB)
- **Best checkpoint:** Step 2500 (selected by lowest validation loss)
- **Model size:** ~1 GB (bf16 safetensors)

## Evaluation

### Testing Data, Factors & Metrics

Evaluated using [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) on 8 standard benchmarks covering hallucination, general VLM capability, and real-world understanding.

#### Metrics

- **POPE:** F1 score across random/popular/adversarial splits (object hallucination)
- **AMBER:** Attribute, Existence, Relation accuracy (multi-dimensional hallucination)
- **HallusionBench:** aAcc, fAcc, qAcc (hallucination detection)
- **A-OKVQA:** Accuracy on outside-knowledge VQA
- **MME:** Perception and Reasoning scores
- **MMStar:** Multi-modal reasoning accuracy
- **MMBench:** General multi-modal understanding
- **RealWorldQA:** Real-world image understanding accuracy

### Results

| Benchmark | Metric | Base Model | **Solari** | Change |
|-----------|--------|------------|------------|--------|
| **POPE** | Overall | 82.67 | **85.08** | **+2.41** |
| **POPE** | Recall | 76.73 | **85.33** | **+8.60** |
| **AMBER** | Avg ACC | 79.38 | **79.77** | **+0.39** |
| **AMBER** | Relation | 72.36 | **75.42** | **+3.06** |
| **HallusionBench** | Overall | 27.58 | **28.14** | **+0.56** |
| **A-OKVQA** | Overall | 68.12 | **69.00** | **+0.88** |
| **MMStar** | Overall | 38.33 | **39.60** | **+1.27** |
| **MMBench** | Test | 53.14 | **53.42** | **+0.28** |
| **RealWorldQA** | Overall | 49.80 | **50.59** | **+0.78** |
| **MME** | Perception | **1216.19** | 1118.51 | -97.68 |
| **MME** | Reasoning | **237.50** | 211.79 | -25.71 |

#### Summary

Solari improves on **7 out of 8 benchmarks** compared to the base model:

- **POPE recall +8.60%** — dramatically better at recognizing objects actually present in images
- **All hallucination benchmarks improved** — POPE, AMBER, and HallusionBench
- **General capabilities preserved or improved** — A-OKVQA, MMStar, MMBench, RealWorldQA all show gains
- **Trade-off on MME** — perception score dropped ~98 points, primarily on counting (-26.7), position (-26.7), and code reasoning (-27.5) subtasks due to the model becoming more conservative

## Bias, Risks, and Limitations

- **Counting and spatial reasoning degraded:** The DPO alignment made the model more conservative, reducing performance on fine-grained counting and positional reasoning tasks (reflected in MME scores).
- **Small model capacity:** At 500M parameters, the model has inherent limitations on complex reasoning tasks.
- **English only:** The model was trained and evaluated only on English-language tasks.
- **Training data bias:** RLAIF-V preferences are AI-generated, which may introduce systematic biases.

### Recommendations

- Best suited for binary object recognition tasks ("Is there a X?") and general scene description
- For tasks requiring precise counting or spatial reasoning, consider using the base model or a larger VLM
- Always validate outputs in safety-critical applications

## Environmental Impact

- **Hardware Type:** NVIDIA L4 (24GB)
- **Hours used:** ~9 hours
- **Cloud Provider:** Lightning AI
- **Compute Region:** US

## Technical Specifications

### Model Architecture and Objective

- **Architecture:** SmolVLM2 (ViT vision encoder + LLM decoder with multi-modal projector)
- **Parameters:** ~500M total
- **Objective:** Direct Preference Optimization (DPO) — learns to prefer accurate descriptions over hallucinated ones

### Compute Infrastructure

#### Hardware

NVIDIA L4 GPU (24GB VRAM) on Lightning AI

#### Software

- Transformers
- TRL (DPO Trainer)
- PEFT (QLoRA)
- BitsAndBytes (4-bit quantization)

## Citation

**BibTeX:**

```bibtex
@misc{solari2026,
    title={Solari: Hallucination-Reduced Vision Language Model via QLoRA DPO on RLAIF-V},
    author={Cubex11},
    year={2026},
    url={https://huggingface.co/Cubex11/Solari}
}
```

## Acknowledgments

- [HuggingFace](https://huggingface.co/) for SmolVLM2 and the RLAIF-V formatted dataset
- [OpenBMB](https://github.com/OpenBMB) for the RLAIF-V and RLHF-V research
- [Lightning AI](https://lightning.ai/) for compute resources
- [OpenCompass](https://github.com/open-compass/VLMEvalKit) for the VLMEvalKit evaluation toolkit