README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
base_model_relation: finetune
language:
  - en
tags:
  - image-cropping
  - aesthetic-image-cropping
  - computer-vision
  - vision-language-model
  - arxiv:2610.17639
---

> **New official model repository:** [zlab-sjtu/FrameLite](https://huggingface.co/zlab-sjtu/FrameLite). Please use this repository for model downloads and updates. This location remains available for existing links.

<div align="center">

# FrameLite: Decision-Oriented Aesthetic Cropping<br>with a Compact Vision–Language Model

**Jianze Li**<sup>1</sup>, **Kai Liu**<sup>1</sup>, **Ziqing Zhang**<sup>1</sup>, **Weihang Zhang**<sup>2</sup>,<br>
**Yong Guo**<sup>1</sup>, **Linghe Kong**<sup>1</sup>, **Yulun Zhang**<sup>1,†</sup>

<sup>1</sup> Shanghai Jiao Tong University<br>
<sup>2</sup> Institute of Media Technology and Experience Design, Huawei<br>
<sup>†</sup> Corresponding author

[**Code**](https://github.com/JianzeLi-114/FrameLite) · [**Paper**](https://arxiv.org/abs/2610.17639) · [**Demo**](https://huggingface.co/spaces/zlab-sjtu/FrameLite) · [**Supplementary Material**](https://github.com/JianzeLi-114/FrameLite/releases/download/v1.0/FrameLite_supplementary.pdf)

[![License](https://img.shields.io/badge/License-Apache%202.0-4f6d7a?style=flat-square)](https://huggingface.co/zlab-sjtu/FrameLite/blob/main/LICENSE)
[![Weights](https://img.shields.io/badge/Weights-224_MB-697d6c?style=flat-square)](https://huggingface.co/zlab-sjtu/FrameLite/tree/main)

</div>

## Abstract

> Aesthetic image cropping improves composition by balancing subjects, context, and image boundaries. Large vision–language models bring semantic knowledge to this task, but their scale limits lightweight photography assistance. We propose **FrameLite**, a compact cropping model built on Qwen3.5-0.8B. Our approach is motivated by the observation that a small model can generate better crops than its first answer suggests. We develop three-stage training to turn this potential into stronger direct predictions. Spatial and composition training establishes localization and framing preferences. Quality-Guided Online Supervision (QOS) then learns the highest-quality complete answers among the current policy's generated candidates. Greedy-Referenced Optimization (GRO) further refines the policy using positive and negative quality differences relative to its current greedy answer. Both online stages use annotation-derived feedback, while inference returns one coordinate answer without search or an external scorer. Experiments show that FrameLite outperforms the compared large-VLM croppers across public benchmarks with a substantially smaller backbone. Controlled studies confirm that selecting high-quality training answers improves the final model beyond online sampling alone.

## Method

<p align="center">
  <img src="assets/method_overview.png" width="100%" alt="FrameLite's three-stage training overview">
</p>

1. **Stage I — Spatial and composition training.** Learn spatial localization, composition preferences, and complete reference-frame prediction.
2. **Stage II — Quality-Guided Online Supervision (QOS).** Generate greedy and sampled answers, evaluate complete-frame quality, and imitate the best answer.
3. **Stage III — Greedy-Referenced Optimization (GRO).** Refine fresh samples using positive and negative quality differences relative to the current greedy answer.

At inference, FrameLite directly predicts one normalized crop through greedy decoding. No candidate search, external scorer, retrieval database, or external API is needed.

## Model

| Model | Backbone | Weights |
|:---|:---|:---|
| FrameLite | Qwen3.5-0.8B | [224 MB](https://huggingface.co/zlab-sjtu/FrameLite/resolve/main/framelite.safetensors) |

The FrameLite checkpoint contains the learned language LoRA and vision merger.
Download [Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) separately and
load both with the `FrameLite` API below.

## Quick Start

### Install the inference code

Use **Python 3.10** and a CUDA GPU with BF16 support.

```bash
git clone https://github.com/JianzeLi-114/FrameLite.git
cd FrameLite

python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install torch==2.6.0 torchvision==0.21.0 \
  --index-url https://download.pytorch.org/whl/cu124
python -m pip install -r requirements.txt
```

### Download the weights

Download the FrameLite weights and the Qwen backbone:

```python
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="zlab-sjtu/FrameLite",
    local_dir="weights/framelite",
    allow_patterns=[
        "framelite.safetensors",
        "inference_config.json",
        "LICENSE",
        "NOTICE",
    ],
)

snapshot_download(
    repo_id="Qwen/Qwen3.5-0.8B",
    local_dir="weights/framelite/base",
)
```

The two downloads place the FrameLite checkpoint and Qwen backbone under `weights/framelite/`.

### Crop an image

```bash
python infer.py \
  --image assets/examples/mountain_reflection/input.jpg \
  --weights weights/framelite \
  --output-dir outputs \
  --device cuda:0
```

The CLI saves predictions to `outputs/predictions.jsonl` and valid crops to `outputs/crops/`. To process a directory, pass its path to `--image`; add `--recursive` to include subdirectories.

### Python API

```python
from framelite import FrameLite, crop_image

model = FrameLite.from_pretrained("weights/framelite", device="cuda:0")
image_path = "path/to/image.jpg"
result = model.predict(image_path)

if result["valid"]:
    print(result["bbox_normalized"])  # [x1, y1, x2, y2] in [0, 1]
    crop_image(image_path, result["bbox_normalized"]).save("crop.png")
else:
    print(result["error"])
```

Coordinates refer to the original image, with the origin at its top-left corner. The model returns a single crop, for example `{"crop":[0.1,0.05,0.9,0.95]}`. This JSON illustrates the output format.

## Evaluation

The following results are reported in the paper. The **same final model** produces one crop per image on all three benchmarks.

| GAICv2 Acc .85 ↑ | GAICv2 Acc .90 ↑ | FLMS IoU ↑ | FLMS BDE ↓ | FCDB IoU ↑ | FCDB BDE ↓ |
|---:|---:|---:|---:|---:|---:|
| 0.9140 | 0.8340 | 0.8679 | 0.0297 | 0.7124 | 0.0704 |

Acc denotes GAICv2 Acc<sub>1/5</sub> at the specified IoU threshold. See the [paper](https://arxiv.org/abs/2610.17639) for the evaluation protocol and the [supplementary material](https://github.com/JianzeLi-114/FrameLite/releases/download/v1.0/FrameLite_supplementary.pdf) for additional analyses.

## Citation

```bibtex
@article{li2026framelite,
  title={FrameLite: Decision-Oriented Aesthetic Cropping with a Compact Vision--Language Model},
  author={Li, Jianze and Liu, Kai and Zhang, Ziqing and Zhang, Weihang and Guo, Yong and Kong, Linghe and Zhang, Yulun},
  journal={arXiv preprint arXiv:2610.17639},
  year={2026}
}
```

## License and Acknowledgments

FrameLite task weights and inference code are provided under the [Apache License 2.0](https://huggingface.co/zlab-sjtu/FrameLite/blob/main/LICENSE). The separately downloaded Qwen3.5-0.8B backbone is distributed by Qwen under the [Apache License 2.0](https://huggingface.co/Qwen/Qwen3.5-0.8B/blob/main/LICENSE); see [NOTICE](https://huggingface.co/zlab-sjtu/FrameLite/blob/main/NOTICE) and the [upstream model card](https://huggingface.co/Qwen/Qwen3.5-0.8B).

We thank the developers of [Qwen](https://github.com/QwenLM), [Transformers](https://github.com/huggingface/transformers), and [PEFT](https://github.com/huggingface/peft).

Correspondence: [Yulun Zhang](mailto:yulun100@gmail.com).