File size: 7,645 Bytes
9e30eba 27afd6e 9e30eba 5288fd3 9e30eba 5288fd3 9e30eba 5288fd3 9e30eba 5288fd3 9e30eba 5288fd3 9e30eba 27afd6e 9e30eba 2fbe935 9e30eba 2fbe935 9e30eba 5288fd3 9e30eba | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 | ---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
base_model_relation: finetune
language:
- en
tags:
- image-cropping
- aesthetic-image-cropping
- computer-vision
- vision-language-model
- arxiv:2610.17639
---
> **New official model repository:** [zlab-sjtu/FrameLite](https://huggingface.co/zlab-sjtu/FrameLite). Please use this repository for model downloads and updates. This location remains available for existing links.
<div align="center">
# FrameLite: Decision-Oriented Aesthetic Cropping<br>with a Compact Vision–Language Model
**Jianze Li**<sup>1</sup>, **Kai Liu**<sup>1</sup>, **Ziqing Zhang**<sup>1</sup>, **Weihang Zhang**<sup>2</sup>,<br>
**Yong Guo**<sup>1</sup>, **Linghe Kong**<sup>1</sup>, **Yulun Zhang**<sup>1,†</sup>
<sup>1</sup> Shanghai Jiao Tong University<br>
<sup>2</sup> Institute of Media Technology and Experience Design, Huawei<br>
<sup>†</sup> Corresponding author
[**Code**](https://github.com/JianzeLi-114/FrameLite) · [**Paper**](https://arxiv.org/abs/2610.17639) · [**Demo**](https://huggingface.co/spaces/zlab-sjtu/FrameLite) · [**Supplementary Material**](https://github.com/JianzeLi-114/FrameLite/releases/download/v1.0/FrameLite_supplementary.pdf)
[](https://huggingface.co/zlab-sjtu/FrameLite/blob/main/LICENSE)
[](https://huggingface.co/zlab-sjtu/FrameLite/tree/main)
</div>
## Abstract
> Aesthetic image cropping improves composition by balancing subjects, context, and image boundaries. Large vision–language models bring semantic knowledge to this task, but their scale limits lightweight photography assistance. We propose **FrameLite**, a compact cropping model built on Qwen3.5-0.8B. Our approach is motivated by the observation that a small model can generate better crops than its first answer suggests. We develop three-stage training to turn this potential into stronger direct predictions. Spatial and composition training establishes localization and framing preferences. Quality-Guided Online Supervision (QOS) then learns the highest-quality complete answers among the current policy's generated candidates. Greedy-Referenced Optimization (GRO) further refines the policy using positive and negative quality differences relative to its current greedy answer. Both online stages use annotation-derived feedback, while inference returns one coordinate answer without search or an external scorer. Experiments show that FrameLite outperforms the compared large-VLM croppers across public benchmarks with a substantially smaller backbone. Controlled studies confirm that selecting high-quality training answers improves the final model beyond online sampling alone.
## Method
<p align="center">
<img src="assets/method_overview.png" width="100%" alt="FrameLite's three-stage training overview">
</p>
1. **Stage I — Spatial and composition training.** Learn spatial localization, composition preferences, and complete reference-frame prediction.
2. **Stage II — Quality-Guided Online Supervision (QOS).** Generate greedy and sampled answers, evaluate complete-frame quality, and imitate the best answer.
3. **Stage III — Greedy-Referenced Optimization (GRO).** Refine fresh samples using positive and negative quality differences relative to the current greedy answer.
At inference, FrameLite directly predicts one normalized crop through greedy decoding. No candidate search, external scorer, retrieval database, or external API is needed.
## Model
| Model | Backbone | Weights |
|:---|:---|:---|
| FrameLite | Qwen3.5-0.8B | [224 MB](https://huggingface.co/zlab-sjtu/FrameLite/resolve/main/framelite.safetensors) |
The FrameLite checkpoint contains the learned language LoRA and vision merger.
Download [Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) separately and
load both with the `FrameLite` API below.
## Quick Start
### Install the inference code
Use **Python 3.10** and a CUDA GPU with BF16 support.
```bash
git clone https://github.com/JianzeLi-114/FrameLite.git
cd FrameLite
python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install torch==2.6.0 torchvision==0.21.0 \
--index-url https://download.pytorch.org/whl/cu124
python -m pip install -r requirements.txt
```
### Download the weights
Download the FrameLite weights and the Qwen backbone:
```python
from huggingface_hub import snapshot_download
snapshot_download(
repo_id="zlab-sjtu/FrameLite",
local_dir="weights/framelite",
allow_patterns=[
"framelite.safetensors",
"inference_config.json",
"LICENSE",
"NOTICE",
],
)
snapshot_download(
repo_id="Qwen/Qwen3.5-0.8B",
local_dir="weights/framelite/base",
)
```
The two downloads place the FrameLite checkpoint and Qwen backbone under `weights/framelite/`.
### Crop an image
```bash
python infer.py \
--image assets/examples/mountain_reflection/input.jpg \
--weights weights/framelite \
--output-dir outputs \
--device cuda:0
```
The CLI saves predictions to `outputs/predictions.jsonl` and valid crops to `outputs/crops/`. To process a directory, pass its path to `--image`; add `--recursive` to include subdirectories.
### Python API
```python
from framelite import FrameLite, crop_image
model = FrameLite.from_pretrained("weights/framelite", device="cuda:0")
image_path = "path/to/image.jpg"
result = model.predict(image_path)
if result["valid"]:
print(result["bbox_normalized"]) # [x1, y1, x2, y2] in [0, 1]
crop_image(image_path, result["bbox_normalized"]).save("crop.png")
else:
print(result["error"])
```
Coordinates refer to the original image, with the origin at its top-left corner. The model returns a single crop, for example `{"crop":[0.1,0.05,0.9,0.95]}`. This JSON illustrates the output format.
## Evaluation
The following results are reported in the paper. The **same final model** produces one crop per image on all three benchmarks.
| GAICv2 Acc .85 ↑ | GAICv2 Acc .90 ↑ | FLMS IoU ↑ | FLMS BDE ↓ | FCDB IoU ↑ | FCDB BDE ↓ |
|---:|---:|---:|---:|---:|---:|
| 0.9140 | 0.8340 | 0.8679 | 0.0297 | 0.7124 | 0.0704 |
Acc denotes GAICv2 Acc<sub>1/5</sub> at the specified IoU threshold. See the [paper](https://arxiv.org/abs/2610.17639) for the evaluation protocol and the [supplementary material](https://github.com/JianzeLi-114/FrameLite/releases/download/v1.0/FrameLite_supplementary.pdf) for additional analyses.
## Citation
```bibtex
@article{li2026framelite,
title={FrameLite: Decision-Oriented Aesthetic Cropping with a Compact Vision--Language Model},
author={Li, Jianze and Liu, Kai and Zhang, Ziqing and Zhang, Weihang and Guo, Yong and Kong, Linghe and Zhang, Yulun},
journal={arXiv preprint arXiv:2610.17639},
year={2026}
}
```
## License and Acknowledgments
FrameLite task weights and inference code are provided under the [Apache License 2.0](https://huggingface.co/zlab-sjtu/FrameLite/blob/main/LICENSE). The separately downloaded Qwen3.5-0.8B backbone is distributed by Qwen under the [Apache License 2.0](https://huggingface.co/Qwen/Qwen3.5-0.8B/blob/main/LICENSE); see [NOTICE](https://huggingface.co/zlab-sjtu/FrameLite/blob/main/NOTICE) and the [upstream model card](https://huggingface.co/Qwen/Qwen3.5-0.8B).
We thank the developers of [Qwen](https://github.com/QwenLM), [Transformers](https://github.com/huggingface/transformers), and [PEFT](https://github.com/huggingface/peft).
Correspondence: [Yulun Zhang](mailto:yulun100@gmail.com).
|