File size: 2,543 Bytes
75ab38b bc675c0 75ab38b abb3fe4 75ab38b abb3fe4 75ab38b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
license: cc-by-nc-4.0
base_model: Qwen/Qwen3-VL-4B-Instruct
pipeline_tag: robotics
language:
- en
tags:
- camera-motion
- active-perception
- flow-matching
- qwen3-vl
---
# LIME: Learning Intent-aware Camera Motion from Egocentric Video
[Paper](https://arxiv.org/abs/2607.02417) · [Project page](https://boysun045.github.io/LIME-Page/) · [Code](https://github.com/cvg/LIME)
**CoRL 2026.** Given one RGB image and a free-form intent ("Find my bike"), LIME predicts
where the camera should move next: a relative target camera pose, and a description of
what the new view should reveal.
These are the `lime_v1` weights: a fine-tuned Qwen3-VL-4B-Instruct backbone and a
flow-matching pose head of about 20M parameters. Running them needs the code in the
[GitHub repository](https://github.com/cvg/LIME).
## Usage
```bash
git clone https://github.com/cvg/LIME && cd LIME
bash scripts/setup_env.sh # creates the conda environment and downloads these weights
conda activate lime
python scripts/predict.py --image examples/living_room.jpg --intent "Find my bike." --seed 0
```
```python
from lime_camera import LIME
model = LIME.from_pretrained("weights/lime_v1")
prediction = model.predict("room.jpg", "Find my bike.", n_samples=5, seed=0)
prediction.observation_gain # what the new view should reveal
prediction.T_mean # 4x4 target pose, the mean of the 5 samples
```
Poses are the end camera in the start camera's frame, in metres, with OpenCV axes (x right,
y down, z forward).
## Files
| File | What it is |
|---|---|
| `config.json`, `model-*.safetensors`, tokenizer and processor files | the fine-tuned Qwen3-VL-4B backbone |
| `flow_head.safetensors`, `flow_head_config.json` | the flow-matching pose head |
| `pose_norm_stats.json` | per-axis mean and std the head's poses are standardized by |
## License
The weights are released under [CC BY-NC 4.0](LICENSE), for non-commercial use. They are a
modified version of [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct),
which is licensed under Apache-2.0; its license and our notice are in
[LICENSE-QWEN](LICENSE-QWEN).
## Citation
```bibtex
@inproceedings{sun2026lime,
title={LIME: Learning Intent-aware Camera Motion from Egocentric Video},
author={Sun, Boyang and Li, Jiajie and Yang, Yung-Hsu and Zhang, Chenyangguang
and Engelbracht, Tim and Hong, Sunghwan and Cadena, Cesar
and Pollefeys, Marc and Blum, Hermann},
booktitle={Conference on Robot Learning (CoRL)},
year={2026}
}
```
|