---
license: cc-by-nc-4.0
base_model: Qwen/Qwen3-VL-4B-Instruct
pipeline_tag: robotics
language:
  - en
tags:
  - camera-motion
  - active-perception
  - flow-matching
  - qwen3-vl
---

# LIME: Learning Intent-aware Camera Motion from Egocentric Video

[Paper](https://arxiv.org/abs/2607.02417) · [Project page](https://boysun045.github.io/LIME-Page/) · [Code](https://github.com/cvg/LIME)

**CoRL 2026.** Given one RGB image and a free-form intent ("Find my bike"), LIME predicts
where the camera should move next: a relative target camera pose, and a description of
what the new view should reveal.

These are the `lime_v1` weights: a fine-tuned Qwen3-VL-4B-Instruct backbone and a
flow-matching pose head of about 20M parameters. Running them needs the code in the
[GitHub repository](https://github.com/cvg/LIME).

## Usage

```bash
git clone https://github.com/cvg/LIME && cd LIME
bash scripts/setup_env.sh        # creates the conda environment and downloads these weights
conda activate lime
python scripts/predict.py --image examples/living_room.jpg --intent "Find my bike." --seed 0
```

```python
from lime_camera import LIME

model = LIME.from_pretrained("weights/lime_v1")
prediction = model.predict("room.jpg", "Find my bike.", n_samples=5, seed=0)
prediction.observation_gain     # what the new view should reveal
prediction.T_mean               # 4x4 target pose, the mean of the 5 samples
```

Poses are the end camera in the start camera's frame, in metres, with OpenCV axes (x right,
y down, z forward).

## Files

| File | What it is |
|---|---|
| `config.json`, `model-*.safetensors`, tokenizer and processor files | the fine-tuned Qwen3-VL-4B backbone |
| `flow_head.safetensors`, `flow_head_config.json` | the flow-matching pose head |
| `pose_norm_stats.json` | per-axis mean and std the head's poses are standardized by |

## License

The weights are released under [CC BY-NC 4.0](LICENSE), for non-commercial use. They are a
modified version of [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct),
which is licensed under Apache-2.0; its license and our notice are in
[LICENSE-QWEN](LICENSE-QWEN).

## Citation

```bibtex
@inproceedings{sun2026lime,
  title={LIME: Learning Intent-aware Camera Motion from Egocentric Video},
  author={Sun, Boyang and Li, Jiajie and Yang, Yung-Hsu and Zhang, Chenyangguang
          and Engelbracht, Tim and Hong, Sunghwan and Cadena, Cesar
          and Pollefeys, Marc and Blum, Hermann},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2026}
}
```