Thinking-In-Boxes / README.md
PradhaanSBhat
Update README.md
429a03e
|
Raw History Blame Contribute Delete
4.38 kB
---
title: Thinking in Boxes
emoji: 📦
colorFrom: red
colorTo: gray
sdk: gradio
sdk_version: 6.30.0
app_file: demo.py
pinned: false
short_description: 3D Editing in Real Images Made Easy (NeurIPS 2026)
---
<div align="center">
# Thinking in Boxes: 3D Editing in Real Images Made Easy
### NeurIPS 2026
[**Pradhaan S Bhat**](https://pradhaansbhat.github.io/)<sup>1</sup><sup>∗</sup> · [**Naveen Chandra R**](https://www.linkedin.com/in/naveen-chandra-r-7230aa192)<sup>1</sup><sup>∗</sup> · [**Rishubh Parihar**](https://rishubhpar.github.io/)<sup>1</sup> · [**Vaibhav Vavilala**](https://www.linkedin.com/in/vaibhav-vavilala)<sup>2</sup> · [**R. Venkatesh Babu**](https://cds.iisc.ac.in/faculty/venky/)<sup>1</sup> · [**D. A. Forsyth**](http://luthuli.cs.uiuc.edu/~daf/)<sup>3</sup> · [**Anand Bhattad**](https://anandbhattad.github.io/)<sup>4</sup>
<sup>1</sup> Indian Institute of Science &nbsp;·&nbsp; <sup>2</sup> Apple &nbsp;·&nbsp; <sup>3</sup> UIUC &nbsp;·&nbsp; <sup>4</sup> Johns Hopkins University
<sup>∗</sup> Equal Contribution
[![arXiv](https://img.shields.io/badge/arXiv-2606.20556-b31b1b.svg)](https://arxiv.org/abs/2606.20556)
[![Project Page](https://img.shields.io/badge/Project-Page-1f6feb.svg)](https://thinking-in-boxes.github.io)
[![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-yellow)](https://huggingface.co/pradhaansbhat/Thinking-In-Boxes)
[![Hugging Face Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-orange)](https://huggingface.co/datasets/pradhaansbhat/Thinking-In-Boxes)
[![GitHub](https://img.shields.io/badge/GitHub-Repository-white.svg?logo=github&logoColor=white)](https://github.com/PradhaanSBhat/Thinking-In-Boxes)
</div>
---
## 📚 Abstract
Text and 2D-conditioning interfaces provide weak, ambiguous control over spatial
transformations in image editing — particularly under large object motions and
camera changes. Prior work has used 3D primitives such as boxes, but only as loose
conditioning signals indicating approximate object location rather than specifying
the transformation.
We instead use **3D boxes as structured specifications**: the user provides the input
and output boxes of the edit, casting editing as a well-posed geometry problem. This
"thinking in boxes" interface, where each box face is color-coded to convey 3D
orientation, gives precise control over translation, rotation, scaling, and viewpoint
changes in real images while preserving scene and object identity, and recovering
previously unseen object regions. To ground transformations in scene appearance, we
introduce a **depth-aligned planar floor** as a global reference frame, shaded with
depth-aware cues.
Conditioned on this structure, an image generator produces consistent results under
large transformations. Trained in two stages — on synthetic multi-object scenes and a
small set of real-world videos from Objectron — the system generalizes to complex,
in-the-wild real images. Our method operates directly on real photographs and
substantially **outperforms recent state-of-the-art methods** on large 3D edits.
---
# Demo Usage
Interactive demo for **[NeurIPS'26] Thinking in Boxes: 3D Editing in Real Images Made Easy**.
- Fit 3D boxes to objects, set a **source** pose, bookmark it on the left rail, edit to a **target** pose, then generate.
- Read the tooltips and hover over icons for details on how to use the 3D Editor.
- Runs on **ZeroGPU** (free, time-sliced RTX PRO 6000). Generation consumes your daily GPU quota.
- 3D auto-detection runs in **mock** mode on the Space (the full WildDet3D detector needs a
dedicated GPU/deps). Place and refine 3D boxes by hand. (See the [GitHub Repository](https://github.com/PradhaanSBhat/Thinking-In-Boxes) to use WildDet3D)
## Setup notes (for whoever deploys this Space)
1. **Hardware:** Settings → Hardware → **ZeroGPU**.
2. **Secret:** add `HF_TOKEN` from an account that has accepted the
[FLUX.1-Kontext-dev](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev) license —
the base model is gated and won't download otherwise.
3. **Files in the Space repo:** `assets/` `demo.py`, `box-editor.html`,
`pipelines/pipeline_flux_kontext.py` (the custom pipeline), and `requirements.txt`.
4. The default model + LoRA download on first boot (slow once, cached after).