---
title: "AnyTraverse Studio 🚜"
emoji: "🚜"
colorFrom: "blue"
colorTo: "indigo"
short_description: "Off-road traversability evaluation with HITL"
tags:
  - computer-vision
  - robotics
  - segmentation
  - vlm
---

# 🚜 AnyTraverse Studio — Live Evaluation & HITL Dashboard

A live Gradio dashboard for evaluating the **AnyTraverse** zero-shot off-road
traversability framework ([paper](https://arxiv.org/abs/2506.16826),
[PyPI](https://pypi.org/project/anytraverse/)).

Try it online:
[huggingface.co/spaces/sattwik21/anytraverse-studio](https://huggingface.co/spaces/sattwik21/anytraverse-studio)
(runs on ZeroGPU), or run it locally on your own machine (see below).

## Features

- **Live per-frame evaluation** of any uploaded off-road video:
  raw image + ROI box · traversability map · uncertainty map · ROI crop
- **Attention maps for all prompts** live in the UI (grid of up to 5 columns,
  aspect-ratio preserved) — never baked into the exported video
- **Human-in-the-Loop**: the run halts whenever the traversal state is not
  `ok`; provide a τ update (e.g. `mud: -0.7; gravel: 0.6`, or just `ok`) and
  press **Resume**
- **Editable live**: thresholds and ROI bounds update the running pipeline
  without restarting
- Live dual-metric chart + 0–1 gauge bars + telemetry table
- Composed analysis video exported as browser-playable H.264 (bundled
  `imageio-ffmpeg`, no system ffmpeg required)
- Hard **frame skip**: pass every k-th frame to the VLM; skipped frames are
  neither shown in the UI nor written to the video

## Run locally

### 1. Prerequisites

- **Linux / macOS / Windows** with **Python 3.12+**
- A **GPU with CUDA** is strongly recommended (CLIPSeg + CLIP inference).
  CPU works but is slow.
- [uv](https://docs.astral.sh/uv/) (fast, optional) or `pip` + a virtualenv

### 2. Clone

```bash
git clone https://huggingface.co/sattwik21/anytraverse-studio
cd anytraverse-studio
```

### 3. Install dependencies

With `uv` (recommended):

```bash
uv sync
```

Or with plain `pip`:

```bash
python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt
```

If `Torch` pulls the wrong build, install the CUDA build explicitly, e.g.:

```bash
pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install torchvision --index-url https://download.pytorch.org/whl/cu128
```

### 4. Run

```bash
python app.py
```

The dashboard opens at http://localhost:7860. Since `share=True` is enabled
outside Hugging Face Spaces, Gradio will also print a temporary public share
link at startup — you can ignore it for local use.

### 5. Usage

1. Upload an off-road video (`.mp4`, `.mov`, `.avi`).
2. Leave τ as `{}` for default preferences, or enter e.g.
   `{"road": 1.0, "grass": 0.0, "bush": -0.8}`.
3. Set the **Ref Scene Sim. Threshold**, **ROI Uncertainty Threshold** and
   **Frame Skip**.
4. Press **▶️ Go / Reset**.
5. When the run halts (`unknown_scene` / `unknown_object`), type an operator
   update in the box and press **✅ Apply & Resume**.

## Notes

- The first run downloads the VLM weights (CLIPSeg + CLIP, ~1 GB) and caches
  them locally.
- `data/weights/` and the generated `*.mp4` files are runtime artifacts; the
  dashboard writes `raw_opencv_temp.mp4` and `anytraverse_h264_output.mp4` in
  the working directory.

## Deployment

The interactive Space lives at
[huggingface.co/spaces/sattwik21/anytraverse-studio](https://huggingface.co/spaces/sattwik21/anytraverse-studio).
This repo is the model card / source for the same dashboard.