---
title: AutoInventory
emoji: 🏠
colorFrom: indigo
colorTo: blue
sdk: docker
pinned: false
---

# AutoInventory

AI-powered property inventory report generator. Upload a 1-minute walkthrough video and receive a professional PDF inventory report.

## Tech Stack

### Frontend
- **React 18** with Vite
- **Tailwind CSS 3** — dark theme with copper accent palette
- Client-side drag-and-drop upload with live elapsed timer and stage tracking

### Backend
- **FastAPI** — single POST endpoint (`/api/generate-report`)
- **Uvicorn** — ASGI server
- **FFmpeg** — frame extraction (1 frame every 5 seconds)
- **ReportLab** — PDF generation with professional layout (cover page, room tables, declaration)

### Machine Learning (Hugging Face Transformers)
- **Vision model**: `nlpconnect/vit-gpt2-image-captioning` — captions each extracted frame
- **Text model**: `google/flan-t5-small` — generates executive summary only
- Both run locally on CPU via PyTorch — no external APIs, no API keys, fully open-source

### Infrastructure
- **Docker** — multi-stage build (Node → React dist, Python → FastAPI)
- **Hugging Face Docker Spaces** — deployment target (port 7860)

## How It Works

### Pipeline

1. **Upload** — User drags an MP4/MOV video onto the React UI
2. **Frame Extraction** — FFmpeg extracts one frame every 5 seconds into `/tmp`
3. **Captioning** — Each frame is passed through `vit-gpt2-image-captioning` to generate a text description (e.g. "a living room with a couch and a coffee table")
4. **Room Classification** — Each caption is matched against keyword lists to determine the room (Living Room, Kitchen, Bedroom, etc.). No LLM involved — pure keyword matching.
5. **Item Generation** — For each room, standard inventory items (Door, Ceiling, Walls, Skirting, Window, Flooring, etc.) are created. Descriptions are sourced from the captions that best match each item type.
6. **Executive Summary** — `flan-t5-small` (via Hugging Face transformers pipeline) writes a one-paragraph summary of the inspection
7. **PDF Build** — ReportLab assembles a professional multi-page PDF with cover page, table of contents, room-by-room tables, and declaration page
8. **Download** — PDF is returned to the browser and automatically downloaded
9. **Cleanup** — Background task removes video, frames, and PDF from `/tmp`

### File Structure

```
├── backend/
│   ├── main.py              # FastAPI app, pipeline, PDF builder
│   └── requirements.txt     # Python dependencies
├── frontend/
│   ├── src/
│   │   └── App.jsx          # React upload UI with timer
│   ├── package.json
│   └── vite.config.js
├── Dockerfile               # Multi-stage build for HF Spaces
└── README.md
```

### Agents

No CrewAI or agent framework is used. The pipeline is a linear sequence of function calls:

| Step | Function | What it does |
|------|----------|-------------|
| Frame extraction | `extract_frames()` | Calls FFmpeg |
| Captioning | `get_frame_captions()` | Runs vision model on each frame |
| Room classification | `classify_room()` | Keyword matching |
| Item assembly | `run_inventory_pipeline()` | Builds structured data in Python |
| Summary | `LocalTextLLM.call()` | Runs text model once |
| PDF | `build_pdf_report()` | ReportLab layout |

Previous iterations attempted multi-agent orchestration with CrewAI and API-based models (Gemini, Groq), but these were replaced with the current fully-local, sequential approach for reliability and zero external dependencies.

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `LOCAL_TEXT_MODEL` | `google/flan-t5-small` | Hugging Face model for text generation |
| `LOCAL_VISION_MODEL` | `nlpconnect/vit-gpt2-image-captioning` | Hugging Face model for image captioning |

## Running Locally

```bash
# Backend
cd backend
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 7860

# Frontend (separate terminal)
cd frontend
npm install
npm run dev
```

## Deployment

Deploys to Hugging Face Docker Spaces. Only requirement is a valid HF token:

```bash
git push https://<USER>:<HF_TOKEN>@huggingface.co/spaces/<USER>/<SPACE> main
```