README.md

metadata

title: AutoInventory
emoji: 🏠
colorFrom: indigo
colorTo: blue
sdk: docker
pinned: false

AutoInventory

AI-powered property inventory report generator. Upload a 1-minute walkthrough video and receive a professional PDF inventory report.

Tech Stack

Frontend

• React 18 with Vite
• Tailwind CSS 3 — dark theme with copper accent palette
• Client-side drag-and-drop upload with live elapsed timer and stage tracking

Backend

• FastAPI — single POST endpoint (/api/generate-report)
• Uvicorn — ASGI server
• FFmpeg — frame extraction (1 frame every 5 seconds)
• ReportLab — PDF generation with professional layout (cover page, room tables, declaration)

Machine Learning (Hugging Face Transformers)

• Vision model: nlpconnect/vit-gpt2-image-captioning — captions each extracted frame
• Text model: google/flan-t5-small — generates executive summary only
• Both run locally on CPU via PyTorch — no external APIs, no API keys, fully open-source

Infrastructure

• Docker — multi-stage build (Node → React dist, Python → FastAPI)
• Hugging Face Docker Spaces — deployment target (port 7860)

How It Works

Pipeline

1. Upload — User drags an MP4/MOV video onto the React UI
2. Frame Extraction — FFmpeg extracts one frame every 5 seconds into /tmp
3. Captioning — Each frame is passed through vit-gpt2-image-captioning to generate a text description (e.g. "a living room with a couch and a coffee table")
4. Room Classification — Each caption is matched against keyword lists to determine the room (Living Room, Kitchen, Bedroom, etc.). No LLM involved — pure keyword matching.
5. Item Generation — For each room, standard inventory items (Door, Ceiling, Walls, Skirting, Window, Flooring, etc.) are created. Descriptions are sourced from the captions that best match each item type.
6. Executive Summary — flan-t5-small (via Hugging Face transformers pipeline) writes a one-paragraph summary of the inspection
7. PDF Build — ReportLab assembles a professional multi-page PDF with cover page, table of contents, room-by-room tables, and declaration page
8. Download — PDF is returned to the browser and automatically downloaded
9. Cleanup — Background task removes video, frames, and PDF from /tmp

File Structure

├── backend/
│   ├── main.py              # FastAPI app, pipeline, PDF builder
│   └── requirements.txt     # Python dependencies
├── frontend/
│   ├── src/
│   │   └── App.jsx          # React upload UI with timer
│   ├── package.json
│   └── vite.config.js
├── Dockerfile               # Multi-stage build for HF Spaces
└── README.md

Agents

No CrewAI or agent framework is used. The pipeline is a linear sequence of function calls:

Step	Function	What it does
Frame extraction	extract_frames()	Calls FFmpeg
Captioning	get_frame_captions()	Runs vision model on each frame
Room classification	classify_room()	Keyword matching
Item assembly	run_inventory_pipeline()	Builds structured data in Python
Summary	LocalTextLLM.call()	Runs text model once
PDF	build_pdf_report()	ReportLab layout

Previous iterations attempted multi-agent orchestration with CrewAI and API-based models (Gemini, Groq), but these were replaced with the current fully-local, sequential approach for reliability and zero external dependencies.

Environment Variables

Variable	Default	Description
LOCAL_TEXT_MODEL	google/flan-t5-small	Hugging Face model for text generation
LOCAL_VISION_MODEL	nlpconnect/vit-gpt2-image-captioning	Hugging Face model for image captioning

Running Locally

# Backend
cd backend
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 7860

# Frontend (separate terminal)
cd frontend
npm install
npm run dev

Deployment

Deploys to Hugging Face Docker Spaces. Only requirement is a valid HF token:

git push https://<USER>:<HF_TOKEN>@huggingface.co/spaces/<USER>/<SPACE> main