Spaces:
Sleeping
Sleeping
| title: AutoInventory | |
| emoji: 🏠 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: docker | |
| pinned: false | |
| # AutoInventory | |
| AI-powered property inventory report generator. Upload a 1-minute walkthrough video and receive a professional PDF inventory report. | |
| ## Tech Stack | |
| ### Frontend | |
| - **React 18** with Vite | |
| - **Tailwind CSS 3** — dark theme with copper accent palette | |
| - Client-side drag-and-drop upload with live elapsed timer and stage tracking | |
| ### Backend | |
| - **FastAPI** — single POST endpoint (`/api/generate-report`) | |
| - **Uvicorn** — ASGI server | |
| - **FFmpeg** — frame extraction (1 frame every 5 seconds) | |
| - **ReportLab** — PDF generation with professional layout (cover page, room tables, declaration) | |
| ### Machine Learning (Hugging Face Transformers) | |
| - **Vision model**: `nlpconnect/vit-gpt2-image-captioning` — captions each extracted frame | |
| - **Text model**: `google/flan-t5-small` — generates executive summary only | |
| - Both run locally on CPU via PyTorch — no external APIs, no API keys, fully open-source | |
| ### Infrastructure | |
| - **Docker** — multi-stage build (Node → React dist, Python → FastAPI) | |
| - **Hugging Face Docker Spaces** — deployment target (port 7860) | |
| ## How It Works | |
| ### Pipeline | |
| 1. **Upload** — User drags an MP4/MOV video onto the React UI | |
| 2. **Frame Extraction** — FFmpeg extracts one frame every 5 seconds into `/tmp` | |
| 3. **Captioning** — Each frame is passed through `vit-gpt2-image-captioning` to generate a text description (e.g. "a living room with a couch and a coffee table") | |
| 4. **Room Classification** — Each caption is matched against keyword lists to determine the room (Living Room, Kitchen, Bedroom, etc.). No LLM involved — pure keyword matching. | |
| 5. **Item Generation** — For each room, standard inventory items (Door, Ceiling, Walls, Skirting, Window, Flooring, etc.) are created. Descriptions are sourced from the captions that best match each item type. | |
| 6. **Executive Summary** — `flan-t5-small` (via Hugging Face transformers pipeline) writes a one-paragraph summary of the inspection | |
| 7. **PDF Build** — ReportLab assembles a professional multi-page PDF with cover page, table of contents, room-by-room tables, and declaration page | |
| 8. **Download** — PDF is returned to the browser and automatically downloaded | |
| 9. **Cleanup** — Background task removes video, frames, and PDF from `/tmp` | |
| ### File Structure | |
| ``` | |
| ├── backend/ | |
| │ ├── main.py # FastAPI app, pipeline, PDF builder | |
| │ └── requirements.txt # Python dependencies | |
| ├── frontend/ | |
| │ ├── src/ | |
| │ │ └── App.jsx # React upload UI with timer | |
| │ ├── package.json | |
| │ └── vite.config.js | |
| ├── Dockerfile # Multi-stage build for HF Spaces | |
| └── README.md | |
| ``` | |
| ### Agents | |
| No CrewAI or agent framework is used. The pipeline is a linear sequence of function calls: | |
| | Step | Function | What it does | | |
| |------|----------|-------------| | |
| | Frame extraction | `extract_frames()` | Calls FFmpeg | | |
| | Captioning | `get_frame_captions()` | Runs vision model on each frame | | |
| | Room classification | `classify_room()` | Keyword matching | | |
| | Item assembly | `run_inventory_pipeline()` | Builds structured data in Python | | |
| | Summary | `LocalTextLLM.call()` | Runs text model once | | |
| | PDF | `build_pdf_report()` | ReportLab layout | | |
| Previous iterations attempted multi-agent orchestration with CrewAI and API-based models (Gemini, Groq), but these were replaced with the current fully-local, sequential approach for reliability and zero external dependencies. | |
| ## Environment Variables | |
| | Variable | Default | Description | | |
| |----------|---------|-------------| | |
| | `LOCAL_TEXT_MODEL` | `google/flan-t5-small` | Hugging Face model for text generation | | |
| | `LOCAL_VISION_MODEL` | `nlpconnect/vit-gpt2-image-captioning` | Hugging Face model for image captioning | | |
| ## Running Locally | |
| ```bash | |
| # Backend | |
| cd backend | |
| pip install -r requirements.txt | |
| uvicorn main:app --host 0.0.0.0 --port 7860 | |
| # Frontend (separate terminal) | |
| cd frontend | |
| npm install | |
| npm run dev | |
| ``` | |
| ## Deployment | |
| Deploys to Hugging Face Docker Spaces. Only requirement is a valid HF token: | |
| ```bash | |
| git push https://<USER>:<HF_TOKEN>@huggingface.co/spaces/<USER>/<SPACE> main | |
| ``` | |