| # CLAUDE.md |
|
|
| ## Project Overview |
|
|
| **SolarWine** — Photosynthesis prediction and tracker control system for an agrivoltaic vineyard (*Vitis vinifera*) in Sde Boker, Israel. |
|
|
| Two-stage pipeline: |
| 1. **Stage 1 (Mechanistic)**: Farquhar et al. (1980) model driven by on-site sensors (PAR, leaf temp, CO₂, VPD) → net photosynthesis rate *A* (µmol CO₂/m²/s) |
| 2. **Stage 2 (ML)**: IMS weather station data → predict *A* without on-site sensors, enabling day-ahead forecasting |
|
|
| ## Repository Layout |
|
|
| ``` |
| backend/ FastAPI REST API (routes, auth, workers) |
| config/ settings.py (paths, IMS channels, panel geometry) |
| context/ Project docs, todo, code review notes |
| frontend/ React + TypeScript + Vite SPA |
| scripts/ CLI tools (data download, pipeline, training) |
| src/ Core Python modules |
| advisor/ Day-ahead advisory engine |
| chatbot/ LLM integration (Google Gemini) |
| data/ IMS client, ThingsBoard client, DataHub |
| forecasting/ ML predictors, time-series, Chronos wrapper |
| models/ Farquhar model, phenology |
| shading/ Solar geometry, tracker optimizer |
| tracker/ Tracker optimization module |
| ui/ Streamlit UI utilities |
| app.py Legacy Streamlit dashboard |
| ``` |
|
|
| ## Tech Stack |
|
|
| **Backend**: Python 3.12, FastAPI, uvicorn, pydantic, PyJWT, slowapi, sentry-sdk |
| **ML/Science**: pandas, numpy, scikit-learn, xgboost, pvlib, scipy, torch, chronos-forecasting (HuggingFace) |
| **LLM**: google-genai (Gemini) |
| **Frontend**: React 18, TypeScript 5, Vite 5, Tailwind CSS, shadcn/ui, Recharts, TanStack React Query, React Router 6 |
| **Data sources**: IMS API (station 43, Sde Boker), ThingsBoard (MQTT/REST), on-site sensors |
| **Deploy**: HuggingFace Spaces (backend, port 7860), Cloudflare Pages (frontend), Streamlit Community Cloud (legacy) |
|
|
| ## Environment Setup |
|
|
| ```bash |
| # Python (from repo root) |
| python -m venv .venv |
| source .venv/bin/activate |
| pip install -r requirements.txt |
| pip install -r backend/requirements.txt |
| |
| # Frontend |
| cd frontend && npm install |
| ``` |
|
|
| Required `.env` variables: |
| ``` |
| IMS_API_TOKEN=<token> |
| JWT_SECRET=<secret> |
| ADMIN_PASSWORD=<password> |
| # Optional |
| SENTRY_DSN=<dsn> |
| ALLOWED_ORIGINS=http://localhost:5173,https://solarwine.pages.dev |
| ``` |
|
|
| ## Common Commands |
|
|
| ### Backend |
|
|
| ```bash |
| # Dev server (auto-reload) |
| uvicorn backend.api.main:app --reload --port 7860 |
| |
| # Production |
| uvicorn backend.api.main:app --host 0.0.0.0 --port 7860 |
| ``` |
|
|
| ### Frontend |
|
|
| ```bash |
| cd frontend |
| npm run dev # Vite dev server → http://localhost:5173 |
| npm run build # Production build → dist/ |
| npm run lint # ESLint |
| npm run test # Vitest |
| npm run test:watch # Watch mode |
| ``` |
|
|
| ### Legacy Streamlit |
|
|
| ```bash |
| streamlit run app.py |
| ``` |
|
|
| ### Data & Pipelines |
|
|
| ```bash |
| # Download IMS weather data |
| python -m scripts.download_ims_data --from 2024-01-01 --to 2025-12-31 |
| |
| # Full Stage 1 → Stage 2 pipeline |
| python -m scripts.run_pipeline |
| |
| # Fine-tune Chronos (long-context forecasting) |
| python scripts/run_chronos_long_training.py --device cpu --context-days 28 --num-steps 4000 |
| |
| # Generate trimmed sample CSV for cloud deploy |
| python -m scripts.create_sample_data |
| |
| # Control system simulation / validation |
| python -m scripts.run_control_simulation |
| python -m scripts.verify_control_system |
| |
| # Budget audit (compliance report) |
| python -m src.budget_audit --report # weekly rollup |
| python -m src.budget_audit --daily 2026-03-29 # single day |
| |
| # Load test (requires: pip install locust) |
| locust -f scripts/load_test.py --host https://solarwine-api.hf.space \ |
| --users 10 --spawn-rate 2 --run-time 5m --headless |
| ``` |
|
|
| ## Architecture Notes |
|
|
| ### Backend (FastAPI) |
|
|
| - **`DataHub`** (`src/data/data_providers.py`) — singleton facade over all providers; injected via `backend/api/deps.py` |
| - **Routes** (`backend/api/routes/`): `health`, `weather`, `sensors`, `energy`, `photosynthesis`, `control`, `chatbot`, `biology`, `login`, `events` (~27 endpoints) |
| - **Auth**: JWT issued at `/api/auth/login`; guest read-only mode when `JWT_SECRET` unset |
| - **Middleware**: CORS, rate-limiting (60 req/min via slowapi), request logging, global exception handler → JSON errors |
| - **Workers** (`backend/workers/`): async background tasks for daily planner and control ticks |
| - **Services** (`backend/services/`): `DataFlowMonitor` (per-source health), `EmailAlerter` (SMTP alerts on red status) |
| - **Background loops**: IMS refresh (2h), sensor poll (2min), data flow alert check (5min) — all push SSE events |
|
|
| ### Core Python Modules |
|
|
| | Module | Purpose | |
| |---|---| |
| | `src/models/farquhar_model.py` | Leaf photosynthesis (FvCB mechanistic, Semillon sigmoid 28–32°C transition, vectorized `compute_all()`) | |
| | `src/data/ims_client.py` | IMS REST API client | |
| | `src/data/data_providers.py` | DataHub facade (6 services: Weather, VineSensor, PS, Energy, Advisory, Biology) | |
| | `src/data/thingsboard_client.py` | ThingsBoard REST client (22 devices, 4 trackers, plant asset) | |
| | `src/forecasting/predictor.py` | Multi-model ML regressor (LR/DT/RF/GBR/XGB) | |
| | `src/forecasting/ts_predictor.py` | Multi-horizon lag-based time-series | |
| | `src/forecasting/chronos_forecaster.py` | HuggingFace Chronos wrapper | |
| | `src/shading/tracker_optimizer.py` | Tilt-angle simulation & stress scheduling | |
| | `src/shading/solar_geometry.py` | Sun position, shadow projection, canopy PAR | |
| | `src/shading/tradeoff_engine.py` | InterventionGate (pipeline pattern) + minimum-dose offset search | |
| | `src/control_loop.py` | Real-time 15-min control orchestration (DI via `__init__()`) | |
| | `src/budget_audit.py` | Slot-level budget audit log (parquet) + weekly compliance report | |
| | `src/energy_budget.py` | Hierarchical Year→Month→Week→Day→Slot budget planner | |
| | `src/command_arbiter.py` | 5-level priority stack + hysteresis filter | |
| | `src/tracker_dispatcher.py` | Send tilt commands to TB trackers, verify execution | |
| | `src/advisor/day_ahead_advisor.py` | Gemini-powered daily planning engine | |
| | `src/day_ahead_planner.py` | DP trajectory optimizer for daily slot plans | |
| | `src/chatbot/vineyard_chatbot.py` | Gemini chatbot with tool dispatch + guardrails | |
| | `src/chatbot/guardrails.py` | Query classifier (EN+HE), response validator, confidence estimation, range validation, cross-source consistency | |
| | `backend/services/data_flow_monitor.py` | Per-source data freshness monitor (green/yellow/red) | |
| | `backend/services/email_alerter.py` | SMTP email alerts with per-source cooldown | |
|
|
| ### Frontend |
|
|
| - **Pages**: Home, Agro, Trackers, Power, Monitoring (alarms + system health), Advisor, Control, Photosynthesis, Shading, Docs, Research |
| - **Custom hooks** (`src/hooks/`): `useWeather`, `useEnergy`, `useSensors`, `useChatbot`, `useControl`, `useAuth`, `useDataFlowStatus`, `useEventStream` |
| - **API client**: `src/lib/` — typed wrappers around backend endpoints |
| - **Auth**: JWT stored in `localStorage`; TanStack React Query for server state |
|
|
| ### Data Flow |
|
|
| ``` |
| IMS API ──→ [Redis/CSV cache] ──→ Backend ──→ API Routes ──→ React Frontend |
| ThingsBoard sensors ────────────→ DataHub ──→ API Routes |
| Farquhar + ML models ───────────→ PhotosynthesisService ──→ /photosynthesis/* |
| TrackerOptimizer + EnergyBudget → ControlSystem ──→ /control/* |
| DataFlowMonitor ────────────────→ /health/data-sources ──→ DataFlowStatus component |
| → EmailAlerter ──→ SMTP alerts |
| Backend events ─────────────────→ SSE /events/stream ──→ auto-invalidate React Query |
| ``` |
|
|
| ## Key Files to Know |
|
|
| - `config/settings.py` — central config (site coordinates, IMS channel IDs, panel geometry, budget params) |
| - `backend/api/main.py` — FastAPI app entry point (middleware, router registration) |
| - `backend/api/routes/` — all API endpoint implementations (~27 endpoints) |
| - `backend/workers/control_tick.py` — 15-min control loop cron entry point |
| - `backend/workers/daily_planner.py` — daily plan generation cron entry point |
| - `src/models/farquhar_model.py` — core mechanistic model |
| - `src/control_loop.py` — main control loop (sensors → plan → gate → arbiter → dispatch) |
| - `src/energy_budget.py` — hierarchical energy budget planner |
| - `src/chatbot/guardrails.py` — anti-hallucination guardrails (query classifier, range validation, cross-source checks) |
| - `backend/services/data_flow_monitor.py` — data source health monitoring |
| - `backend/services/email_alerter.py` — SMTP email alerts for data outages |
| - `backend/api/routes/events.py` — SSE event stream for live frontend updates |
| - `frontend/src/lib/api.ts` — API client with JWT auth |
| - `frontend/src/hooks/` — TanStack Query hooks for all API endpoints |
| - `scripts/load_test.py` — Locust load test (16 weighted endpoints) |
| - `context/3_todo.md` — current work items and backlog |
| - `ims_api_documentation.md` — IMS API reference (Hebrew + English) |
|
|
| ## Deployment |
|
|
| | Platform | Target | Notes | |
| |---|---|---| |
| | HuggingFace Spaces | Backend | Docker, port 7860, see `backend/HF_README.md` | |
| | Cloudflare Pages | Frontend | `VITE_API_URL` env var points to HF backend | |
| | Streamlit Cloud | Legacy app | `IMS_API_TOKEN` secret, falls back to `sensors_wide_sample.csv` | |
|
|
| Large files (`Data/Seymour/sensors_wide.csv` ~982 MB, `outputs/`) are gitignored. Cloud deployments use `Data/Seymour/sensors_wide_sample.csv`. |
|
|
