Blame for CLAUDE.md

# CLAUDE.md

## Project Overview

**SolarWine** — Photosynthesis prediction and tracker control system for an agrivoltaic vineyard (*Vitis vinifera*) in Sde Boker, Israel.

Two-stage pipeline:
1. **Stage 1 (Mechanistic)**: Farquhar et al. (1980) model driven by on-site sensors (PAR, leaf temp, CO₂, VPD) → net photosynthesis rate *A* (µmol CO₂/m²/s)
2. **Stage 2 (ML)**: IMS weather station data → predict *A* without on-site sensors, enabling day-ahead forecasting

## Repository Layout

```
backend/          FastAPI REST API (routes, auth, workers)
config/           settings.py (paths, IMS channels, panel geometry)
context/          Project docs, todo, code review notes
frontend/         React + TypeScript + Vite SPA
scripts/          CLI tools (data download, pipeline, training)
src/              Core Python modules
  advisor/        Day-ahead advisory engine
  chatbot/        LLM integration (Google Gemini)
  data/           IMS client, ThingsBoard client, DataHub
  forecasting/    ML predictors, time-series, Chronos wrapper
  models/         Farquhar model, phenology
  shading/        Solar geometry, tracker optimizer
tracker/          Tracker optimization module
ui/               Streamlit UI utilities
app.py            Legacy Streamlit dashboard
```

## Tech Stack

**Backend**: Python 3.12, FastAPI, uvicorn, pydantic, PyJWT, slowapi, sentry-sdk
**ML/Science**: pandas, numpy, scikit-learn, xgboost, pvlib, scipy, torch, chronos-forecasting (HuggingFace)
**LLM**: google-genai (Gemini)
**Frontend**: React 18, TypeScript 5, Vite 5, Tailwind CSS, shadcn/ui, Recharts, TanStack React Query, React Router 6
**Data sources**: IMS API (station 43, Sde Boker), ThingsBoard (MQTT/REST), on-site sensors
**Deploy**: HuggingFace Spaces (backend, port 7860), Cloudflare Pages (frontend), Streamlit Community Cloud (legacy)

## Environment Setup

```bash
# Python (from repo root)
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -r backend/requirements.txt

# Frontend
cd frontend && npm install
```

Required `.env` variables:
```
IMS_API_TOKEN=<token>
JWT_SECRET=<secret>
ADMIN_PASSWORD=<password>
# Optional
SENTRY_DSN=<dsn>
ALLOWED_ORIGINS=http://localhost:5173,https://solarwine.pages.dev
```

## Common Commands

### Backend

```bash
# Dev server (auto-reload)
uvicorn backend.api.main:app --reload --port 7860

# Production
uvicorn backend.api.main:app --host 0.0.0.0 --port 7860
```

### Frontend

```bash
cd frontend
npm run dev          # Vite dev server → http://localhost:5173
npm run build        # Production build → dist/
npm run lint         # ESLint
npm run test         # Vitest
npm run test:watch   # Watch mode
```

### Legacy Streamlit

```bash
streamlit run app.py
```

### Data & Pipelines

```bash
# Download IMS weather data
python -m scripts.download_ims_data --from 2024-01-01 --to 2025-12-31

# Full Stage 1 → Stage 2 pipeline
python -m scripts.run_pipeline

# Fine-tune Chronos (long-context forecasting)
python scripts/run_chronos_long_training.py --device cpu --context-days 28 --num-steps 4000

# Generate trimmed sample CSV for cloud deploy
python -m scripts.create_sample_data

# Control system simulation / validation
python -m scripts.run_control_simulation
python -m scripts.verify_control_system

# Budget audit (compliance report)
python -m src.budget_audit --report        # weekly rollup
python -m src.budget_audit --daily 2026-03-29  # single day

# Load test (requires: pip install locust)
locust -f scripts/load_test.py --host https://solarwine-api.hf.space \
       --users 10 --spawn-rate 2 --run-time 5m --headless
```

## Architecture Notes

### Backend (FastAPI)

- **`DataHub`** (`src/data/data_providers.py`) — singleton facade over all providers; injected via `backend/api/deps.py`
- **Routes** (`backend/api/routes/`): `health`, `weather`, `sensors`, `energy`, `photosynthesis`, `control`, `chatbot`, `biology`, `login`, `events` (~27 endpoints)
- **Auth**: JWT issued at `/api/auth/login`; guest read-only mode when `JWT_SECRET` unset
- **Middleware**: CORS, rate-limiting (60 req/min via slowapi), request logging, global exception handler → JSON errors
- **Workers** (`backend/workers/`): async background tasks for daily planner and control ticks
- **Services** (`backend/services/`): `DataFlowMonitor` (per-source health), `EmailAlerter` (SMTP alerts on red status)
- **Background loops**: IMS refresh (2h), sensor poll (2min), data flow alert check (5min) — all push SSE events

### Core Python Modules

| Module | Purpose |
|---|---|
| `src/models/farquhar_model.py` | Leaf photosynthesis (FvCB mechanistic, Semillon sigmoid 28–32°C transition, vectorized `compute_all()`) |
| `src/data/ims_client.py` | IMS REST API client |
| `src/data/data_providers.py` | DataHub facade (6 services: Weather, VineSensor, PS, Energy, Advisory, Biology) |
| `src/data/thingsboard_client.py` | ThingsBoard REST client (22 devices, 4 trackers, plant asset) |
| `src/forecasting/predictor.py` | Multi-model ML regressor (LR/DT/RF/GBR/XGB) |
| `src/forecasting/ts_predictor.py` | Multi-horizon lag-based time-series |
| `src/forecasting/chronos_forecaster.py` | HuggingFace Chronos wrapper |
| `src/shading/tracker_optimizer.py` | Tilt-angle simulation & stress scheduling |
| `src/shading/solar_geometry.py` | Sun position, shadow projection, canopy PAR |
| `src/shading/tradeoff_engine.py` | InterventionGate (pipeline pattern) + minimum-dose offset search |
| `src/control_loop.py` | Real-time 15-min control orchestration (DI via `__init__()`) |
| `src/budget_audit.py` | Slot-level budget audit log (parquet) + weekly compliance report |
| `src/energy_budget.py` | Hierarchical Year→Month→Week→Day→Slot budget planner |
| `src/command_arbiter.py` | 5-level priority stack + hysteresis filter |
| `src/tracker_dispatcher.py` | Send tilt commands to TB trackers, verify execution |
| `src/advisor/day_ahead_advisor.py` | Gemini-powered daily planning engine |
| `src/day_ahead_planner.py` | DP trajectory optimizer for daily slot plans |
| `src/chatbot/vineyard_chatbot.py` | Gemini chatbot with tool dispatch + guardrails |
| `src/chatbot/guardrails.py` | Query classifier (EN+HE), response validator, confidence estimation, range validation, cross-source consistency |
| `backend/services/data_flow_monitor.py` | Per-source data freshness monitor (green/yellow/red) |
| `backend/services/email_alerter.py` | SMTP email alerts with per-source cooldown |

### Frontend

- **Pages**: Home, Agro, Trackers, Power, Monitoring (alarms + system health), Advisor, Control, Photosynthesis, Shading, Docs, Research
- **Custom hooks** (`src/hooks/`): `useWeather`, `useEnergy`, `useSensors`, `useChatbot`, `useControl`, `useAuth`, `useDataFlowStatus`, `useEventStream`
- **API client**: `src/lib/` — typed wrappers around backend endpoints
- **Auth**: JWT stored in `localStorage`; TanStack React Query for server state

### Data Flow

```
IMS API ──→ [Redis/CSV cache] ──→ Backend ──→ API Routes ──→ React Frontend
ThingsBoard sensors ────────────→ DataHub ──→ API Routes
Farquhar + ML models ───────────→ PhotosynthesisService ──→ /photosynthesis/*
TrackerOptimizer + EnergyBudget → ControlSystem ──→ /control/*
DataFlowMonitor ────────────────→ /health/data-sources ──→ DataFlowStatus component
                                → EmailAlerter ──→ SMTP alerts
Backend events ─────────────────→ SSE /events/stream ──→ auto-invalidate React Query
```

## Key Files to Know

- `config/settings.py` — central config (site coordinates, IMS channel IDs, panel geometry, budget params)
- `backend/api/main.py` — FastAPI app entry point (middleware, router registration)
- `backend/api/routes/` — all API endpoint implementations (~27 endpoints)
- `backend/workers/control_tick.py` — 15-min control loop cron entry point
- `backend/workers/daily_planner.py` — daily plan generation cron entry point
- `src/models/farquhar_model.py` — core mechanistic model
- `src/control_loop.py` — main control loop (sensors → plan → gate → arbiter → dispatch)
- `src/energy_budget.py` — hierarchical energy budget planner
- `src/chatbot/guardrails.py` — anti-hallucination guardrails (query classifier, range validation, cross-source checks)
- `backend/services/data_flow_monitor.py` — data source health monitoring
- `backend/services/email_alerter.py` — SMTP email alerts for data outages
- `backend/api/routes/events.py` — SSE event stream for live frontend updates
- `frontend/src/lib/api.ts` — API client with JWT auth
- `frontend/src/hooks/` — TanStack Query hooks for all API endpoints
- `scripts/load_test.py` — Locust load test (16 weighted endpoints)
- `context/3_todo.md` — current work items and backlog
- `ims_api_documentation.md` — IMS API reference (Hebrew + English)

## Deployment

| Platform | Target | Notes |
|---|---|---|
| HuggingFace Spaces | Backend | Docker, port 7860, see `backend/HF_README.md` |
| Cloudflare Pages | Frontend | `VITE_API_URL` env var points to HF backend |
| Streamlit Cloud | Legacy app | `IMS_API_TOKEN` secret, falls back to `sensors_wide_sample.csv` |

Large files (`Data/Seymour/sensors_wide.csv` ~982 MB, `outputs/`) are gitignored. Cloud deployments use `Data/Seymour/sensors_wide_sample.csv`.