CLAUDE.md

CLAUDE.md

Project Overview

SolarWine — Photosynthesis prediction and tracker control system for an agrivoltaic vineyard (Vitis vinifera) in Sde Boker, Israel.

Two-stage pipeline:

1. Stage 1 (Mechanistic): Farquhar et al. (1980) model driven by on-site sensors (PAR, leaf temp, CO₂, VPD) → net photosynthesis rate A (µmol CO₂/m²/s)
2. Stage 2 (ML): IMS weather station data → predict A without on-site sensors, enabling day-ahead forecasting

Repository Layout

backend/          FastAPI REST API (routes, auth, workers)
config/           settings.py (paths, IMS channels, panel geometry)
context/          Project docs, todo, code review notes
frontend/         React + TypeScript + Vite SPA
scripts/          CLI tools (data download, pipeline, training)
src/              Core Python modules
  advisor/        Day-ahead advisory engine
  chatbot/        LLM integration (Google Gemini)
  data/           IMS client, ThingsBoard client, DataHub
  forecasting/    ML predictors, time-series, Chronos wrapper
  models/         Farquhar model, phenology
  shading/        Solar geometry, tracker optimizer
tracker/          Tracker optimization module
ui/               Streamlit UI utilities
app.py            Legacy Streamlit dashboard

Tech Stack

Backend: Python 3.12, FastAPI, uvicorn, pydantic, PyJWT, slowapi, sentry-sdk ML/Science: pandas, numpy, scikit-learn, xgboost, pvlib, scipy, torch, chronos-forecasting (HuggingFace) LLM: google-genai (Gemini) Frontend: React 18, TypeScript 5, Vite 5, Tailwind CSS, shadcn/ui, Recharts, TanStack React Query, React Router 6 Data sources: IMS API (station 43, Sde Boker), ThingsBoard (MQTT/REST), on-site sensors Deploy: HuggingFace Spaces (backend, port 7860), Cloudflare Pages (frontend), Streamlit Community Cloud (legacy)

Environment Setup

# Python (from repo root)
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -r backend/requirements.txt

# Frontend
cd frontend && npm install

Required .env variables:

IMS_API_TOKEN=<token>
JWT_SECRET=<secret>
ADMIN_PASSWORD=<password>
# Optional
SENTRY_DSN=<dsn>
ALLOWED_ORIGINS=http://localhost:5173,https://solarwine.pages.dev

Common Commands

Backend

# Dev server (auto-reload)
uvicorn backend.api.main:app --reload --port 7860

# Production
uvicorn backend.api.main:app --host 0.0.0.0 --port 7860

Frontend

cd frontend
npm run dev          # Vite dev server → http://localhost:5173
npm run build        # Production build → dist/
npm run lint         # ESLint
npm run test         # Vitest
npm run test:watch   # Watch mode

Legacy Streamlit

streamlit run app.py

Data & Pipelines

# Download IMS weather data
python -m scripts.download_ims_data --from 2024-01-01 --to 2025-12-31

# Full Stage 1 → Stage 2 pipeline
python -m scripts.run_pipeline

# Fine-tune Chronos (long-context forecasting)
python scripts/run_chronos_long_training.py --device cpu --context-days 28 --num-steps 4000

# Generate trimmed sample CSV for cloud deploy
python -m scripts.create_sample_data

# Control system simulation / validation
python -m scripts.run_control_simulation
python -m scripts.verify_control_system

# Budget audit (compliance report)
python -m src.budget_audit --report        # weekly rollup
python -m src.budget_audit --daily 2026-03-29  # single day

# Load test (requires: pip install locust)
locust -f scripts/load_test.py --host https://solarwine-api.hf.space \
       --users 10 --spawn-rate 2 --run-time 5m --headless

Architecture Notes

Backend (FastAPI)

• DataHub (src/data/data_providers.py) — singleton facade over all providers; injected via backend/api/deps.py
• Routes (backend/api/routes/): health, weather, sensors, energy, photosynthesis, control, chatbot, biology, login, events (~27 endpoints)
• Auth: JWT issued at /api/auth/login; guest read-only mode when JWT_SECRET unset
• Middleware: CORS, rate-limiting (60 req/min via slowapi), request logging, global exception handler → JSON errors
• Workers (backend/workers/): async background tasks for daily planner and control ticks
• Services (backend/services/): DataFlowMonitor (per-source health), EmailAlerter (SMTP alerts on red status)
• Background loops: IMS refresh (2h), sensor poll (2min), data flow alert check (5min) — all push SSE events

Core Python Modules

Module	Purpose
src/models/farquhar_model.py	Leaf photosynthesis (FvCB mechanistic, Semillon sigmoid 28–32°C transition, vectorized compute_all())
src/data/ims_client.py	IMS REST API client
src/data/data_providers.py	DataHub facade (6 services: Weather, VineSensor, PS, Energy, Advisory, Biology)
src/data/thingsboard_client.py	ThingsBoard REST client (22 devices, 4 trackers, plant asset)
src/forecasting/predictor.py	Multi-model ML regressor (LR/DT/RF/GBR/XGB)
src/forecasting/ts_predictor.py	Multi-horizon lag-based time-series
src/forecasting/chronos_forecaster.py	HuggingFace Chronos wrapper
src/shading/tracker_optimizer.py	Tilt-angle simulation & stress scheduling
src/shading/solar_geometry.py	Sun position, shadow projection, canopy PAR
src/shading/tradeoff_engine.py	InterventionGate (pipeline pattern) + minimum-dose offset search
src/control_loop.py	Real-time 15-min control orchestration (DI via __init__())
src/budget_audit.py	Slot-level budget audit log (parquet) + weekly compliance report
src/energy_budget.py	Hierarchical Year→Month→Week→Day→Slot budget planner
src/command_arbiter.py	5-level priority stack + hysteresis filter
src/tracker_dispatcher.py	Send tilt commands to TB trackers, verify execution
src/advisor/day_ahead_advisor.py	Gemini-powered daily planning engine
src/day_ahead_planner.py	DP trajectory optimizer for daily slot plans
src/chatbot/vineyard_chatbot.py	Gemini chatbot with tool dispatch + guardrails
src/chatbot/guardrails.py	Query classifier (EN+HE), response validator, confidence estimation, range validation, cross-source consistency
backend/services/data_flow_monitor.py	Per-source data freshness monitor (green/yellow/red)
backend/services/email_alerter.py	SMTP email alerts with per-source cooldown

Frontend

• Pages: Home, Agro, Trackers, Power, Monitoring (alarms + system health), Advisor, Control, Photosynthesis, Shading, Docs, Research
• Custom hooks (src/hooks/): useWeather, useEnergy, useSensors, useChatbot, useControl, useAuth, useDataFlowStatus, useEventStream
• API client: src/lib/ — typed wrappers around backend endpoints
• Auth: JWT stored in localStorage; TanStack React Query for server state

Data Flow

IMS API ──→ [Redis/CSV cache] ──→ Backend ──→ API Routes ──→ React Frontend
ThingsBoard sensors ────────────→ DataHub ──→ API Routes
Farquhar + ML models ───────────→ PhotosynthesisService ──→ /photosynthesis/*
TrackerOptimizer + EnergyBudget → ControlSystem ──→ /control/*
DataFlowMonitor ────────────────→ /health/data-sources ──→ DataFlowStatus component
                                → EmailAlerter ──→ SMTP alerts
Backend events ─────────────────→ SSE /events/stream ──→ auto-invalidate React Query

Key Files to Know

• config/settings.py — central config (site coordinates, IMS channel IDs, panel geometry, budget params)
• backend/api/main.py — FastAPI app entry point (middleware, router registration)
• backend/api/routes/ — all API endpoint implementations (~27 endpoints)
• backend/workers/control_tick.py — 15-min control loop cron entry point
• backend/workers/daily_planner.py — daily plan generation cron entry point
• src/models/farquhar_model.py — core mechanistic model
• src/control_loop.py — main control loop (sensors → plan → gate → arbiter → dispatch)
• src/energy_budget.py — hierarchical energy budget planner
• src/chatbot/guardrails.py — anti-hallucination guardrails (query classifier, range validation, cross-source checks)
• backend/services/data_flow_monitor.py — data source health monitoring
• backend/services/email_alerter.py — SMTP email alerts for data outages
• backend/api/routes/events.py — SSE event stream for live frontend updates
• frontend/src/lib/api.ts — API client with JWT auth
• frontend/src/hooks/ — TanStack Query hooks for all API endpoints
• scripts/load_test.py — Locust load test (16 weighted endpoints)
• context/3_todo.md — current work items and backlog
• ims_api_documentation.md — IMS API reference (Hebrew + English)

Deployment

Platform	Target	Notes
HuggingFace Spaces	Backend	Docker, port 7860, see backend/HF_README.md
Cloudflare Pages	Frontend	VITE_API_URL env var points to HF backend
Streamlit Cloud	Legacy app	IMS_API_TOKEN secret, falls back to sensors_wide_sample.csv

Large files (Data/Seymour/sensors_wide.csv ~982 MB, outputs/) are gitignored. Cloud deployments use Data/Seymour/sensors_wide_sample.csv.