Blame for context/3_todo.md

# SolarWine – Todo Tracker

This file mirrors the high-level plan in `2_plan.md` and tracks day-to-day progress.

---

## Baseline v1 (Complete)

- [x] **scaffold** – project directory structure, `.env`, `.gitignore`, `requirements.txt`
- [x] **context-files** – create `context/1_purpose.md`, `context/2_plan.md`, `context/3_todo.md`
- [x] **git-init** – create git repository and make initial commit
- [x] **oop-architecture** – implement OOP modules in `src/`
- [x] **ims-fetch** – implement `IMSClient` and cache IMS data
- [x] **stage1-farquhar** – implement mechanistic Farquhar/Greer–Weedon model
- [x] **stage1-validate** – validate Stage 1 A outputs and plots
- [x] **stage2-preprocess** – preprocessing and temporal split
- [x] **stage2-models** – train and evaluate ML models
- [x] **stage2-visualize** – build evaluation visualizations
- [x] **streamlit-app** – build Streamlit UI (6 tabs)

---

## SolarWine 2.0

### Phase 1: Configuration & Data Infrastructure

- [x] **config-extend** – add energy budget, no-shade windows, fruiting zone, and tradeoff params to `config/settings.py` (most params added in earlier session; no-shade windows `NO_SHADE_BEFORE_HOUR`, `NO_SHADE_MONTHS`, `NO_SHADE_GHI_BELOW`, `NO_SHADE_TLEAF_BELOW` and `SHADE_ELIGIBLE_HOURS` added 2026-03-10; guardrails wired to reference settings)
- [x] **data-schema** – `src/data/data_schema.py` (telemetry dataclasses: SensorRaw, BiologicalState, TrackerKinematics, SimulationLog). Re-exported via `src/data_schema.py`
- [x] **import-layout** – `scripts/import_layout.py` generates `Data/layout.json` from config geometry + TB device registry (4 rows, 22 devices with 3D positions). Added 2026-03-10

### Phase 2: Biological Digital Twin Upgrades

- [x] **fvcb-semillon** – upgrade `src/farquhar_model.py` with `calc_photosynthesis_semillon()` → returns (A, limiting_state, shading_helps). Added 2026-03-10: uses configurable `transition_temp` (default `SEMILLON_TRANSITION_TEMP_C=30°C`); `shading_helps` is True only when Rubisco-limited AND Aj > Ac (light abundant relative to enzyme capacity)
- [x] **shadow-fruiting** – upgrade `src/solar_geometry.py` with `fruiting_zone_shadow()` and `evaluate_candidate_offsets()` (added 2026-03-10)
- [x] **canopy-expose** – `compute_vine_A()` now returns `fruiting_zone_A`, `fruiting_zone_par`, `top_canopy_A`, `top_canopy_par` alongside existing fields. Added 2026-03-10
- [x] **safety-rails** – `src/advisor/safety_rails.py` (FvCB vs ML divergence check, 12% threshold → fallback). Re-exported via `src/safety_rails.py`
- [x] **spectral-agg** – `src/spectral_aggregator.py` with `aggregate_spectral()` (single-timestep) and `aggregate_spectral_df()` (batch DataFrame). CWSI from explicit value → delta-T proxy → VPD fallback. Physical bounds clipping with quality flags. Added 2026-03-10

### Phase 3: The Control Algorithm (Core)

- [x] **energy-budget** – create `src/energy_budget.py` (hierarchical Year→Month→Week→Day→Slot budget planner). Added 2026-03-10: `EnergyBudgetPlanner` with `compute_annual_plan()`, `compute_weekly_plan()`, `compute_daily_plan()`, `spend_slot()`, `emergency_draw()`, `compute_daily_rollover()`. Analytical fallback when no ShadowModel. Budget: ~920 kWh potential → 46 kWh budget (5%), July gets 17.6 kWh (45%)
- [x] **intervention-gate** – `InterventionGate` in `src/shading/tradeoff_engine.py` (physiology-only pass/fail: GHI, Tleaf, CWSI, shading_helps; no time/month hard-coding — geometry decides). Re-exported via `src/tradeoff_engine.py`
- [x] **min-dose-engine** – `TradeoffEngine` in `src/shading/tradeoff_engine.py` (smallest offset satisfying: fruiting PAR < 400 µmol, top canopy ≥ 85% baseline, sacrifice ≤ budget; geometric feasibility pre-check; face-PAR sun-side selection)
- [x] **command-arbiter** – `src/command_arbiter.py` `CommandArbiter` (5-level priority stack: weather→harvest→safety→timeout→engine; hysteresis filter with configurable window/tolerance; `arbitrate()` combines selection + filter; weather/harvest bypass hysteresis). Added 2026-03-10
- [x] **astro-tracker** – `AstronomicalTracker` in `src/command_arbiter.py` (pure sun-following via ShadowModel; `get_angle(timestamp)` interface). Added 2026-03-10

### Phase 3.5: Agronomic Weighting & Day-Ahead Planner

- [x] **zone-crop-weights** – spatial zone weighting implemented in `DayAheadPlanner._get_crop_multiplier()` using `STAGE_CROP_MULTIPLIER` + `ZONE_CROP_WEIGHTS` from settings. Added 2026-03-10
- [x] **day-ahead-dp** – created `src/day_ahead_planner.py` with `DayAheadPlanner` (DP trajectory optimizer: forward pass over daylight slots × candidate offsets, budget constraint via discretised DP, movement cost penalty for smooth trajectories, simplified InterventionGate using forecast data). Returns `DayAheadPlan` with `SlotPlan` list + explainability tags. Added 2026-03-10
- [x] **daily-plan-output** – `DayAheadPlan.to_dict()` serialises plan to JSON-ready dict with per-slot offset, energy cost, gate status, and tags. Added 2026-03-10
- [x] **phenology-tracker** – `src/models/phenology.py` upgraded with three estimation methods: (1) GDD-based via `compute_gdd_from_ims()` + `estimate_stage_by_gdd()` using IMS tdmax/tdmin and `PHENOLOGY_GDD_THRESHOLDS`, (2) camera-based via `detect_stage_from_camera()` using Gemini Vision on live vineyard feed, (3) calendar fallback. `estimate_stage_combined()` merges all three (camera high-confidence > GDD > calendar). Backward-compatible API. Added 2026-03-10
- [x] **baseline-ps** – created `src/baseline_predictor.py` (hybrid FvCB+ML with RoutingAgent rule-based selection per slot); wired into DayAheadPlanner as optional `baseline_predictor` param, replaces temperature heuristic with actual A predictions for crop value
- [x] **arbiter-plan-integration** – ControlLoop now tracks plan divergence (cumulative kWh + consecutive slots), triggers re-plan when thresholds exceeded, with cooldown to avoid thrashing
- [x] **tracker-rpc** – added `send_rpc_command()` and `set_device_attributes()` to ThingsBoardClient (two-way RPC with one-way fallback, shared attribute writes)
- [x] **tracker-verify** – TrackerDispatcher verifies |actual - target| < tolerance after dispatch via `_verify_all()`
- [x] **tracker-dispatch** – created `src/tracker_dispatcher.py`: takes `ArbiterDecision` → sends to all 4 trackers via RPC/attribute fallback → verifies execution → logs result
- [x] **control-loop** – created `src/control_loop.py`: 15-min main loop (fetch sensors → plan lookup → live gate check → budget guard → arbitrate → fleet overrides → dispatch → budget spend → divergence check → re-plan → log). One-shot, continuous, and dry_run modes. TrackerFleet/Scheduler wired for per-tracker overrides (default = all same). EnergyBudgetPlanner wired for real-time budget consumption. Plan divergence tracking with auto re-plan

### Phase 4: Operational Modes

- [x] **ops-modes** – created `src/operational_modes.py` (WindStow, HailStow, HeatShield, HarvestMode + composite `OperationalModeChecker`; integrated into ControlLoop)

### Phase 5: ROI Reporting

- [x] **roi-service** – created `src/roi_service.py` (BudgetStatus, InterventionStats, LERResult, ROIService with budget tracking + LER computation)

### Phase 6: Dashboard v2

- [x] **tab-system-status** – unified System Status tab (`ui/tab_system_status.py`) with 5 sub-sections:
  - **Tracker Status**: live angles from TB, coherence check, bar chart
  - **Energy Budget**: annual/monthly budget allocation, gauge, weekly plan
  - **Control Replay**: load simulation logs, timeline charts, slot-by-slot detail
  - **Fruiting Zone**: live canopy temp/PAR/VPD, sunburn risk assessment, thresholds
  - **ROI Dashboard**: budget utilisation gauge, intervention stats, LER calculator

### Phase 7: Integration Testing

- [x] **sim-script** – created `scripts/run_control_simulation.py` (historical replay through full control loop with per-day summary, CLI args, JSON output)
- [x] **validate-budget** – `scripts/validate_control_system.py` verifies cumulative sacrifice never exceeds `max_energy_reduction_pct` (tested with real sim log + synthetic violations)
- [x] **validate-gates** – validates no interventions in May, before 10:00, below transition temp, or below GHI threshold (3/3 violation types caught in self-test)
- [x] **validate-dose** – validates average offset < 10°, max offset < 20°, intervention rate < 30% (catches excessive dose in self-test)

### Phase 8: AI-Enhanced Forecasting & Routing

#### 8A — Chronos-2 Multivariate Day-Ahead Forecasting

- [x] **install-chronos** – install `chronos-forecasting>=2.0` and verify compatibility with existing env (done: conda env `solarwine` with Python 3.12, chronos 2.2.2, torch 2.10, llvm-openmp for XGBoost)
- [x] **chronos-data-prep** – build multivariate input pipeline: combine historical A_n (target) with past IMS station 43 measurements + on-site sensors (PAR, T_canopy, VPD, CWSI) as historical context, plus IMS *forecast* data (predicted GHI, air temperature, wind speed, humidity) as known-future covariates for the prediction horizon; IMS forecasts are available at inference time so this is not a data leak — only actual future sensor measurements would be
- [x] **chronos-predict** – execute zero-shot 96-step (24h @ 15-min) day-ahead A_n prediction using Chronos-2 (`amazon/chronos-t5-small` or `-base`) with multivariate context (past A_n + past sensor measurements as history, IMS forecasts as known-future covariates); integrate into `src/day_ahead_planner.py` baseline prediction
- [x] **chronos-benchmark** – benchmark Chronos-2 MAE against existing ML baseline (MAE ≈ 2.7) on held-out test set; compare univariate (A_n only) vs multivariate (A_n + IMS covariates) performance; document results and decide on integration path

#### 8B — Gemini LLM Data Engineering

- [x] **install-gemini** – `google-genai>=1.0` added to `requirements.txt`; already installed in `solarwine` conda env (v1.65.0)
- [x] **llm-data-cleaning** – `src/llm_data_engineer.py` `LLMDataEngineer.analyze_anomalies()`: sends per-column descriptive stats + domain context (physical ranges, Negev site notes) to Gemini; receives Z-score/IQR/bound thresholds as JSON; `apply_cleaning()` applies clip/nan/drop strategies; graceful fallback to physics-based bounds when API key unavailable
- [x] **llm-feature-eng** – `LLMDataEngineer.engineer_features()`: adds `hour_sin`, `hour_cos`, `doy_sin`, `doy_cos` (cyclical time encodings) and `stress_risk_score` (normalised weighted VPD + CWSI, weights confirmed by Gemini); `run_pipeline()` orchestrates clean → feature-engineer in one call
- [x] **dashboard-8b** – Data Explorer: added **AI Data Engineering** as third radio option (Gemini anomaly thresholds, cleaning summary, before/after charts, engineered features, daytime stress profile). Presentation tab: added Gemini LLM data engineering to “What’s Next” completed list, Key takeaways, and “What each tab shows” (Data Explorer bullet). All data precomputed in-tab (no cache loader).

#### 8C — Gemini Intelligent Routing Layer

- [x] **routing-prompt** – define supervisory system prompt for Gemini model router: given real-time telemetry (Temp, Irradiance, CWSI), route to FvCB (Model A, accurate under standard conditions) or ML (Model B, handles non-linear stress) — reply only `MODEL_A` or `MODEL_B`
- [x] **routing-agent** – build `RoutingAgent` function: executes Gemini API call at start of 15-min control loop, constructs prompt from live telemetry, parses response to determine model path
- [x] **arbiter-routing** – RoutingAgent wired into ControlLoop tick (step 2b): routes each slot to FvCB or ML via rule-based fast path (>90% of cases) with Gemini fallback for transition zone; `model_route` field added to TickResult; safety_rails divergence check remains as independent safety layer

#### 8E — Gemini Day-Ahead Advisor

- [x] **day-ahead-advisor** – `src/day_ahead_advisor.py` DayAheadAdvisor: Gemini analyzes IMS forecast + vine state → structured stress profile, budget recommendations, model routing, Chronos sanity check; dashboard display in Predictions tab

#### 8D — Shadow Mode Validation

- [x] **shadow-mode-test** – created `scripts/shadow_mode_test.py`: 4 validation tests — (1) LLM data cleaning filters catch anomalies, (2) BaselinePredictor generates valid A profiles (peak=17.0 µmol, daylight mean=6.1), (3) RoutingAgent routes cool→fvcb and hot→ml correctly, (4) full pipeline shadow simulation over date range

### Phase 9: Vineyard Advisor Chatbot

- [x] **chatbot-todo** – add Phase 9 to `context/3_todo.md`
- [x] **chatbot-scaffold** – create `src/vineyard_chatbot.py` scaffold + Gemini client (lazy init, same pattern as DayAheadAdvisor)
- [x] **chatbot-system-prompt** – write `CHATBOT_SYSTEM_PROMPT` (persona, site facts, 8 biology rules, tool definitions, response rules)
- [x] **chatbot-tool-defs** – implement `TOOL_DEFINITIONS` list + `_dispatch_tool()` router
- [x] **chatbot-tools-weather** – implement `_tool_current_weather()`, `_tool_calc_photosynthesis()`, `_tool_explain_rule()`
- [x] **chatbot-tools-shading** – implement `_tool_simulate_shading()`, `_tool_compare_angles()`, `_tool_daily_schedule()`, `_tool_day_ahead_advisory()`
- [x] **chatbot-chat-flow** – implement two-pass chat flow (`_build_messages()`, `_call_gemini()`, `chat()`)
- [x] **chatbot-tab** – add Vineyard Advisor tab to `app.py` (Vineyard Advisor tab, chat UI, session state)
- [x] **chatbot-quick-actions** – add quick-action buttons + `_render_tool_data()` helper + clear chat
- [x] **chatbot-test** – syntax check, import check, tabs render, tool call test, fallback test

### Phase 9.5: Chatbot Data Integration

Wire all five data sources into the chatbot (IMS weather, TB sensors, PS prediction,
energy generation, energy prediction) via a loosely-coupled provider layer.

#### Architecture (`src/data_providers.py`)

```
VineyardChatbot  ──▶  DataHub  ──┬── WeatherService       (IMS, 30-min TTL cache)
  (tool dispatch)     (registry)  ├── VineSensorService    (TB, 5-min TTL snapshot)
                                  ├── PhotosynthesisService (FvCB + ML + forecast)
                                  ├── EnergyService        (TB generation + prediction)
                                  ├── AdvisoryService      (Gemini day-ahead)
                                  └── BiologyService       (rules lookup)
```

- Chatbot never imports data clients directly — only talks to `self.hub`.
- Each Service owns its caching (TTLCache), error handling, lazy init.
- Adding a new source = write a Service subclass + `hub.register()`.
- `summarise_dataframe()` auto-compresses large results for LLM context.

#### Data strategy

| Source | Service | Strategy | Detail |
|---|---|---|---|
| IMS weather | WeatherService | Cache (preload) | `ims_merged_15min.csv`, 30-min TTL |
| TB sensor snapshot | VineSensorService | On-demand + 5-min TTL | `get_vine_snapshot()` |
| TB sensor time-series | VineSensorService | On-demand | `get_timeseries()` → hourly downsample |
| PS prediction (FvCB/ML) | PhotosynthesisService | On-demand (compute) | Lazy-load models |
| Energy generation (TB) | EnergyService | On-demand | Fetch from inverter/meter devices |
| Energy prediction | EnergyService | On-demand (compute) | GHI × panel × efficiency × cos(θ) |

#### Steps

- [x] **chatbot-provider-scaffold** – Create `src/data_providers.py` with BaseService ABC, TTLCache, summarise_dataframe, DataHub registry, and 6 service classes (Weather, VineSensor, Photosynthesis, Energy, Advisory, Biology)
- [x] **chatbot-hub-wiring** – Refactored `VineyardChatbot.__init__` to accept a `DataHub` (default: `DataHub.default()`); rewrote `_dispatch_tool()` to delegate to hub services (15 tools); removed all direct imports of IMSClient/ThingsBoardClient/FarquharModel from chatbot
- [x] **chatbot-ims-history** – Implemented `WeatherService.get_history()` with hourly resample + `summarise_dataframe()`; added `get_weather_history` tool to system prompt and dispatch
- [x] **chatbot-tb-timeseries** – Implemented `VineSensorService.get_history()` (device_type, area, hours_back → hourly averages via `get_timeseries()`); added `get_sensor_history` tool to system prompt and dispatch
- [x] **chatbot-ml-predictor** – Implemented `PhotosynthesisService.predict_ml()` with train-once-cache pattern: lazy loads sensor data + IMS, runs Preprocessor + PhotosynthesisPredictor, picks best model by MAE, auto-fills features from latest IMS if not provided. *Note: requires IMS cache with temperature/GHI overlapping sensor data dates*
- [x] **chatbot-ps-forecast** – Implemented `PhotosynthesisService.forecast_day_ahead()`: FvCB-based projection over IMS hourly weather (PAR~2×GHI, Tleaf~Tair+2°C, VPD from T+RH). Returns hourly A profile + peak/stress summary
- [x] **chatbot-energy-devices** – Discovered Plant asset "Yeruham Vineyard" (ASSET, not DEVICE) with `power` (W) + `production` (Wh/5min) keys. Added `ASSET_REGISTRY`, `get_asset_timeseries()`, `get_asset_latest()` to TB client. Also registered 4 Tracker devices (501-509) with angle/mode telemetry
- [x] **chatbot-energy-tool** – `EnergyService` rewritten: past/today → real TB Plant asset data (daily ~270-290 kWh for 48 kW system); future → analytical estimate (persistence forecast × system capacity). Hourly profiles in Israel local time
- [x] **chatbot-energy-predict** – `EnergyService.predict()` now routes: past/today → `get_daily_production()` (real TB), future → `_predict_analytical()` (IMS GHI × 48 kW)
- [x] **chatbot-prompt-update** – Updated system prompt with all 15 tools (grouped: weather, vine sensors, photosynthesis, shading, energy, advisory, biology); added energy/prediction/irrigation/fertiliser keywords to fallback matcher
- [x] **chatbot-integration-test** – All 15 tools dispatch correctly; verified imports + instantiation (no circular imports); tested fallback keyword matching; `summarise_dataframe()` compresses >48-row results. Remaining data gaps: IMS cache lacks GHI/temperature (only wind_speed_ms), no date overlap between IMS and sensors for ML training

### Phase 10: Reliable Advisor — Anti-Hallucination Guardrails

Deterministic guardrails to prevent the chatbot from hallucinating data,
violating biology rules, or answering data questions without tool grounding.

#### Step 1: Structured Response Schema
- [x] **structured-response** – Extended `ChatResponse` with `confidence`, `sources`, `caveats`, `rule_violations` fields; all responses carry grounding metadata

#### Step 2: Mandatory Tool Grounding
- [x] **query-classifier** – `classify_query()` in `src/chatbot/guardrails.py`: regex-based classification (data / knowledge / greeting / ambiguous); data queries force tool calls
- [x] **tool-enforcement** – If LLM answers a data query without calling a tool, chatbot re-prompts: "You MUST call a tool"

#### Step 3: Post-Response Rule Validator
- [x] **response-validator** – `validate_response()` in `src/chatbot/guardrails.py`: deterministic checks for no_shade_before_10 (block), no_shade_in_may (block), temperature_transition (warn), no_leaves_no_shade_problem (warn), no_shading_must_explain (warn)
- [x] **validator-integration** – Validator runs on every response in `chat()`; block-severity overrides response text; warn-severity appends caveats

#### Step 4: Source-Tagged Context Injection
- [x] **source-tagging** – `tag_tool_result()` adds `_source`, `_data_age_minutes`, `_freshness_warning` to tool results before Gemini Pass 2
- [x] **citation-prompt** – Pass 2 prompt instructs Gemini to cite source and timestamp when quoting numbers

#### Step 5: RAG Knowledge Base
- [x] **rag-knowledge-base** – Keyword-indexed rule retrieval with pinned rules (no_shade_before_10, energy_budget, temperature_transition) + top-scoring rules per query; `retrieve_relevant_rules()` + `build_contextual_prompt()` in vineyard_chatbot.py

#### Step 6: Conversation Memory
- [x] **sliding-context** – Summarize older messages into topics, keep recent 6 verbatim, pin critical context; `_summarize_history()` in vineyard_chatbot.py

#### Step 7: Confidence-Gated Responses
- [x] **confidence-estimator** – `estimate_confidence()` maps data freshness to high/medium/low/insufficient_data
- [x] **confidence-ui** – `_render_grounding_metadata()` in `ui/tab_advisor.py` shows colored confidence badge, sources, caveats, rule warnings

#### Step 8: Guardrail Test Suite
- [x] **qa-tests-5** – 5 adversarial QA tests in `tests/test_advisor_chatbot.py`: shade-before-10 blocked, data-question forces tool, stale-data warns, no-shade-without-reason warns, tool-failure returns insufficient_data
- [x] **guardrail-unit-tests** – Unit tests for QueryClassifier, ResponseValidator, ConfidenceEstimation, SourceTagging (42 tests total, all passing)
- [x] **qa-tests-expand** – Expanded to 21 adversarial prompts (QA1-QA21) covering May shading, dormancy, night queries, prompt injection, boundary conditions, multi-rule violations, energy/irrigation classification, ambiguous queries. 58 tests total, all passing

#### Step 9: Dual-Channel Advisory
- [x] **dual-channel** – `classify_response_mode()` separates info mode (blue, factual) from advisory mode (amber, recommendations); `response_mode` field on ChatResponse; UI renders mode indicator

#### Step 10: Feedback Loop
- [x] **feedback-ui** – Thumbs up/down/flag buttons on each response in `ui/tab_advisor.py`; `_render_feedback_buttons()` + `_submit_feedback()`
- [x] **feedback-storage** – `src/chatbot/feedback.py` logs feedback to JSONL (query, response, tool_calls, rules, confidence, comment)

---

## Phase 11: Production Migration

Full architecture documented in `context/4_production.md`.
3-layer architecture: Lovable/React frontend (Cloudflare Pages) → FastAPI middleware (HuggingFace Spaces) → existing Python core.
Cron workers via GitHub Actions (free for private repos).

### Phase 11.1: Redis Caching Layer

Replace in-memory TTLCache with Upstash Redis so API and workers share state.

- [x] **upstash-setup** – Upstash Redis instance created (many-mammal-76248.upstash.io), credentials in `.env`, connectivity verified with dual-layer TTLCache round-trip
- [x] **redis-client** – Create `src/data/redis_cache.py`: thin wrapper around Upstash Redis REST API with `get_json()`, `set_json(ttl)`, `delete()`, `exists()`, `ping()`. Singleton via `get_redis()`, returns None when not configured
- [x] **datahub-redis** – Refactored `TTLCache` in `data_providers.py`: tries Redis first (via `redis_prefix`), falls back to in-memory. WeatherService prefix `weather:`, VineSensorService prefix `vine:`. Streamlit works unchanged (no Redis = pure in-memory)
- [x] **tick-result-redis** – `backend/workers/control_tick.py` writes TickResult to Redis `control:last_tick` (20-min TTL) after each tick
- [x] **plan-redis** – `backend/workers/daily_planner.py` writes plan to Redis `control:plan` (24h TTL) + backup `Data/daily_plan.json`
- [x] **budget-redis** – ControlLoop `_ensure_daily_budget()` restores from Redis `control:budget` on startup; `_persist_budget()` saves after every `spend_slot()`. Falls back to recompute if Redis unavailable

### Phase 11.2: FastAPI Middleware (HuggingFace Spaces)

Create the API gateway in `backend/api/`, deploy as HF Space (Docker SDK).

- [x] **fastapi-scaffold** – Created `backend/api/main.py` with FastAPI app, CORS (configurable via `ALLOWED_ORIGINS`), lifespan, all route includes
- [x] **fastapi-deps** – Created `backend/api/deps.py` with `get_datahub()` (lru_cache singleton) and `get_redis_client()`
- [x] **route-weather** – `backend/api/routes/weather.py`: GET `/api/weather/current`, `/api/weather/history?start_date&end_date`, `/api/weather/forecast`
- [x] **route-sensors** – `backend/api/routes/sensors.py`: GET `/api/sensors/snapshot`, `/api/sensors/history?type&area&hours`
- [x] **route-energy** – `backend/api/routes/energy.py`: GET `/api/energy/current`, `/api/energy/daily/{date}`, `/api/energy/history`, `/api/energy/predict/{date}`
- [x] **route-photosynthesis** – `backend/api/routes/photosynthesis.py`: GET `/api/photosynthesis/current?model`, `/api/photosynthesis/forecast`
- [x] **route-control** – `backend/api/routes/control.py`: GET `/api/control/status`, `/api/control/plan`, `/api/control/budget`, `/api/control/trackers`
- [x] **route-chatbot** – `backend/api/routes/chatbot.py`: POST `/api/chatbot/message`, POST `/api/chatbot/feedback` with Pydantic request models
- [x] **route-biology** – `backend/api/routes/biology.py`: GET `/api/biology/phenology`, `/api/biology/rules`, `/api/biology/rules/{name}`
- [x] **rate-limiting** – Added `slowapi`: 60 req/min global default, 10 req/min on `/api/chatbot/message`
- [x] **fastapi-dockerfile** – Created `backend/Dockerfile` (Python 3.12 slim, port 7860, copies src/config/backend/Data)
- [x] **hf-space-readme** – Created `backend/HF_README.md` (sdk: docker, app_port: 7860, private: true, endpoint summary)
- [x] **fastapi-test** – Smoke tested: `/api/health` returns OK (uptime + redis status), `/api/weather/current` returns live IMS data

### Phase 11.3: Background Workers (GitHub Actions)

Extract ControlLoop and DayAheadPlanner into standalone entry points, run as scheduled GitHub Actions.

- [x] **worker-control** – Created `backend/workers/control_tick.py`: loads ControlLoop, runs single tick, writes TickResult to Redis, supports `--dry-run`
- [x] **worker-planner** – Created `backend/workers/daily_planner.py`: runs DayAheadPlanner, writes plan to Redis + JSON backup
- [x] **gh-action-tick** – Created `.github/workflows/control-tick.yml`: schedule `*/15 * * * *`, Python 3.12 + pip cache, all secrets
- [x] **gh-action-planner** – Created `.github/workflows/daily-planner.yml`: schedule `0 2 * * *`, Python 3.12 + pip cache, all secrets
- [x] **gh-secrets** – All 7 secrets added to GitHub repo Settings → Actions secrets
- [x] **worker-logging** – Structured JSON logging (asctime + level + name + message) in both workers
- [x] **worker-test** – Dry-run tested: `python -m backend.workers.control_tick --dry-run` loads plan (43 slots), computes angle (-10.5°), dispatches to 4 trackers (DRY RUN), logs tick result

### Phase 11.4: Lovable Frontend (React + Cloudflare Pages)

Generate and customize the production frontend via Lovable.

- [x] **lovable-project** – Created via Lovable, exported to `solarwine-ai/solarvine-dashboard` repo
- [x] **lovable-dashboard** – Dashboard page: weather card, energy (yesterday/today/tomorrow), phenology, quick stats
- [x] **lovable-advisor** – Advisor page: chat panel with bubbles, advisory cards, confidence/channel badges, feedback
- [x] **lovable-control** – Control page: tracker bars, energy budget, last tick, plan timeline
- [x] **lovable-photosynthesis** – Photosynthesis page: FvCB vs ML, forecast chart, sensor snapshot
- [x] **lovable-shading** – Shading page: offset slider, 3-zone canopy, stress heatmap
- [x] **lovable-docs** – Docs page: biology rules accordion, architecture cards, API reference (18 endpoints)
- [x] **lovable-export** – Imported into `frontend/` directory in main repo
- [x] **api-hooks** – 7 hooks (weather, energy, sensors/PS, control, chatbot, auth) + `api.ts` + `types/api.ts`
- [x] **api-integration** – All 6 pages wired to real API: mock data replaced with live TanStack Query hooks, field names mapped to backend snake_case responses
- [x] **chatbot-component** – Real mutations (useSendMessage/useSendFeedback), confidence badges, source citations, caveats, dual-channel (INFO/ADVISORY), auto-scroll
- [x] **responsive-polish** – Sidebar hamburger already implemented (lg breakpoint). Fixed: Power chart responsive height, Shading heatmap mobile min-width, Home grid breakpoints, camera/map sizing

### Phase 11.5: Deployment & Integration

Wire everything together and deploy.

- [x] **frontend-hosting** – Switched from Vercel (requires Pro for private org repos) to **Cloudflare Pages** (free, unlimited bandwidth, private org repos supported). Connect GitHub repo → root `frontend/`, build `npm run build`, output `dist`, set `VITE_API_URL` env var
- [x] **hf-deploy** – HF Space `SolarWine/api` deployed (Docker SDK, public), secrets injected via huggingface_hub, `/api/health` returns OK with Redis connected. 14/17 endpoints passing; 3 fixed (phenology missing arg, photosynthesis missing method, sensors null area)
- [ ] **upstash-connect** – Verify Redis connectivity from HF Space and GitHub Actions, test cache read/write round-trip
- [x] **e2e-smoke** – Created `scripts/smoke_test.py`: tests 12 endpoints (health, weather, sensors, energy, PS, control, biology, chatbot). Run: `python scripts/smoke_test.py [url]`
- [x] **streamlit-deprecate** – Added migration banner to `app.py`; shows only when `SOLARWINE_FRONTEND_URL` env var is set (invisible until new frontend is live)
- [ ] **dns-custom** – (Optional) Configure custom domain on Cloudflare Pages if available

### Phase 11.6: Security & Monitoring (Post-Launch)

- [x] **auth-jwt** – Created `backend/api/auth.py` (create_token, require_auth, optional_auth) + `backend/api/routes/login.py` (POST `/api/auth/login`). Auth returns `role: "guest"` when JWT_SECRET not set (read-only). PyJWT added to backend/requirements.txt
- [x] **api-hardening** – Standardized error handling (HTTPException everywhere, no JSONResponse for errors), data provider error dict detection, startup path validation, Sentry fail-loud, chatbot session_id validation + blank message guard + init failure circuit breaker, Redis singleton retry-storm fix, CORS defaults include HF Space + Cloudflare Pages URLs
- [x] **auth-frontend** – Created `useAuth.ts` hook (login/logout/token restore from localStorage) + updated `api.ts` to auto-attach JWT Bearer header + auto-clear on 401
- [ ] **monitoring-health** – UptimeRobot or similar free monitor on HF Space `/api/health` (5-min interval, email alerts)
- [x] **monitoring-logs** – Structured logging (asctime/level/name/message) in `main.py` + request logging middleware (method, path, status, duration). Health endpoint extended with ThingsBoard/IMS/Gemini connectivity checks
- [x] **error-tracking** – Added Sentry integration to `backend/api/main.py` (optional, enabled via `SENTRY_DSN` env var). `sentry-sdk[fastapi]>=2.0` in backend/requirements.txt
- [x] **budget-alerts** – `control_tick.py` checks budget after each tick; logs warning if >80% spent before 14:00 IST; sends webhook to `BUDGET_ALERT_WEBHOOK` if configured

---

## Phase 12: Robustness & Efficiency

### 12.1: Immediate (Week 1)

- [x] **tb-health-fix** – Fixed ThingsBoard health check: `/api/noauth/health` returns 404, switched to root URL. Needs HF Space redeploy
- [x] **hf-redeploy** – Redeployed HF Space via `huggingface_hub` API (git push blocked by binary file policy). Uploaded 8 files, space auto-rebuilt
- [x] **redis-verify** – Smoke test against live HF URL: 11/12 pass (control plan 404 expected — no cron run yet). Redis connected, IMS/Gemini configured
- [x] **weather-fix** – Fixed WeatherService crash: Redis deserialised DataFrame as dict/list → `.empty` AttributeError. Added type guard + skip DataFrame Redis serialisation
- [x] **uptime-monitor** – UptimeRobot configured on `/api/health` (5-min interval). Added HEAD method support to health endpoint for monitoring compatibility
- [x] **monitoring-page** – Redesigned Alarms page → Monitoring & Alarms: overall status banner, 3 data source cards (IMS/TB/Energy), backend services panel (Redis/TB/IMS/Gemini), data flow issues surface as system alarms. Sidebar renamed Alarms → Monitoring
- [x] **code-review-fixes** – Fixed email alerter string formatting bug (CRITICAL), duplicate className in Home.tsx, DataFlowStatus null safety, energy price consistency (0.162 USD/kWh), table accessibility (scope attrs), negative age logging
- [x] **data-flow-monitor** – New `/api/health/data-sources` endpoint returning per-source green/yellow/red status (IMS weather, TB sensors, TB energy) with age tracking and configurable thresholds. `DataFlowMonitor` in `backend/services/data_flow_monitor.py`. Thresholds in `config/settings.py`: IMS yellow 60m/red 180m, TB yellow 15m/red 60m
- [x] **data-flow-alerts** – `EmailAlerter` in `backend/services/email_alerter.py`: SMTP email alerts when sources go red, per-source cooldown (60 min). Activated by `SMTP_HOST` + `ALERT_EMAIL_TO` env vars. Background loop in `main.py` checks every 5 min, pushes SSE `health` event
- [x] **data-flow-frontend** – `DataFlowStatus` component on Home page: colored dots (green/yellow/red) for each source, hover tooltips, auto-refresh via SSE. Hook: `useDataFlowStatus`, types: `DataSourceStatus` / `DataFlowStatusResponse`
- [x] **chatbot-range-validation** – `validate_numeric_ranges()` in guardrails.py: checks tool results against physical bounds (temp, GHI, PAR, VPD, CO2, power, etc.), flags sensor faults as caveats
- [x] **chatbot-cross-source** – `check_cross_source_consistency()` in guardrails.py: compares IMS vs TB temperature readings, warns when they diverge by >5°C
- [x] **chatbot-rag-scoring** – Improved `retrieve_relevant_rules()`: weighted scoring (exact match +2, partial word overlap +0.5), increased max_rules to 6

### 12.2: Short-term (Week 2–4)

- [x] **responsive-polish** – Sidebar hamburger already implemented (lg breakpoint). Fixed: Power chart responsive height (280/350/450px), Shading heatmap mobile min-width (400px), Home grid md breakpoint for map/camera/power row, camera/map responsive min-heights
- [x] **load-test** – `scripts/load_test.py` (Locust): 16 weighted endpoints, 10 users × 1 min = 157 requests, p50=250ms, p95=1700ms, 0 errors. All endpoints healthy except `/api/health/data-sources` (not yet deployed at test time)
- [x] **circuit-breaker** – Added `CircuitBreaker` class (3 failures in 60s → open for 5 min) to VineSensorService and EnergyService. Prevents retry storms when TB is down
- [x] **tracker-timeout** – Added exponential backoff retry (3 attempts: 1s, 2s) for TB shared attribute writes in TrackerDispatcher

### 12.2b: Code quality improvements (done 2026-03-29)

- [x] **code-review-critical** – Fixed: Redis health try-catch, chatbot init race condition (lock before null check), background task try-finally, daily planner IST timezone, JWT login parse safety, error boundary on Outlet
- [x] **error-standardization** – Extracted `check_service_error()` to `backend/api/utils.py`, used by energy + sensors routes. Fixed energy route ignoring errors when `daily_kwh` present
- [x] **control-auth** – Added `optional_auth` to all `/control/*` endpoints (status, plan, budget)
- [x] **farquhar-sigmoid** – Replaced hard 30°C RuBP→Rubisco cutoff with smooth sigmoid (28–32°C). New "Transition" state. Prevents flip-flopping near threshold
- [x] **divergence-fix** – Fixed inverted logic in `control_loop.py:704` divergence check
- [x] **frontend-extract** – Extracted `StatCard` to `/components/StatCard.tsx`, date formatters to `/lib/dateUtils.ts`, energy price to `/config/constants.ts`. Home.tsx reduced by ~50 lines
- [x] **strict-types** – Added TypeScript interfaces: `ChillUnitResponse`, `SoilMoistureRow`, `RainRow`, `TrackerInfo`, `ControlTrackers`, `ControlBudget`. Replaced inline `any` types
- [x] **chart-memoize** – `useMemo` on tempData chart transformation in Home.tsx

### 12.2c: Architecture refactors (done 2026-03-29)

- [x] **vectorize-farquhar** – `compute_all()` rewritten with vectorized pandas/numpy (no iterrows). 10K rows: 7.5ms (was ~750ms). Numerically identical output (max diff=0)
- [x] **control-loop-di** – `ControlLoop.__init__()` accepts all 8 dependencies as keyword args. Pass None (default) for lazy auto-create; pass mocks for testing
- [x] **gate-pipeline** – `InterventionGate.evaluate()` split into 5 methods (`_check_meaningful_sun`, `_check_heat_stress`, `_check_water_stress`, `_check_radiation_load`, `_check_biology`) composed as pipeline. First rejection short-circuits
- [x] **budget-audit** – `src/budget_audit.py`: `BudgetAuditLog` appends one parquet row per control tick. `daily_summary(date)` and `weekly_report()` for compliance verification. Wired into `control_loop.py` tick step 12. CLI: `python -m src.budget_audit --report`
- [x] **chatbot-hebrew** – Added 18 Hebrew keywords to query classifier. System prompt: language matching, phenology-first, conciseness. IMS stale → auto-supplement with TB sensors. Confidence rule override for dormancy/blocks
- [x] **refactor-tests** – 23 new tests in `tests/test_refactors.py`: vectorized vs scalar (4), sigmoid transition (2), gate pipeline (6), ControlLoop DI (3), budget audit (3), Hebrew classification (3), confidence override (2)

### 12.2e: Data + energy-model refresh — done 2026-05-18

- [x] **ims-refresh** – Pulled IMS station 43 from 2025-12-01 → 2026-05-10 (10,748 rows, 15-min). API has ~8-day lag; data ends 2026-05-10.
- [x] **scripts-refresh-energy-data** – New `scripts/refresh_energy_data.py`: joins TB Plant `production` (1h SUM agg) with IMS hourly weather, reindexes to contiguous hours (nighttime = 0 kWh), adds time + solar-geometry features, writes `Data/energy_weather_merged.csv` (3,477 rows × 18 cols).
- [x] **scripts-train-energy** – New `scripts/train_energy_predictor.py`: temporal hold-out (last 14 days), sklearn `GradientBoostingRegressor` + `LinearRegression` fallback (xgboost requires libomp on macOS; not installed). Bundle slot named `xgb_model` for backward compat with `EnergyPredictor.__init__`.
- [x] **energy-backtest-fix** – `EnergyPredictor.backtest()` now reads weather from `IMSClient` instead of the dead Air1 device.
- [x] **energy-model-validation** – 1,343 train rows / 337 hold-out rows. Hourly MAE = 4.5 kWh. **Daily mean signed error = +50%** — model systematically over-predicts in late April/May. Cause: model has no `tracker_angle` feature, so it can't see the agrivoltaic interventions that actively detune trackers as growing season starts. See 12.5 below for the planned fix.
- [x] **energy-tracker-feature-attempted** – Wired `tracker_angle_mean` + `tracker_angle_std` + `abs_tracker_angle` from TB into the refresh pipeline (`scripts/refresh_energy_data.py:fetch_tb_tracker_angles`). Two variants tried: (a) with imputation of pre-Feb history via hour-of-day medians → bias worsened to **+57%**; (b) without imputation + dropped `sin_elevation` → bias worsened to **+75%**. **Root cause is distribution shift, not features**: training tracker_angle avg = -15° (mild), hold-out avg = -51° (aggressive interventions). The model can't extrapolate operational regimes it didn't see in training. Reverted features to the original best config. Tracker telemetry is still collected into the CSV (columns `tracker_angle_mean`/`_std`) — ready to be consumed once 2026 growing-season data accumulates. See 12.6 / `bigml-tracker-feature` for the proper fix path.

### 12.2d: 2026 sensor migration — done 2026-05-18

- [x] **tb-2026-registry** – Clean-replaced `DEVICE_REGISTRY` in `src/data/thingsboard_client.py` with the 2026 fleet (35 entries: 19 Crop_2Soil + 12 Thermocouples + 4 Trackers). Legacy `Air1-4`, `Crop1-7`, `Soil*`, `Irrigation1`, `Thermocouples1-2` removed. Added `DeviceInfo.position` field (north/south/center/etc.). Convenience subsets `TREATMENT_DEVICES`/`REFERENCE_DEVICES`/`CROP_2SOIL_DEVICES`/`THERMOCOUPLE_DEVICES` derived at import. Treatment = rows 501/502/503/504/509; Reference = row 202.
- [x] **vinesnapshot-2026** – Rewrote `VineSnapshot` with 61 fields. Removed PAR/DLI/VPD/CO2/Irrigation (no 2026 sensor); added NDVI/PRI/PSRI/SIPI/GCI/LCI/DUVI ratios, dual soil profile (shallow + deep moisture/temp/pore-water EC), `panel_temp_active_count`. Legacy field names kept as None-valued compat shims so downstream consumers don't crash. Per-position keys use `{row}-{cardinal}` form (e.g. `"502-south-east"`).
- [x] **vinesnapshot-aggregation** – `get_vine_snapshot()` no longer hard-codes device names; iterates `DEVICE_REGISTRY` filtered by `area`. Full fleet fetched in 1.1 s via `ThreadPoolExecutor`.
- [x] **bounds-fixes** – Added physical bounds for PSRI/SIPI/GCI/LCI/DUVI/leaf-wetness in `_BOUNDS`. **Critical fix**: GCI was being clipped to `[-1, 1]` (real range 0..30), causing all daytime values to be discarded. Split `treatment_soil_pore_water_ec` into shallow + `_deep`. De-duped the `treatment_pri` computation.

### 12.3: Medium-term (Month 1–2)

- [ ] **chatbot-feedback** – Review first 20 thumbs-down/flag submissions, identify top 3 guardrail failures, tune rules
- [ ] **model-versioning** – Create `model_registry/` to track Chronos/ML/Farquhar parameter versions, enable A/B testing
- [ ] **phenology-validation** – Cross-reference GDD-based stage estimates vs camera observations, refine thresholds for Sde Boker

### 12.5: 2026 sensor follow-ups (queued after the foundational registry rewrite)

Bugs and design debt surfaced during the May 2026 sensor migration. Foundational
work landed in 12.2d above; these are the deferred items.

**Remaining bugs**

- [ ] **panel-tc-semantics** – `panel_temp_active_count` is per-device but the panel-temp avg is per-channel — inconsistent. Pick one. (`src/data/thingsboard_client.py:get_vine_snapshot`)
- [x] **tab-system-status-2026** – `ui/tab_system_status.py` `_render_fruiting_zone` rewritten to use NDVI / PRI / leaf-air ΔT / PSRI; risk assessment now triggers on ΔT > 3°C and PSRI > 0.2 in addition to canopy temp. Done 2026-05-18.
- [ ] **frontend-types-2026** – `frontend/src/types/api.ts` and the React hooks (`useSensors`, etc.) still declare `treatment_par_umol: number | null` and similar. After the migration these are always null — type narrows to `null`. Audit `frontend/src/lib/` + `frontend/src/hooks/` and remove or mark deprecated.

**Architectural concerns (numbered to match the post-migration code review)**

- [ ] **2026-A-auto-sync-registry** – Hardcoded `DEVICE_REGISTRY` is brittle: each TB change requires a code edit + redeploy. Build `scripts/sync_devices.py` that reads the `Crop 2Soil 2026` + `Thermocouples 2026` entity groups, writes `Data/devices.json`, and have `thingsboard_client.py` prefer that file with the hardcoded dict as offline fallback. Position decoding belongs in this sync script (regex over the Hebrew TB labels: `צפון`→north, `דרום`→south, `מזרח`→east, `מערב`→west, `מרכז`→center).
- [ ] **2026-B-vinesnapshot-nest** – `VineSnapshot` is 61 flat fields and growing. Nest as `snapshot.treatment.{air,canopy,soil_shallow,soil_deep}` and `snapshot.panel`, expose flat `to_dict()` only at the JSON boundary. Defer until you're already breaking the API for another reason.
- [ ] **2026-C-schema-version** – Compat shims (`treatment_par_umol=None` etc.) silently say "sensor down" when the truth is "this metric no longer exists." Add `schema_version: "2026.1"` to the snapshot and let consumers branch / error explicitly; remove the shims one release later.
- [ ] **2026-D-split-fetch-aggregate** – `get_vine_snapshot()` mixes ~80 lines of I/O with ~70 lines of transformation. Split into `_fetch_all_latest(plan) → raw_dict` + `_aggregate_snapshot(raw, now) → VineSnapshot`. Then unit-test the aggregate step with fixture dicts (currently no test covers it).
- [ ] **2026-E-position-source-of-truth** – Position labels were decoded by hand from Hebrew TB labels — not reproducible. After 2026-A lands, decode at sync time. Until then, store the original TB label as `DeviceInfo.tb_label_hebrew` so the human-readable source survives.
- [ ] **2026-F-per-device-freshness** – `DataFlowMonitor` checks aggregate "newest TB timestamp" — one fast-reporting device makes the fleet look healthy even if 11 thermocouples are silent. Move to per-device staleness with grace per device-type (Crop_2Soil ~15min, Thermocouples up to ~60min due to deep-sleep cycles).
- [x] **2026-G-layout-regen** – `scripts/import_layout.py` updated to handle 2026 rows (202 reference; 501/502/503/504/509 treatment) and the `DeviceInfo.position` field via `_POSITION_OFFSETS`. `Data/layout.json` regenerated: 6 rows, 35 devices, each with `position_m: [x, y, z]`. Done 2026-05-18.
- [ ] **2026-H-position-taxonomy** – Position strings (`"south-east"`, `"center-east"`, …) are free-form. Convert to a `CardinalPosition` enum in `src/data/data_schema.py` (or document the canonical form in CLAUDE.md) so future labels match.
- [ ] **2026-I-aggregation-tests** – No unit test covers the new aggregation logic. Add at least 3 fixture-based tests in `tests/test_thingsboard_client.py`: full fleet, 1 device per area, all-empty. Depends on 2026-D.

**Follow-on integration work**

- [ ] **2026-J-ims-ambient-bridge** – Wire IMS Sde Boker station 43 → `VineSnapshot.ambient_*` fields so they stop returning None. Currently `Alarms.tsx` and `Home.tsx` fall through to `treatment_air_temp_c`, which is misleading (it's not ambient).
- [ ] **2026-K-chatbot-spectral-tools** – Add chatbot tools `get_vegetation_indices()` and `get_canopy_health()` exposing the new 2026-only data (NDVI/PRI/PSRI/SIPI per row + position).
- [ ] **2026-L-spectrometer-page** – New frontend page (or extend `Agro.tsx`): vegetation-index time series, NDVI/SIPI heatmap across the 17 active Crop_2Soil devices, panel surface-temp 4×12 grid from the thermocouples.

### 12.6: "Big" — Stage 2 photosynthesis ML retrain on 2026 schema

Plan, not yet executed. The Stage 2 ML predictor (`src/forecasting/predictor.py`)
forecasts A_n (photosynthesis rate) from weather features. Currently trained on
`Data/Seymour/sensors_wide.csv` (frozen at 2026-02-01) — features include PAR,
airTemperature, CO2, VPD, leafTemperature from the legacy Air1/Crop fleet.
The 2026 sensor refresh deprecated all of those sensors. The new fleet emits
NDVI/PRI/PSRI/spectrometer bands + dual-soil profile + IRT ambient — a
fundamentally different feature space. This is a redesign, not a refresh.

**Phase 1 — Data collection pipeline (2-3 days work)**
- [x] **bigml-collection** – `scripts/collect_2026_training_data.py` written
  and smoke-tested 2026-05-18. Pulls 15-min telemetry from the 10 treatment +
  7 reference Crop_2Soil devices via TB, averages per area per timestep,
  inner-joins with IMS 15-min weather, writes to
  `Data/2026/sensor_history.parquet`. Append-mode dedupes on
  (timestamp, area). May 1–10 sample: **1,896 rows** (948 per area).
- [x] **bigml-stage1-2026** – Farquhar inputs derived from the 2026 fleet:
  PAR ≈ 2.0 × GHI (Akitsu et al. 2017, ±10 %), VPD via Tetens from IMS
  RH + Tair, CO2 = 420 ppm assumed, CWSI = (Tleaf − Tair)/15 clipped to
  [0, 1]. `compute_farquhar_a()` runs `FarquharModel.calc_photosynthesis()`
  row-by-row; result column `a_farquhar_umol`. Validation: daytime mean
  13.1 µmol CO₂/m²/s, peak 19.6 µmol — squarely in Greer & Weedon's
  empirical Semillon range for the leaf-temp window seen.
- [x] **bigml-collection-shading-correction** – `apply_shading_correction()`
  added to `scripts/collect_2026_training_data.py`. For each treatment-area
  timestep: pulls `tracker_angle_mean` (or astronomical fallback when telemetry
  missing), runs `ShadowModel.project_shadow()` + `compute_par_distribution()`,
  takes the fruiting-zone mean PAR, and multiplies the IMS-derived
  `par_umol_derived` by that factor. Reference rows untouched. May 1–10 sample:
  treatment PAR factor distribution is 0.10–1.0 (median 0.82, mean 0.73) —
  panels meaningfully reduce fruiting-zone PAR. Farquhar A label is recomputed
  after the correction. Done 2026-05-18.
- [ ] **bigml-baseline** – Until enough 2026 data accumulates (≥3 months
  growing-season hours), run Farquhar-only as the photosynthesis estimate
  (skip Stage 2 ML for 2026). The chatbot's photosynthesis tool already
  supports this via `PhotosynthesisService.compute_fvcb()`.
- [ ] **bigml-collection-cron** – Schedule
  `scripts.collect_2026_training_data` to run weekly via GitHub Actions
  (similar pattern to `control-tick.yml` / `daily-planner.yml`). Window =
  last 8 days (overlaps to be safe against IMS's ~8-day publication lag).

**Phase 2 — Feature engineering (1-2 days)**
- [ ] **bigml-features** – Define the 2026 ML feature set. Candidates:
  IMS GHI/Tair/RH/wind (forecastable), Crop_2Soil NDVI/PRI/PSRI/SIPI (now),
  soilMoisture (shallow + deep), leafTemperature, tracker angle (current),
  hour, doy_sin/cos. Decide whether to keep the per-device dimension (input
  = 17 device readings per timestep) or aggregate to area-level (~5 features).
- [ ] **bigml-spectral-pca** – With 17 spectrometer bands per device × 17
  devices = 289 columns, dimensionality reduction is needed. Run PCA on the
  spectral subset and keep the top 5-8 components.

**Phase 3 — Training + validation (1 day)**
- [ ] **bigml-train** – Train RF / GBR / Linear regressors (analogous to
  Stage 2 v1). Use temporal hold-out from the latest 14 days of the
  collected 2026 data. Compare MAE vs Farquhar-only baseline.
- [ ] **bigml-tracker-feature** – Add `tracker_angle` and
  `intervention_active` as features to the **energy predictor too** (fixes
  the +50% over-prediction bias seen in 12.2e). Requires pulling tracker
  telemetry alongside production into the energy refresh script.
- [ ] **bigml-deploy** – Wire the trained model into
  `PhotosynthesisService.predict_ml()`, replacing the current
  Stage 2 v1 model. Update model_versioning in `model_registry/`
  (Phase 12.3 — `model-versioning`).

**Dependencies / open questions**
- Need ≥3 months of 2026 growing-season data before Phase 3 is statistically
  meaningful. With data collection starting May 2026, retraining is realistic
  for **August–September 2026**.
- CO2 atmospheric assumption (420 ppm) is conservative but the lack of a
  real sensor is a real degradation vs the legacy fleet. Consider sourcing
  hourly CO2 from a nearby Mauna Loa proxy if precision matters.
- Tracker-angle feature needs ThingsBoard telemetry alignment — trackers
  emit `angle` as shared-attribute, not telemetry by default. Verify the
  control loop is writing telemetry to TB.

### 12.7: Frontend redesign — Claude Design handoff (done 2026-05-18)

- [x] **redesign-foundation** — Replaced `:root` tokens with Negev sand + vine green + sun gold; loaded Source Serif 4 + IBM Plex Sans + IBM Plex Mono; added brand stripe motif + masked-PNG wordmark; re-targeted shadcn aliases. `tailwind.config.ts` extended with serif/mono families and brand color tokens.
- [x] **redesign-layout** — Rewrote `AppSidebar` to a 240 px sand sidebar with 4-section nav (Vineyard / Energy / Operations / Knowledge), masked logo, brand stripes, vine left-bar on active. New `Topbar.tsx` (breadcrumb · ⌘K search · refresh · bell · avatar). New `PageHeader.tsx` (eyebrow + serif H1 with optional italic emphasis + deck + 3-row meta panel). `AppLayout` adds Topbar + footer hairline strip.
- [x] **redesign-shared-components** — New `Sparkline`, `CardHead`, `DataFlowRail`, `TrackerArc`, `PlanTimeline` (96-cell ribbon), `StressHeatmap`. `StatCard` rewritten with eyebrow + delta + serif tabular number + caption + Sparkline. New chart wrappers `charts/SwAreaChart.tsx` and `charts/SwLineChart.tsx` with shared SolarWine defaults (hairline grid, mono ticks, brand palette, dashed reference lines).
- [x] **redesign-home** — Editorial header, data-flow rail, 4-stat row, 96-slot plan ribbon + power chart, tracker fleet with arc dials + status pills, FvCB-vs-ML photosynthesis card, stress heatmap, **dark Advisor card** with sun-gold "Apply at next tick" CTA, 7-day climate w/ 32°C dashed line, chill-accumulation bars.
- [x] **redesign-pages-all** — All 10 remaining pages rewritten:
  - **Agronomy** — 4-stat row + A1–A4 block table w/ vigor bars + soil profile + agronomy log + water budget
  - **Photosynthesis** — model-comparison big numbers + Chronos forecast + sensor explorer
  - **Shading** — 64 px serif offset slider w/ vine→sun gradient, 3-zone canopy diagram, stress×offset map, 8-rule policy grid
  - **Power** — 4-cell KPI strip + today-vs-yesterday-vs-astro line, inverter cards, weekly bars, revenue ledger
  - **Trackers** — fleet table w/ arc dials + status pills + 24h sparklines, dispatch log, reliability fields
  - **Control loop** — 5-stage pipeline with active highlight, L0..L4 priority stack, hierarchical energy budget, plan ribbon
  - **Advisor** — chat surface w/ confidence pills + citations + feedback, dark day-ahead brief, active recommendations, quick-asks
  - **Monitoring** (was Alarms) — alert stream w/ severity pills + ack, 6-source health column
  - **Documentation** — numbered serif accordion (10 biology rules), API ref table (21 endpoints), 12-term glossary
  - **Research** (new) — vineyard-photo hero band, 3 paper cards, datasets table, 7-person crew strip. `/research` route added.
- [x] **redesign-build** — TypeScript clean, vite build 2.5 s, 240 KB gzipped JS. Dev server at http://localhost:8080/.

**TODO markers inline (would benefit from new backend endpoints)**
- [ ] **redesign-stress-grid** — `/api/photosynthesis/stress-grid?date=...` for live 4×24 CWSI grid (used by Home + Shading; currently synthetic).
- [ ] **redesign-inverters** — `/api/energy/inverters` for per-inverter live state on Power.
- [ ] **redesign-dispatch-stats** — `/api/control/dispatches` summary for p95 ack latency on Trackers.
- [ ] **redesign-events-dispatch** — surface dispatch + alert events through `/api/events/*` so the Trackers log + Monitoring stream are live, not seeded.
- [ ] **redesign-frontend-types-purge** — `frontend/src/types/api.ts` and hooks still declare `treatment_par_umol: number | null` etc. After the 2026 fleet migration these are always null; trim or mark deprecated.

### 12.8: Streamlit staleness audit (done 2026-05-18)

- [x] **streamlit-control-replay-freshness** — `ui/tab_system_status.py:_render_control_replay`: empty-state message now uses today's date; loaded log shows a freshness warning when older than 30 days; caption surfaces the simulated date range (sim_2026-05-01 → 2026-05-10).
- [x] **scripts-sim-defaults** — `scripts/run_control_simulation.py` defaults changed from `2025-07-01 / 2025-07-07` to `today − 7 d / today` so future runs are fresh.
- [x] **fresh-sim-log** — Generated `Data/simulation_logs/sim_2026-05-01_2026-05-10.json` (960 ticks). Required installing `astral` into the venv (was missing; fixed 960/960 errors on the first attempt).
- [x] **streamlit-shading-caption** — Added "historical analysis · `sensors_wide.csv` frozen at Feb 2026" caption to the Shading tab so the date picker isn't read as live.
- [x] **camera-caption** — Camera image caption now says "if the view shows only sky, the on-site camera needs realignment". Physical site issue; can't be fixed from code.

### 12.4: Long-term (3+ months)

- [ ] **multi-vineyard** – Refactor `config/settings.py` to multi-tenant pattern (vineyard ID → settings), add Postgres (Neon free tier)
- [ ] **licor-calibration** – Ingest first real LiCor measurements, retrain Farquhar Vcmax/Jmax via `src/calibration_pipeline.py`
- [ ] **chronos-vs-ml** – 3-month live comparison of Chronos-2 vs ML ensemble, promote winner to primary predictor

---

## Session notes

### 2026-02-27 — Phase 8B: Gemini LLM Data Engineering
- Created `src/llm_data_engineer.py` with `LLMDataEngineer` class.
- `analyze_anomalies()`: constructs a structured prompt from per-column descriptive stats + site-specific domain context (physical ranges, Negev notes) → Gemini returns a JSON with `lower_bound`, `upper_bound`, `zscore_threshold`, `iqr_multiplier`, `rationale` per column; falls back to physics-based bounds when API unavailable.
- `apply_cleaning()`: applies union of bound violations + Z-score + IQR anomaly flags; three strategies: `clip` (default), `nan`, `drop`. Detected 14 PAR anomalies (0.05%), 9 VPD anomalies (0.03%), 168 CO₂ anomalies (0.58%) on `sensors_wide_sample.csv`.
- `engineer_features()`: adds `hour_sin`, `hour_cos`, `doy_sin`, `doy_cos` (cyclical encodings); `stress_risk_score = vpd_weight × norm(VPD) + cwsi_weight × norm(CWSI)` in [0, 1]; weights sourced from Gemini (default: 0.6/0.4, falls back gracefully without CWSI).
- `run_pipeline()`: single-call orchestration of anomaly detection → cleaning → feature engineering.
- Added `google-genai>=1.0` to `requirements.txt`.

### 2026-02-27 — Dashboard & pvlib
- **Data Explorer:** Intro text now lists all three data sources (Vineyard sensors, Weather station data, **AI Data Engineering**). Third option runs full LLM pipeline in-tab (thresholds, cleaning summary, before/after PAR/VPD, feature spec, stress profile); no cache loader, all precomputed on selection.
- **Presentation tab:** “What’s Next” completed list includes **Gemini LLM data engineering** with pointer to Data Explorer → AI Data Engineering. “What each tab shows” updated so Data Explorer bullet mentions AI Data Engineering. Key takeaways add Gemini data engineering (anomaly cleaning + stress/time features).
- **pvlib:** `src/solar_geometry.py` — `compute_tracker_tilt()` now calls `pvlib.tracking.singleaxis()` with first two args positional (zenith, azimuth) so it works with both pvlib &lt; 0.13.1 (`apparent_azimuth`) and ≥ 0.13.1 (`solar_azimuth`). Fixes “Shadow model demo unavailable: singleaxis() got an unexpected keyword argument 'solar_azimuth'”.

### 2026-03-05 — ThingsBoard live telemetry integration (Step 1)
- Rewrote `src/thingsboard_client.py`: full device registry (22 devices, UUIDs from devices.csv), area classification (treatment rows 501–502 / reference rows 503–504 / ambient Air1), parallel fetch with `ThreadPoolExecutor`, JWT auth with token refresh.
- Added `VineSnapshot` dataclass: aggregated treatment vs reference readings — Air, Crop (per panel position: west-bottom/east-upper/east-bottom/west-upper), Soil, Irrigation, Thermocouples. Includes `to_advisor_text()` and `to_dict()`.
- Added `_bounded_avg()` with `_BOUNDS` dict: physical plausibility filter rejects sensor faults (e.g. Soil5 thermocouple reading 653°C). Soil, Air, PAR, NDVI, VPD, CO₂ all filtered by domain bounds.
- Wired into `src/vineyard_chatbot.py`: new `_tool_get_vine_state()` tool + `get_vine_state` in system prompt + dispatch router; `_tool_day_ahead_advisory()` now passes `VineSnapshot` to advisor.
- Wired into `src/day_ahead_advisor.py`: optional `vine_snapshot` param in `advise()`; `to_advisor_text()` inserted before weather forecast in Gemini prompt.
- TB credentials (`TB_USERNAME`, `TB_PASSWORD`) added to `.env`; config also accepts `THINGSBOARD_USERNAME`/`THINGSBOARD_PASSWORD`.
- Tested live: all 22 devices fetched in 1.1 s, PAR shading ratio 0.33 (67% reduction), soil temp 15.5°C (outlier-filtered).
- Synced Phase 9 checklist: all chatbot tasks marked done (implemented in a prior session, todo file not updated).

### 2026-03-09 — Phase 10: Anti-Hallucination Guardrails
- Created `src/chatbot/guardrails.py` with 4 components: QueryClassifier (regex data/knowledge/greeting routing), ResponseValidator (deterministic rule checks with block/warn severity), estimate_confidence (data freshness → confidence level), tag_tool_result (source metadata injection).
- Rewrote `src/chatbot/vineyard_chatbot.py`: integrated all guardrails into `chat()` flow — query classification → forced tool re-prompting → source-tagged tool results → response validation → confidence estimation. Extended `ChatResponse` with confidence/sources/caveats/rule_violations. Strengthened system prompt: "NEVER invent sensor readings", "MUST call a tool for data questions", "cite source and timestamp".
- Updated `ui/tab_advisor.py`: new `_render_grounding_metadata()` renders colored confidence badge, source citations, caveats, and rule violation warnings under each chat response. Metadata persisted in chat history.
- Added 23 new tests to `tests/test_advisor_chatbot.py`: 5 adversarial QA scenarios (shade-before-10 blocked, data-question forces tool call, stale-data warns, no-shade-without-reason warns, tool-failure returns insufficient_data) + unit tests for all guardrail components. Total: 42 tests, all passing.
- Updated `context/2_plan.md` with Phase 10 architecture, validation rules, confidence mapping, and remaining steps. Updated dependency order.

### 2026-02-17 — Code quality fixes
- Fixed `Preprocessor.merge_ims_with_labels` position-based fallback: now raises `ValueError` instead of silently misaligning rows when `timestamp_index_labels=False` and labels have no named index.
- Removed all ~50 deprecated `use_container_width=True` calls in `app.py` (Streamlit 1.54): `st.image` → `width='stretch'`; `st.plotly_chart` / `st.dataframe` → argument removed (default is already `'stretch'`).
- Fixed follow-up `SyntaxError`: regex removal of `use_container_width=True` in the middle of argument lists left double commas (`,,`); collapsed all `,,` → `,`.
- Updated `context/CODE_REVIEW.md` sections 16–17 with applied fixes and remaining work list.
- Updated "What's next" sections in `app.py` (tab_photosynthesis section 6 and About tab) to reflect Chronos-2 and Gemini routing as completed.