# SolarWine 2.0 — Implementation Plan
## Executive Summary
Evolve the existing two-stage photosynthesis prediction pipeline into a biology-driven agrivoltaic control system. The core principle: **full astronomical tracking is the default — the panels follow the sun and generate maximum energy 100% of the time**. Anti-tracking (tilting away from the sun to shade the vine) is a rare, targeted intervention used ONLY when the vine's fruiting zone is at measurable risk of heat or radiation damage, and ONLY at the minimum dose necessary to protect yield.
The user sets a hard ceiling on annual energy sacrifice (e.g., 5%). The system distributes that budget across Year → Month → Week → Day → 15-min slots, concentrating almost all of it in July–August midday windows when heat stress actually threatens grape quality. Margins at each level allow the system to respond to unexpected stress events while guaranteeing the energy ceiling is never breached.
---
## Biological Design Principles
These principles, drawn from viticultural research for *Vitis vinifera* in hot-arid climates, constrain every decision the algorithm makes:
### Principle 1: Morning light is sacred — never shade mornings
Morning radiation (sunrise to ~10:00) is the most valuable light the vine receives:
- **Stomata are maximally open** in early morning (low VPD, cool air). This is when gas exchange is most efficient and photosynthetic carbon loading peaks.
- **Spring mornings (May–June)** are critical for flowering, fruit set, and early berry development. Shading during this period directly reduces yield potential.
- The vine "charges" its carbohydrate reserves in the morning to survive afternoon heat stress. Reducing morning PAR weakens the vine's ability to cope later in the day.
**Hard constraint**: Shading is PROHIBITED before 10:00 local time, regardless of any other signal.
### Principle 2: The fruiting zone matters more than the canopy top
In a VSP (Vertical Shoot Positioning) trellis, the grape clusters hang in the **mid-canopy zone** (~0.4–0.8m height, zone index 1 in our 3-zone ShadowModel at center height ~0.6m). This zone determines grape quality:
- **Berry color** (anthocyanins) requires moderate light exposure — but direct midday sun in the Negev causes sunburn (berry surface > 35°C).
- **Sugar accumulation** (Brix) depends on canopy photosynthesis above, not direct fruit irradiance.
- **Flavor compounds** (terpenes, phenolics) develop best under filtered/diffuse light in the fruiting zone, not harsh direct radiation.
**Optimization target**: When intervention IS needed, choose the minimum tilt angle that shades the **fruiting zone** specifically while keeping the **upper canopy** (zone index 2, where 50% of leaf area lives) in full sun for photosynthesis.
### Principle 3: Shade only when it actually helps — minimum effective dose
Shading is not inherently beneficial. Below 30°C (RuBP-limited state), the vine is light-limited and shading HURTS both energy generation AND photosynthesis. The system should only intervene when:
- The vine crosses into **Rubisco-limited state** (Tleaf > 30°C for Semillon)
- CWSI indicates real water stress (> 0.4), meaning stomata are closing due to heat/VPD
- The FvCB model confirms that reducing PAR at the current temperature would actually improve or maintain A (not reduce it)
And the intervention must be the **smallest angle offset** that brings the fruiting zone below the damage threshold. Once the stress signal drops, return immediately to full tracking.
### Principle 4: Most of the season needs zero intervention
In a typical Sde Boker growing season:
- **May**: Canopy developing, temps moderate (25–32°C). Almost no stress events. Budget needed: ~0%.
- **June**: Temps rising, occasional heat spikes. Rare intervention. Budget needed: ~5%.
- **July**: Peak heat (35–42°C), daily midday stress. This is where 40–45% of the budget goes.
- **August**: Sustained heat, fruit ripening. Critical sunburn risk. Budget needed: 40–45%.
- **September**: Declining heat, pre-harvest. Occasional late heat waves. Budget needed: ~10%.
The budget is heavily concentrated in ~4 hours/day (11:00–15:00) during ~8 weeks (mid-July to mid-August). Outside those windows, the tracker runs at full astronomical tracking with zero sacrifice.
---
## What Already Exists (Baseline v1)
| Component | Module | Status |
|---|---|---|
| Farquhar/Greer-Weedon FvCB model | `src/farquhar_model.py` | Done — Vcmax/Jmax temp response, CWSI, stomatal conductance |
| Shadow/solar geometry | `src/solar_geometry.py` | Done — ShadowModel, 3-zone vertical grid (bottom/mid/top), ray-tracing, PAR distribution |
| Canopy-level photosynthesis | `src/canopy_photosynthesis.py` | Done — zone-weighted Farquhar, LAI weights [0.15, 0.35, 0.50] bottom→top |
| Tilt-angle simulation | `src/tracker_optimizer.py` | Done — cosine-factor PAR scaling, stress heatmap |
| IMS weather client | `src/ims_client.py` | Done — fetch/cache/resample IMS station 43 |
| ML predictor (Stage 2) | `src/predictor.py` | Done — LR, DT, RF, GBR, XGBoost |
| Time-series forecasting | `src/ts_predictor.py` | Done — multi-horizon lag-based |
| Preprocessor | `src/preprocessor.py` | Done — merge, time features, temporal split |
| Sensor data loader | `src/sensor_data_loader.py` | Done |
| Streamlit dashboard | `app.py` | Done — 6 tabs, branded UI |
| Site/panel config | `config/settings.py` | Done — panel 1.13m wide at 2.05m height, canopy 1.2m VSP, 3.0m row spacing |
### Key existing geometry (from ShadowModel)
```
Vertical zones (n_vertical=3):
Zone 0 (bottom): center ~0.2m — trunk/cordons — LAI weight 15%
Zone 1 (mid): center ~0.6m — FRUITING ZONE — LAI weight 35%
Zone 2 (top): center ~1.0m — main leaf canopy — LAI weight 50%
Panel: 1.13m wide, mounted at 2.05m, row spacing 3.0m
Canopy: 1.2m tall, 0.6m wide, VSP trellis
```
---
## Gap Analysis
### New Components
1. **EnergyBudgetPlanner** — Hierarchical budget with no-shade windows and fruiting-zone targeting
2. **TradeoffEngine** — Budget-constrained, minimum-dose optimization
3. **Command Arbiter** — Fallback logic, hysteresis, mode priority
4. **Safety Rails** — FvCB vs ML divergence validation
5. **Spectral Aggregator** — CWSI/NDVI/PRI from sensor data
6. **ROI Reporting Service** — Budget utilization, LER
7. **Weather Protection / Harvest Modes** — Override protocols
8. **Data Schema** — Telemetry storage definitions
### Existing Components to Upgrade
1. **FarquharModel** — Add 30°C Semillon transition returning limiting state
2. **ShadowModel** — Add fruiting-zone-specific shadow query
3. **CanopyPhotosynthesisModel** — Expose per-zone A and fruiting zone metrics
4. **Config/Settings** — Add no-shade windows, fruiting zone thresholds, budget parameters
---
## Implementation Phases
### Phase 1: Configuration & Data Infrastructure
#### 1.1 Extend `config/settings.py`
```python
# === ENERGY BUDGET ===
MAX_ENERGY_REDUCTION_PCT = 5.0 # user's hard ceiling (%)
ANNUAL_RESERVE_PCT = 15.0 # emergency reserve from total budget
WEEKLY_RESERVE_PCT = 20.0 # within-week flexibility
DAILY_MARGIN_PCT = 20.0 # real-time response pool
# Monthly budget distribution — concentrated in peak heat months
MONTHLY_BUDGET_WEIGHTS = {
5: 0.00, # May — no intervention, canopy establishing
6: 0.05, # June — rare, only extreme heat spikes
7: 0.45, # July — peak heat, primary shading window
8: 0.40, # August — sustained heat, fruit ripening/sunburn risk
9: 0.10, # Sept — occasional late heat waves
}
# === NO-SHADE WINDOWS (hard constraints — shading PROHIBITED) ===
NO_SHADE_BEFORE_HOUR = 10 # local solar time — morning light is sacred
NO_SHADE_MONTHS = [5] # May — full spring exposure for fruit set
NO_SHADE_GHI_BELOW = 300 # W/m2 — overcast, already diffuse
NO_SHADE_TLEAF_BELOW = 28.0 # C — no heat stress risk
NO_SHADE_CWSI_BELOW = 0.3 # vine is healthy, wants light
# === SHADE-ELIGIBLE CONDITIONS (ALL must be true to allow intervention) ===
SHADE_ELIGIBLE_TLEAF_ABOVE = 30.0 # C — Semillon Rubisco transition
SHADE_ELIGIBLE_CWSI_ABOVE = 0.4 # moderate stress confirmed
SHADE_ELIGIBLE_GHI_ABOVE = 500 # W/m2 — high direct radiation
SHADE_ELIGIBLE_HOURS = (10, 16) # local solar time window
# === FRUITING ZONE ===
FRUITING_ZONE_INDEX = 1 # mid-canopy zone in 3-zone ShadowModel
FRUITING_ZONE_HEIGHT_M = 0.6 # center height of grape cluster zone
BERRY_SUNBURN_TEMP_C = 35.0 # berry surface temperature damage threshold
FRUITING_ZONE_TARGET_PAR = 400 # umol/m2/s — sufficient for quality, below burn
# === TRADEOFF ENGINE ===
CANDIDATE_OFFSETS = [0, 3, 5, 8, 10, 15, 20] # degrees off astronomical (small steps)
SIMULATION_TIMEOUT_SEC = 5
# === SAFETY RAILS ===
DIVERGENCE_THRESHOLD = 0.12 # 12% FvCB vs ML mismatch triggers fallback
# === SEMILLON FvCB ===
SEMILLON_TRANSITION_TEMP_C = 30.0
# === WEATHER PROTECTION ===
WIND_STOW_SPEED_MS = 15.0
HEAT_SHIELD_TEMP_C = 38.0
HEAT_SHIELD_CWSI = 0.6
# === MECHANICAL HARVESTING ===
HARVEST_PARK_CLEARANCE_CM = 250
HARVEST_LATERAL_WIDTH_CM = 18
HARVESTER_RPM_RANGE = (430, 460)
# === HYSTERESIS ===
HYSTERESIS_WINDOW_MIN = 15
ANGLE_TOLERANCE_DEG = 2.0
# === ROI ===
TARGET_LER = 1.5
```
#### 1.2 Create `src/data_schema.py`
Dataclass definitions for the four telemetry tables (SensorRawTelemetry, BiologicalState, TrackerKinematics, SimulationLog). CSV/Parquet backend for now; schema supports future TimescaleDB migration.
#### 1.3 Create `scripts/import_layout.py`
Spatial initialization: read site layout, assign 3D coordinates to assets, output `layout.json` for ShadowModel.
---
### Phase 2: Biological Digital Twin Upgrades
#### 2.1 Upgrade `src/farquhar_model.py` — Semillon State Transition
```python
def calc_photosynthesis_semillon(self, PAR, Tleaf, CO2, VPD, Tair, CWSI=None):
"""FvCB with explicit 30C Semillon state transition.
Returns (A, limiting_state, shading_helps).
shading_helps is True ONLY when the vine is Rubisco-limited AND
reducing PAR would not reduce A (because the bottleneck is heat, not light).
"""
Tk = Tleaf + 273.15
Kc, Ko = self.calc_Kc(Tk), self.calc_Ko(Tk)
gamma_star = self.calc_gamma_star(Tk)
Vcmax, Jmax = self.calc_Vcmax(Tk), self.calc_Jmax(Tk)
J = self.calc_electron_transport(PAR, Jmax)
Rd = self.params["rd_frac"] * Vcmax
ci = self._ci_from_ca(CO2, VPD, CWSI or 0.0)
Ac = Vcmax * (ci - gamma_star) / (ci + Kc * (1.0 + OI / Ko))
Aj = J * (ci - gamma_star) / (4.0 * ci + 8.0 * gamma_star)
if Tleaf < SEMILLON_TRANSITION_TEMP_C:
# RuBP-limited: light is the bottleneck
# Shading HURTS — reduces J, reduces Aj, reduces A
state = "RuBP_Limited"
shading_helps = False
An = min(Ac, Aj) - Rd # standard min-based
else:
# Rubisco-limited: enzyme/heat is the bottleneck
# Shading helps IF it cools the leaf without cutting PAR below
# the point where Aj drops below Ac
state = "Rubisco_Limited"
shading_helps = (Aj > Ac) # True when light is abundant relative to Vcmax
An = min(Ac, Aj) - Rd
return max(0.0, An), state, shading_helps
```
The third return value `shading_helps` is the gate: the TradeoffEngine will NOT shade unless this is True. Even above 30°C, if PAR is already low (cloudy), Aj may be below Ac and shading would still hurt.
#### 2.2 Upgrade `src/solar_geometry.py` — Fruiting Zone Shadow Query
Add to ShadowModel:
```python
def fruiting_zone_shadow(self, shadow_mask: np.ndarray,
fruiting_zone_idx: int = 1) -> dict:
"""Report shading specifically on the fruiting zone (mid-canopy)."""
fz_row = shadow_mask[fruiting_zone_idx, :] # shape (n_horizontal,)
fz_shaded_fraction = float(fz_row.mean())
return {
"fruiting_zone_shaded_pct": fz_shaded_fraction * 100,
"fruiting_zone_sunlit_pct": (1 - fz_shaded_fraction) * 100,
"fruiting_zone_mask": fz_row,
}
def evaluate_candidate_offsets(self, solar_elevation, solar_azimuth,
theta_astro, offsets, total_par):
"""Evaluate shadow at astronomical angle + each offset.
Returns dict keyed by offset with shadow mask, PAR distribution,
and fruiting zone metrics."""
results = {}
for offset in offsets:
theta = theta_astro + offset
mask = self.project_shadow(solar_elevation, solar_azimuth, theta)
par_dist = self.compute_par_distribution(total_par, mask)
fz = self.fruiting_zone_shadow(mask)
results[offset] = {
"shadow_mask": mask,
"par_distribution": par_dist,
"sunlit_fraction": self.sunlit_fraction(mask),
"fruiting_zone": fz,
}
return results
```
#### 2.3 Upgrade `src/canopy_photosynthesis.py` — Expose Fruiting Zone Metrics
Add to `compute_vine_A()` return dict:
```python
# After computing A_zones:
fruiting_zone_A = float(A_zones[FRUITING_ZONE_INDEX, :].mean())
fruiting_zone_par = float(par_zones[FRUITING_ZONE_INDEX, :].mean())
top_canopy_A = float(A_zones[2, :].mean()) # zone 2 = top, 50% LAI
return {
"A_vine": A_vine,
"A_zones": A_zones,
"sunlit_fraction": sunlit_frac,
"par_zones": par_zones,
"fruiting_zone_A": fruiting_zone_A,
"fruiting_zone_par": fruiting_zone_par,
"top_canopy_A": top_canopy_A,
}
```
#### 2.4 Create `src/safety_rails.py`
FvCB vs ML divergence check. 12% threshold triggers fallback to astronomical tracking (zero sacrifice).
#### 2.5 Create `src/spectral_aggregator.py`
Batch preprocessing of CWSI, NDVI, PRI from sensor columns. Operates as a function, not a service (message bus deferred to production).
---
### Phase 3: The Control Algorithm
**Default behavior: FULL ASTRONOMICAL TRACKING. The system generates maximum energy and only deviates when the vine's fruiting zone is genuinely at risk.**
---
#### 3.0 Algorithm Overview
```
┌──────────────────────────────────────────────────────────────┐
│ EVERY 15 MINUTES │
│ │
│ 1. Compute θ_astro (pvlib single-axis tracking) │
│ 2. DEFAULT DECISION: θ = θ_astro (full tracking, 0 sacrifice)│
│ │
│ 3. CHECK: Is this a no-shade window? │
│ - Before 10:00 local? → STAY at θ_astro. Done. │
│ - May? → STAY at θ_astro. Done. │
│ - Tleaf < 28C? → STAY at θ_astro. Done. │
│ - CWSI < 0.3? → STAY at θ_astro. Done. │
│ - GHI < 300 W/m2? → STAY at θ_astro. Done. │
│ │
│ 4. CHECK: Are shade-eligible conditions ALL met? │
│ - Tleaf > 30C (Rubisco-limited)? │
│ - CWSI > 0.4 (real stress)? │
│ - FvCB says shading_helps = True? │
│ - GHI > 500 W/m2 (high direct radiation)? │
│ - Hour between 10:00 and 16:00? │
│ → If ANY condition is False: STAY at θ_astro. Done. │
│ │
│ 5. All gates passed → INTERVENTION ALLOWED │
│ - Budget available? (slot_budget + margin > 0) │
│ → If no budget: STAY at θ_astro. Done. │
│ │
│ 6. FIND MINIMUM EFFECTIVE DOSE: │
│ For offsets [3, 5, 8, 10, 15, 20] degrees: │
│ - Ray-trace shadow at θ_astro + offset │
│ - Check: does fruiting zone PAR drop below 400 umol? │
│ - Check: does top canopy stay mostly sunlit (>70%)? │
│ - Check: sacrifice(offset) ≤ available budget? │
│ → Select SMALLEST offset that protects fruiting zone │
│ while keeping top canopy productive │
│ │
│ 7. Record sacrifice, update budget ledger. │
└──────────────────────────────────────────────────────────────┘
```
The critical difference from the prior plan: **the utility function does not pick the "best" angle — it picks the SMALLEST angle that achieves the biological target.** Energy sacrifice is minimized, not balanced.
---
#### 3.1 Energy Budget Planner — `src/energy_budget.py`
##### 3.1.1 The Budget Hierarchy
```
USER INPUT: max_energy_reduction_pct = 5%
ANNUAL PLAN (computed at season start)
total_potential = Σ E_astro(slot) for May 1 – Sep 30
total_budget = potential × 5%
annual_reserve = budget × 15% (for heat waves)
distributable = budget × 85%
Monthly allocation (where the budget goes):
May: 0% = 0 kWh ← NO intervention ever in May
June: 5% = small ← rare extreme events only
July: 45% = bulk ← peak heat, daily midday shading
August: 40% = bulk ← sustained heat, sunburn risk
Sept: 10% = moderate ← late heat waves
WEEKLY PLAN (generated Monday or rolling)
weekly_raw = monthly_remaining / weeks_left + rollover
weekly_reserve = weekly_raw × 20%
distributable = weekly_raw × 80%
Daily allocation weighted by forecast Tmax:
weight(day) = max(0, forecast_Tmax - 30)²
Hot days get quadratically more budget.
Days with Tmax < 30: zero allocation (no stress expected).
DAILY PLAN (generated at sunrise)
daily_raw = weekly_allocation + yesterday's rollover
daily_margin = daily_raw × 20% (unexpected afternoon spike)
planned = daily_raw × 80%
Slot allocation:
Before 10:00: ZERO (no-shade window)
10:00-11:00: planned × 0.05 (transition period)
11:00-14:00: planned × 0.60 (peak stress window)
14:00-16:00: planned × 0.30 (sustained heat)
After 16:00: planned × 0.05 (rare late stress)
SLOT EXECUTION (every 15 min)
available = slot_budget + margin_share
→ If intervention triggered AND budget > 0: shade
→ If no intervention needed: budget preserved (rolls forward)
```
##### 3.1.2 Why the Budget Is Small
At 5% annual sacrifice with Sde Boker's solar geometry:
- Annual energy potential (May–Sep) ≈ 750 kWh/kWp equivalent
- Total budget ≈ 37.5 kWh sacrifice allowed
- July gets ≈ 14 kWh → ≈ 0.45 kWh/day → spread across ≈ 3 hours of eligibility
- Each 15-min slot gets ≈ 0.04 kWh — enough for 3–8° offset during peak
This is intentionally tight. The algorithm must be surgical: intervene only when truly necessary, at the minimum dose, and return to full tracking immediately.
##### 3.1.3 Budget Implementation
```python
class EnergyBudgetPlanner:
def __init__(self, max_energy_reduction_pct, shadow_model):
self.max_pct = max_energy_reduction_pct
self.shadow = shadow_model
def compute_annual_plan(self, year: int) -> dict:
"""Compute energy potential under astronomical tracking, allocate budget."""
season_start = pd.Timestamp(f"{year}-05-01", tz="UTC")
season_end = pd.Timestamp(f"{year}-09-30 23:45", tz="UTC")
times = pd.date_range(season_start, season_end, freq="15min")
solar_pos = self.shadow.get_solar_position(times)
energy_per_slot = []
for _, sp in solar_pos.iterrows():
if sp["solar_elevation"] <= 0:
energy_per_slot.append(0.0)
continue
tracker = self.shadow.compute_tracker_tilt(
sp["solar_azimuth"], sp["solar_elevation"])
e = max(0, np.cos(np.radians(tracker["aoi"]))) * 0.25
energy_per_slot.append(e)
total_potential = sum(energy_per_slot)
total_budget = total_potential * self.max_pct / 100
annual_reserve = total_budget * ANNUAL_RESERVE_PCT / 100
distributable = total_budget - annual_reserve
monthly = {}
for month, weight in MONTHLY_BUDGET_WEIGHTS.items():
monthly[month] = distributable * weight
return {
"year": year,
"total_potential_kWh": total_potential,
"total_budget_kWh": total_budget,
"annual_reserve_kWh": annual_reserve,
"monthly_budgets": monthly,
"budget_spent_kWh": 0.0,
}
def compute_weekly_plan(self, week_start, monthly_remaining,
forecast_tmax=None, rollover=0.0):
"""Distribute to days weighted by (Tmax - 30)^2. Days < 30C get zero."""
month = week_start.month
weeks_left = max(1, (pd.Timestamp(
f"{week_start.year}-{month:02d}-30") - week_start).days // 7)
weekly_raw = monthly_remaining / weeks_left + rollover
weekly_reserve = weekly_raw * WEEKLY_RESERVE_PCT / 100
distributable = weekly_raw - weekly_reserve
if forecast_tmax and len(forecast_tmax) == 7:
weights = [max(0, t - 30.0) ** 2 for t in forecast_tmax]
total_w = sum(weights)
if total_w > 0:
daily = [distributable * w / total_w for w in weights]
else:
daily = [0.0] * 7 # all days < 30C → no budget needed
else:
daily = [distributable / 7] * 7
return {
"weekly_total_kWh": weekly_raw,
"weekly_reserve_kWh": weekly_reserve,
"daily_budgets_kWh": daily,
}
def compute_daily_plan(self, date, daily_budget, rollover=0.0):
"""Distribute to 15-min slots. Zero before 10:00. Peak at 11-14."""
daily_raw = daily_budget + rollover
daily_margin = daily_raw * DAILY_MARGIN_PCT / 100
planned = daily_raw - daily_margin
slot_budgets = {}
blocks = [
((5, 10), 0.00), # NO SHADE — morning is sacred
((10, 11), 0.05), # transition
((11, 14), 0.60), # peak stress window
((14, 16), 0.30), # sustained heat
((16, 20), 0.05), # rare late stress
]
for (h_start, h_end), weight in blocks:
block_budget = planned * weight
n_slots = (h_end - h_start) * 4
per_slot = block_budget / n_slots if n_slots > 0 else 0
for h in range(h_start, h_end):
for m in (0, 15, 30, 45):
slot_budgets[f"{h:02d}:{m:02d}"] = per_slot
return {
"date": date,
"daily_total_kWh": daily_raw,
"daily_margin_kWh": daily_margin,
"daily_margin_remaining_kWh": daily_margin,
"slot_budgets": slot_budgets,
"cumulative_spent": 0.0,
}
def emergency_draw(self, annual_plan, amount):
"""Draw from annual reserve for extreme heat events."""
available = annual_plan["annual_reserve_kWh"]
drawn = min(amount, available)
annual_plan["annual_reserve_kWh"] -= drawn
return drawn
```
---
#### 3.2 The Intervention Gate — `src/tradeoff_engine.py`
The gate is a series of hard checks. The engine does NOT run a utility function to decide WHETHER to shade — it runs a strict pass/fail gate. Only after all gates pass does it search for the minimum effective dose.
##### 3.2.1 The Gate Logic
```python
class InterventionGate:
"""Determines whether shading is allowed at this timestep.
Returns True only when ALL conditions for intervention are met.
This is not an optimization — it is a hard pass/fail filter."""
def check(self, timestamp, tleaf, tair, cwsi, ghi, par,
co2, vpd, farquhar_model) -> dict:
local_hour = timestamp.hour # approximate (should use solar noon)
# --- NO-SHADE WINDOWS (any one blocks intervention) ---
if local_hour < NO_SHADE_BEFORE_HOUR:
return {"allowed": False, "reason": "morning_protected"}
if timestamp.month in NO_SHADE_MONTHS:
return {"allowed": False, "reason": "spring_protected"}
if tleaf < NO_SHADE_TLEAF_BELOW:
return {"allowed": False, "reason": "no_heat_stress"}
if cwsi < NO_SHADE_CWSI_BELOW:
return {"allowed": False, "reason": "vine_healthy"}
if ghi < NO_SHADE_GHI_BELOW:
return {"allowed": False, "reason": "overcast"}
# --- SHADE-ELIGIBLE CONDITIONS (all must pass) ---
if tleaf < SHADE_ELIGIBLE_TLEAF_ABOVE:
return {"allowed": False, "reason": "below_rubisco_threshold"}
if cwsi < SHADE_ELIGIBLE_CWSI_ABOVE:
return {"allowed": False, "reason": "stress_insufficient"}
h_start, h_end = SHADE_ELIGIBLE_HOURS
if not (h_start <= local_hour < h_end):
return {"allowed": False, "reason": "outside_eligible_hours"}
if ghi < SHADE_ELIGIBLE_GHI_ABOVE:
return {"allowed": False, "reason": "low_radiation"}
# --- BIOLOGICAL CONFIRMATION ---
_, state, shading_helps = farquhar_model.calc_photosynthesis_semillon(
par, tleaf, co2, vpd, tair
)
if not shading_helps:
return {"allowed": False, "reason": "fvcb_says_shade_hurts",
"state": state}
return {"allowed": True, "reason": "all_gates_passed", "state": state}
```
##### 3.2.2 Minimum Effective Dose Selection
When the gate passes, the engine does NOT maximize a utility function. Instead, it searches for the **smallest offset** that achieves the biological target: bring fruiting zone PAR below the sunburn threshold while keeping the top canopy productive.
```python
class TradeoffEngine:
def __init__(self, shadow_model, canopy_model, farquhar_model):
self.shadow = shadow_model
self.canopy = canopy_model
self.farquhar = farquhar_model
self.gate = InterventionGate()
def evaluate_slot(self, timestamp, par, tleaf, tair, co2, vpd,
cwsi, ghi, slot_budget_kWh, margin_kWh) -> dict:
# 1. Astronomical optimal (always computed — this is the default)
sp = self.shadow.get_solar_position(pd.DatetimeIndex([timestamp]))
solar_elev = sp["solar_elevation"].iloc[0]
solar_az = sp["solar_azimuth"].iloc[0]
if solar_elev <= 2.0:
return {"angle": 0, "offset": 0, "sacrifice": 0, "action": "night"}
tracker = self.shadow.compute_tracker_tilt(solar_az, solar_elev)
theta_astro = tracker["tracker_theta"]
E_astro = max(0, np.cos(np.radians(tracker["aoi"]))) * 0.25
# 2. DEFAULT: full astronomical tracking
result = {
"angle": theta_astro,
"offset": 0,
"sacrifice_kWh": 0.0,
"margin_spent_kWh": 0.0,
"budget_saved_kWh": slot_budget_kWh, # entire slot budget preserved
"action": "full_tracking",
"E_kWh": E_astro,
}
# 3. Check intervention gate
gate = self.gate.check(timestamp, tleaf, tair, cwsi, ghi,
par, co2, vpd, self.farquhar)
result["gate"] = gate
if not gate["allowed"]:
return result # stay at full tracking
# 4. Budget check
total_available = slot_budget_kWh + margin_kWh
if total_available <= 0:
result["action"] = "gate_passed_but_no_budget"
return result
# 5. MINIMUM EFFECTIVE DOSE — scan offsets smallest to largest
for offset in CANDIDATE_OFFSETS:
if offset == 0:
continue # 0 = full tracking, already the default
theta = theta_astro + offset
E_theta = E_astro * np.cos(np.radians(offset))
sacrifice = E_astro - E_theta
# Budget constraint
if sacrifice > total_available:
break # larger offsets will cost even more — stop scanning
# Ray-trace at this angle
mask = self.shadow.project_shadow(solar_elev, solar_az, theta)
# Fruiting zone check: is the fruit zone now shaded enough?
fz = self.shadow.fruiting_zone_shadow(mask)
fz_par = par * (1.0 - fz["fruiting_zone_shaded_pct"] / 100)
# Target: reduce fruiting zone PAR below sunburn threshold
# but not below minimum useful level (~100 umol for diffuse photosynthesis)
fz_par_with_diffuse = max(fz_par, par * 0.15) # diffuse floor
# Top canopy check: is the top canopy still mostly sunlit?
top_mask = mask[2, :] # zone 2 = top canopy
top_sunlit_pct = float(1.0 - top_mask.mean()) * 100
if top_sunlit_pct < 70:
continue # this angle shades too much of the productive canopy
# Does this offset actually help the fruiting zone?
if fz_par_with_diffuse > FRUITING_ZONE_TARGET_PAR:
continue # not enough shade on fruit zone yet, try larger offset
# SUCCESS — this is the minimum offset that protects the fruit
# while keeping the top canopy productive
vine_result = self.canopy.compute_vine_A(
par, tleaf, co2, vpd, tair, mask)
margin_spent = max(0, sacrifice - slot_budget_kWh)
return {
"angle": theta,
"offset": offset,
"sacrifice_kWh": sacrifice,
"margin_spent_kWh": margin_spent,
"budget_saved_kWh": max(0, slot_budget_kWh - sacrifice),
"action": "minimum_dose_shade",
"E_kWh": E_theta,
"A_vine": vine_result["A_vine"],
"fruiting_zone_par": fz_par_with_diffuse,
"fruiting_zone_shaded_pct": fz["fruiting_zone_shaded_pct"],
"top_canopy_sunlit_pct": top_sunlit_pct,
"gate": gate,
}
# No offset achieved the target within budget/constraints
# → stay at full tracking (do not shade ineffectively)
result["action"] = "no_effective_dose_found"
return result
```
##### 3.2.3 Why This is NOT a Utility Function
The prior plan used `U(θ) = α·E(θ) + β(CWSI)·A(θ)` — a weighted sum that trades energy for photosynthesis. This is wrong for three reasons:
1. **The default must be zero intervention.** A utility function always finds a "best" angle, which may not be zero. The gate-then-minimum-dose approach ensures the system stays at full tracking unless the biological case is clear.
2. **Minimum dose, not maximum benefit.** A utility function would pick the angle with the highest A, which may be a large offset. We want the SMALLEST offset that prevents damage — less energy sacrifice for the same biological protection.
3. **The vine doesn't always benefit from shade.** In RuBP-limited state (< 30°C), shading reduces A. A utility function with high β could still recommend shading if the math works out. The gate prevents this categorically.
##### 3.2.4 The Daily Cycle Under This Algorithm
```
Typical July day in Sde Boker:
05:00-10:00 θ = θ_astro (FULL TRACKING)
Gate blocked: morning_protected
Budget spent: 0. All slot budgets preserved → margin pool.
Vine: stomata open, photosynthesizing efficiently.
10:00-10:30 θ = θ_astro (FULL TRACKING)
Gate blocked: below_rubisco_threshold (Tleaf = 29C)
Budget spent: 0.
10:45 θ = θ_astro (FULL TRACKING)
Gate blocked: stress_insufficient (CWSI = 0.35)
Vine is warming but not yet stressed enough.
11:00 θ = θ_astro + 3° (MINIMUM DOSE)
Gate passed: Tleaf=32C, CWSI=0.45, GHI=850 W/m2,
FvCB: Rubisco_Limited, shading_helps=True
Offset 3°: fruiting zone PAR drops to ≈ 380, top canopy ≈ 95% sunlit
Sacrifice: ≈ 0.03 kWh. Fruiting zone protected.
11:15-13:45 θ = θ_astro + 5° to +8° (ESCALATING AS NEEDED)
CWSI rising to 0.55, Tleaf 34-37C.
System uses 5° offset for most slots, 8° for peak.
Top canopy stays ≈ 80–90% sunlit.
Sacrifice: ≈ 0.04–0.08 kWh per slot.
14:00 θ = θ_astro + 5° (MAINTAINING)
CWSI still elevated but no longer rising.
Minimum dose maintained.
15:00 θ = θ_astro + 3° (DE-ESCALATING)
CWSI dropping to 0.38.
15:30 θ = θ_astro (BACK TO FULL TRACKING)
Gate blocked: stress_insufficient (CWSI = 0.28)
Immediate return to full energy generation.
16:00-19:00 θ = θ_astro (FULL TRACKING)
Vine recovering, afternoon light for late photosynthesis.
Total day sacrifice: ≈ 1.2 kWh out of ≈ 25 kWh potential (≈ 4.8%)
Shading duration: ≈ 4.5 hours out of ≈ 14 hours of daylight
Budget used: within daily allocation + small margin draw
```
##### 3.2.5 The Seasonal Pattern
```
MAY: 100% full tracking. Zero sacrifice. Zero intervention.
Vine establishing canopy, flowering, fruit set.
Every photon is valuable. Budget allocation: 0%.
JUNE: ≈ 99% full tracking. Rare intervention on extreme days (>38C).
Maybe 2–3 days in the month trigger 1–2 hours of shading.
Budget allocation: 5%.
JULY: ≈ 85% full tracking. Daily midday shading ≈ 3–5 hours.
This is where the budget is consumed.
Typical: 3–5° offset, 11:00–15:00.
Budget allocation: 45%.
AUGUST: ≈ 85% full tracking. Similar to July but with fruit ripening.
Berry sunburn is the primary concern now.
Fruiting zone targeting is most critical this month.
Budget allocation: 40%.
SEPT: ≈ 95% full tracking. Occasional late heat waves.
Pre-harvest period: balance sugar/acid/phenolics.
Budget allocation: 10%.
SEASON TOTAL: ≈ 92% of time at full astronomical tracking.
≈ 8% of time with minimum-dose shading.
Energy sacrifice: ≤ 5% (guaranteed by budget ceiling).
```
---
#### 3.3 Command Arbiter — `src/command_arbiter.py`
##### 3.3.1 Priority Stack
```
Priority 1 (highest): Weather Protection → stow angle
Priority 2: Mechanical Harvesting → vertical park (90°)
Priority 3: Safety Rail Alert → θ_astro (zero sacrifice)
Priority 4: Simulation Timeout → θ_astro (zero sacrifice)
Priority 5 (normal): TradeoffEngine output → θ_astro or θ_astro + offset
```
All fallbacks default to **full astronomical tracking** — the safe, energy-maximizing default. There is no scenario where a fallback costs energy.
##### 3.3.2 Hysteresis — Motor Protection
```python
class CommandArbiter:
def __init__(self, hysteresis_window_min=15, angle_tolerance_deg=2.0):
self.window_min = hysteresis_window_min
self.tolerance = angle_tolerance_deg
self.buffer = []
self.current_angle = 0.0
def should_move(self, requested_angle, timestamp):
self.buffer.append((timestamp, requested_angle))
cutoff = timestamp - pd.Timedelta(minutes=self.window_min)
self.buffer = [(t, a) for t, a in self.buffer if t >= cutoff]
if len(self.buffer) < 2:
return {"dispatch": False, "angle": self.current_angle}
stable = all(abs(a - requested_angle) <= self.tolerance
for _, a in self.buffer)
if stable:
self.current_angle = requested_angle
return {"dispatch": True, "angle": requested_angle}
return {"dispatch": False, "angle": self.current_angle}
def select_source(self, engine_result, safety_result,
sim_time_sec, weather_override, harvest_active):
if weather_override:
return {"angle": weather_override["target_angle"],
"source": "weather"}
if harvest_active:
return {"angle": 90, "source": "harvest"}
if not safety_result["valid"]:
return {"angle": engine_result.get("angle", 0),
"source": "safety_fallback"}
if sim_time_sec > SIMULATION_TIMEOUT_SEC:
return {"angle": engine_result.get("angle", 0),
"source": "timeout_fallback"}
return {"angle": engine_result["angle"],
"source": "engine"}
```
##### 3.3.3 Return-to-Tracking Speed
When the gate closes (stress drops below threshold), the system returns to full astronomical tracking **at the next 15-min slot** — no ramp-down, no smoothing. The vine transitions from Rubisco-limited back to RuBP-limited and immediately wants full light. The hysteresis only prevents sub-slot jitter; it does not delay the return-to-tracking decision across slots.
---
#### 3.4 Astronomical Tracker (Fallback) — `src/astronomical_tracker.py`
```python
class AstronomicalTracker:
"""Pure sun-following. The always-safe default."""
def __init__(self, shadow_model):
self.shadow = shadow_model
def get_angle(self, timestamp):
sp = self.shadow.get_solar_position(pd.DatetimeIndex([timestamp]))
result = self.shadow.compute_tracker_tilt(
sp["solar_azimuth"].iloc[0], sp["solar_elevation"].iloc[0])
return result["tracker_theta"]
```
---
#### 3.5 The Complete 15-Minute Control Loop
```
┌────────────────────────────────────────────────────────────────┐
│ STEP 1: READ SENSORS │
│ PAR, Tleaf, Tair, CO2, VPD (on-site) │
│ GHI, wind, Tmax (IMS) │
│ CWSI (from SpectralAggregator) │
│ │
│ STEP 2: COMPUTE θ_astro (pvlib) │
│ This is the default. System starts here. │
│ │
│ STEP 3: CHECK WEATHER / HARVEST OVERRIDES │
│ Weather stow? → dispatch stow angle. END. │
│ Harvest mode? → dispatch 90°. END. │
│ │
│ STEP 4: RUN INTERVENTION GATE │
│ No-shade window? → θ = θ_astro. END. │
│ Shade conditions not met? → θ = θ_astro. END. │
│ FvCB says shade hurts? → θ = θ_astro. END. │
│ │
│ STEP 5: BUDGET CHECK │
│ slot_budget + margin > 0? → proceed │
│ No budget? → θ = θ_astro. END. │
│ │
│ STEP 6: MINIMUM DOSE SEARCH │
│ Scan offsets 3°, 5°, 8°, 10°, 15°, 20° (smallest first) │
│ For each: ray-trace → check fruiting zone PAR < 400 │
│ check top canopy > 70% sunlit │
│ check sacrifice ≤ available budget │
│ Select FIRST (smallest) offset that passes all checks. │
│ No offset works? → θ = θ_astro. END. │
│ │
│ STEP 7: SAFETY RAILS │
│ FvCB vs ML divergence > 12%? → θ = θ_astro. END. │
│ │
│ STEP 8: COMMAND ARBITER │
│ Apply hysteresis (15-min stability, ±2° tolerance) │
│ Stable? → dispatch move command. │
│ Not stable? → hold current angle. │
│ │
│ STEP 9: BUDGET ACCOUNTING │
│ Record sacrifice. Update daily margin. │
│ If zero sacrifice: slot budget rolls forward as savings. │
└────────────────────────────────────────────────────────────────┘
```
---
#### 3.6 Edge Cases and Guarantees
| Scenario | System Response |
|---|---|
| Heat wave, 5 days of 42°C | Daily budgets consumed rapidly. Weekly reserve engaged. If exhausted, annual reserve drawn. If ALL reserves empty → full tracking (energy guarantee holds). |
| Cool July week (Tmax 28°C) | Weekly Tmax < 30°C → daily budgets = 0. Gate never opens. Full tracking. Entire week's budget rolls forward to next week. |
| Morning heat spike (09:30, Tleaf 33°C, CWSI 0.5) | Gate BLOCKED: before 10:00. Morning light is protected regardless of stress level. |
| May heat wave (Tmax 40°C) | Gate BLOCKED: May is a no-shade month. Full tracking. Budget for May = 0. Vine exposed but this is critical for fruit set; spring stress is accepted. |
| CWSI = 0.45 but FvCB says shading hurts | Gate BLOCKED: `shading_helps = False`. This means Aj < Ac even above 30°C (e.g., late afternoon with declining PAR). Vine is actually light-limited despite stress signals. No intervention. |
| Offset 3° protects fruit zone sufficiently | Selected immediately (smallest effective dose). Sacrifice minimized. Top canopy stays 95% sunlit. Energy cost: ~0.03 kWh per slot. |
| No offset protects fruit zone within budget | No intervention. System stays at θ_astro. Action = "no_effective_dose_found". Budget preserved. |
---
### Phase 4: Operational Modes
#### 4.1 `src/operational_modes.py`
```python
class OperationalModes:
def check_weather_protection(self, weather_state):
if weather_state.get("hail_alert"):
return {"mode": "hail_stow", "target_angle": 75}
if weather_state.get("wind_speed_ms", 0) > WIND_STOW_SPEED_MS:
return {"mode": "wind_stow", "target_angle": 0}
if (weather_state.get("air_temp_c", 0) > HEAT_SHIELD_TEMP_C
and weather_state.get("cwsi", 0) > HEAT_SHIELD_CWSI):
return {"mode": "heat_shield", "target_angle": "maximize_shade"}
return None
def enter_harvest_mode(self):
return {"mode": "harvest", "target_angle": 90,
"clearance_cm": HARVEST_PARK_CLEARANCE_CM,
"vibration_alarms_disabled": True}
```
Weather and harvest overrides do NOT consume budget. They either generate full energy (wind stow at 0°, harvest at 90°) or protect hardware (hail stow at 75°). The heat shield mode is the only one that costs energy, and it draws from the budget system.
---
### Phase 5: ROI Reporting
#### 5.1 `src/roi_service.py`
Reports on budget utilization and biological effectiveness:
- Budget spent vs allocated (daily, weekly, monthly, annual)
- Number of intervention slots and total shading hours
- Fruiting zone temperature during interventions (was it effective?)
- Energy sacrifice as % of potential (must always be ≤ user ceiling)
- LER computation
---
### Phase 6: Dashboard v2
New tabs in `app.py`:
- **Budget Planner** — Annual/monthly/weekly/daily budget gauges, rollover visualization, margin utilization
- **Control Replay** — Select a day, step through 15-min slots, see gate decisions, offset selection, shadow projection on fruiting zone
- **Fruiting Zone Monitor** — PAR and temperature at mid-canopy over time, sunburn risk indicator
- **ROI Dashboard** — Cumulative sacrifice vs ceiling, LER, intervention frequency
---
### Phase 7: Integration Testing
#### 7.1 `scripts/run_control_simulation.py`
Replay historical data through the full control loop.
#### 7.2 Validation Criteria
- **Hard guarantee**: cumulative sacrifice NEVER exceeds `max_energy_reduction_pct`
- **Default tracking rate**: > 90% of daytime slots remain at full astronomical tracking
- **Gate accuracy**: no interventions in no-shade windows (May, mornings, low stress)
- **Minimum dose**: average intervention offset < 10° (small, surgical corrections)
- **Fruiting zone effectiveness**: during interventions, mid-canopy PAR reduced below 400 umol target in > 80% of cases
- **Top canopy preservation**: during interventions, top canopy sunlit fraction > 70% in all cases
- **Budget utilization**: July–August consume > 80% of total budget; May consumes 0%
---
## Agronomic Value Weighting
A unit of carbon assimilated by the vine does not have a fixed value. Its contribution to the final economic yield depends on **which leaf zone** is doing the work and the **phenological stage** of the vine. The control algorithm must account for both dimensions when evaluating whether a candidate angle is worth its energy cost.
### Spatial Weighting: Contribution by Canopy Zone
The 3-zone ShadowModel maps directly to the vine's source-sink physiology:
#### Zone 0 — Basal Leaves (trunk/cordons, ~0.2m, LAI 15%)
Early in the season, these mature leaves provide the primary carbon source for developing inflorescences. As the season progresses they age and photosynthetic efficiency declines. Removing them later (basal defoliation) has minimal negative impact because their efficiency is already low.
**Algorithmic implication**: Photons reaching zone 0 have declining marginal value as the season progresses. By veraison, shading zone 0 carries almost no penalty.
#### Zone 1 — Fruiting Zone (grape clusters, ~0.6m, LAI 35%)
The grape clusters themselves do not photosynthesize, but the leaves interspersed at this height supply carbon directly to the berries via short transport paths. Berry quality (anthocyanins, phenolics, flavor compounds) develops best under filtered/diffuse light — not harsh direct radiation. Berry surface temperatures above 35°C cause sunburn.
**Algorithmic implication**: This zone's light environment is the primary optimization target. The goal is not maximum PAR but *optimal* PAR — enough for quality development, below the sunburn threshold (400 µmol/m²/s target).
#### Zone 2 — Apical/Lateral Leaves (upper canopy, ~1.0m, LAI 50%)
Young developing leaves do not export carbon until they reach ~50% of final size. However, from veraison onwards, these mid-to-apical leaves and lateral shoots become the most efficient photosynthetic engines — they drive sugar accumulation (°Brix) in the berries.
**Algorithmic implication**: Photons hitting zone 2 during ripening have the highest marginal value. The minimum-dose search already enforces >70% sunlit fraction on zone 2; the spatial weight reinforces this — sacrificing zone 2 light is the most expensive possible intervention.
#### Zone-Weighted Crop Value
When the TradeoffEngine evaluates a candidate offset, the crop value of the resulting light distribution is:
```python
# Spatial value weights by zone (phenology-dependent)
ZONE_CROP_WEIGHTS = {
"pre_veraison": [0.25, 0.35, 0.40], # zone 0, 1, 2
"veraison": [0.10, 0.30, 0.60], # apical leaves dominate sugar loading
"post_harvest": [0.15, 0.15, 0.70], # reserve building, top canopy matters most
}
def zone_weighted_crop_value(A_zones, par_zones, stage, base_crop_value):
"""Compute the economic crop value of a given light distribution."""
weights = ZONE_CROP_WEIGHTS[stage]
weighted_A = sum(w * A for w, A in zip(weights, A_zones))
return base_crop_value * weighted_A
```
---
### Temporal Weighting: Contribution across Growth Stages
The vine's carbon demands shift drastically throughout the season. The crop value multiplier must scale dynamically based on phenological stage, tracked via Growing Degree Days (GDD):
#### Pre-Flowering to Fruit Set (May–early June)
Carbon reserves from trunk and roots are mobilized to support early shoot growth. Maximum daily temperatures and radiation during this window have the largest statistical impact on determining bunch number and overall yield mass.
**Value**: High. Maximizing photosynthesis sets the baseline yield capacity.
**Budget policy**: NO_SHADE_MONTHS = [5] already enforces this — zero intervention, full astronomical tracking.
#### Fruit Set to Veraison (June–mid July)
Rapid berry cell division. Canopy expanding. Carbon demand is high but the vine is generally not yet heat-stressed (Negev temperatures rising but not yet at sustained peaks).
**Value**: High. Intervention rare; the gate's CWSI and temperature thresholds protect this window naturally.
**Seasonal multiplier**: 1.0× (baseline).
#### Veraison to Ripening (mid July–August)
The fruit becomes the dominant carbon sink. Vegetative growth slows to allow preferential carbon redistribution into berries. The ratio of leaf area to fruit weight directly dictates berry mass, sugar concentration, and anthocyanin accumulation.
**Value**: Extremely high, but with a thermal caveat. Above 30°C, carbon assimilation is hampered and organic acids degrade. This is where the intervention system earns its keep — protecting the fruiting zone from sunburn while keeping the top canopy productive for sugar loading.
**Seasonal multiplier**: 1.5× (reflects that each unit of photosynthesis during ripening has disproportionate impact on final grape value).
#### Post-Harvest (September–October)
Berries are gone. The carbon sink shifts back to roots and trunk for starch storage (next spring's budburst reserves).
**Value**: Moderate. The system only needs to maintain sufficient photosynthesis to replenish reserves. The tracker can heavily favor energy generation.
**Seasonal multiplier**: 0.5× (energy production prioritized).
#### Phenology Tracker
```python
# Growing Degree Day thresholds for Semillon (Sde Boker)
PHENOLOGY_GDD_THRESHOLDS = {
"budburst": 0, # GDD base, ~March
"flowering": 350, # ~May
"fruit_set": 500, # ~early June
"veraison": 1200, # ~mid July
"harvest": 1800, # ~late August / September
}
STAGE_CROP_MULTIPLIER = {
"pre_flowering": 1.2, # setting yield capacity
"fruit_set": 1.0, # baseline
"veraison": 1.5, # sugar loading, highest crop value
"post_harvest": 0.5, # reserve building only
}
def get_phenological_stage(gdd_cumulative):
"""Return current growth stage based on accumulated GDD."""
if gdd_cumulative < PHENOLOGY_GDD_THRESHOLDS["flowering"]:
return "pre_flowering"
elif gdd_cumulative < PHENOLOGY_GDD_THRESHOLDS["veraison"]:
return "fruit_set"
elif gdd_cumulative < PHENOLOGY_GDD_THRESHOLDS["harvest"]:
return "veraison"
else:
return "post_harvest"
```
---
### Predicting Baseline Photosynthesis
To assess whether a candidate angle's photosynthesis sacrifice is acceptable, the system needs a baseline prediction — the expected photosynthesis under full astronomical tracking for the day ahead.
#### Mechanistic Baseline (FvCB)
The existing Farquhar model computes A at each 15-min slot under full tracking. This serves as the denominator: `A_baseline(t) = FvCB(PAR_astro(t), Tleaf_forecast(t), CO2, VPD_forecast(t))`.
#### ML-Enhanced Baseline (Auto-regressive)
The Stage 2 ML predictor (RF/XGBoost) ingests the previous day's metrics (approximated A, CWSI, soil moisture) and day-ahead IMS weather forecast (radiation, temperature, wind) to predict next-day A at each slot. This catches microclimate effects the pure mechanistic model may miss.
#### Hybrid Consensus
```python
def predict_baseline_ps(slot, farquhar_model, ml_predictor, weather_forecast):
"""Predict baseline photosynthesis (full tracking) for a future slot."""
A_fvcb = farquhar_model.calc_photosynthesis(
weather_forecast["par"], weather_forecast["tleaf"],
weather_forecast["co2"], weather_forecast["vpd"],
weather_forecast["tair"])
A_ml = ml_predictor.predict(weather_forecast)
# Safety rails: if FvCB and ML diverge > 12%, flag uncertainty
divergence = abs(A_fvcb - A_ml) / max(A_fvcb, A_ml, 1e-6)
if divergence > DIVERGENCE_THRESHOLD:
return A_fvcb, {"confident": False, "divergence": divergence}
# Weighted average (FvCB anchored, ML corrects)
A_baseline = 0.6 * A_fvcb + 0.4 * A_ml
return A_baseline, {"confident": True, "divergence": divergence}
```
---
## Day-Ahead Trajectory Optimization
The real-time control loop (Phase 3) makes slot-by-slot decisions. A greedy per-slot approach can cause excessive motor movement and misallocate the daily budget. A **day-ahead planner** uses Discrete Dynamic Programming to compute the optimal trajectory for the entire day, which the real-time arbiter then executes (or overrides if live sensor data diverges from forecast).
### Algorithm: Day-Ahead Agrivoltaic Trajectory Optimization
#### Step 1: Retrieve Inputs
For the next 24-hour horizon (96 slots at 15-min intervals):
| Input | Source |
|---|---|
| GHI, DHI, ambient temp, wind | IMS station 43 day-ahead forecast |
| Energy market price `Price_energy(t)` | Grid tariff schedule (or flat rate for now) |
| Crop value `Price_crop(t)` | Phenology stage multiplier × base crop value per hectare |
| Baseline PS `A_baseline(t)` | Hybrid FvCB + ML prediction (see above) |
| Candidate angles Θ | `CANDIDATE_OFFSETS` mapped to absolute angles: `{θ_astro(t) + offset}` for each slot |
#### Step 2: Pre-compute State Space
For every slot `t` and every candidate angle `θ ∈ Θ`, pre-compute:
- **`E_t(θ)`**: Expected PV energy generation, accounting for sun position and cosine losses
- **`A_t(θ)`**: Predicted photosynthesis, from ray-traced PAR distribution at angle θ through the FvCB model
- **Gate status**: Whether the intervention gate would allow θ ≠ θ_astro at slot t (no-shade windows, temperature thresholds, etc.)
Slots where the gate blocks intervention are locked to `θ_astro` — the DP does not consider alternatives for those slots.
#### Step 3: Utility Function
For each slot `t` and angle `θ`:
```
U_t(θ) = Price_energy(t) · E_t(θ) + Price_crop(t) · A_t(θ)
```
The crop price term incorporates:
- The **phenological multiplier** (1.5× during veraison, 0.5× post-harvest)
- The **spatial zone weights** (penalizes angles that shade zone 2 more than angles that shade zone 0)
- A **heat-stress penalty** when forecasted Tleaf > 30°C and the angle exposes the fruiting zone to direct sun
#### Step 4: Dynamic Programming with Movement Cost
A greedy approach (best angle per slot independently) causes excessive motor wear. The DP includes a mechanical movement penalty:
```
max Σ_{t=0}^{96} [ U_t(θ_t) - Cost_move · |θ_t - θ_{t-1}| ]
```
- **Forward pass**: For each slot, compute the optimal cumulative utility to reach each candidate angle, considering all possible transitions from the previous slot.
- **Backward pass**: Trace back from slot 96 to extract the single optimal angle sequence `[θ_0, θ_1, ..., θ_96]`.
The movement cost biases the optimizer toward smooth trajectories — holding a consistent offset during the midday stress window rather than oscillating between angles.
#### Step 5: Budget Constraint Integration
The DP trajectory must respect the daily energy budget:
```
Σ_{t=0}^{96} [ E_t(θ_astro) - E_t(θ_t) ] ≤ daily_budget_kWh
```
This is enforced as a constraint during the forward pass: any path whose cumulative sacrifice exceeds the daily budget is pruned. The DP naturally finds the trajectory that maximizes combined utility while staying within the energy ceiling.
#### Step 6: Output — `daily_plan.json`
The optimizer outputs a schedule with explainability tags:
```python
{
"date": "2025-07-15",
"slots": [
{
"time": "05:00", "angle": 45.2, "offset": 0,
"action": "full_tracking",
"tag": "Morning protected — maximizing photosynthesis"
},
{
"time": "11:15", "angle": 48.7, "offset": 5,
"action": "minimum_dose_shade",
"tag": "Sacrificing 0.04 kWh to protect fruiting zone (Tleaf=33°C)"
},
{
"time": "15:30", "angle": 32.1, "offset": 0,
"action": "full_tracking",
"tag": "Stress subsided — maximizing energy revenue"
}
],
"total_sacrifice_kWh": 1.18,
"daily_budget_kWh": 1.35,
"budget_remaining_kWh": 0.17
}
```
### Relationship to Real-Time Control
The day-ahead plan is a **schedule**, not a command. The real-time arbiter (Phase 3.3) executes the planned trajectory but can override it when:
- Live sensor data diverges significantly from the forecast (e.g., unexpected cloud cover, sudden heat spike)
- Weather protection or harvest mode is triggered (Priority 1–2 overrides)
- The safety rails detect FvCB vs ML divergence > 12%
When the arbiter overrides the plan, it falls back to the real-time gate + minimum-dose logic. The daily plan is recalculated at sunrise each morning using the latest forecast.
```
DAY-AHEAD PLANNER (runs once at sunrise)
│
▼ daily_plan.json
REAL-TIME ARBITER (runs every 15 min)
├── Plan matches live conditions? → Execute planned angle
├── Live conditions diverge? → Fall back to real-time gate + min-dose
└── Emergency override? → Weather stow / harvest park
```
### New Module: `src/day_ahead_planner.py`
```python
class DayAheadPlanner:
"""Discrete DP optimizer for day-ahead tracker trajectory."""
def __init__(self, shadow_model, canopy_model, farquhar_model,
ml_predictor, energy_budget):
self.shadow = shadow_model
self.canopy = canopy_model
self.farquhar = farquhar_model
self.ml_predictor = ml_predictor
self.budget = energy_budget
self.gate = InterventionGate()
def generate_plan(self, date, weather_forecast, energy_prices,
crop_value_base, gdd_cumulative,
daily_budget_kWh, cost_move=0.01):
"""Generate optimal angle trajectory for the next day.
Args:
date: Target date
weather_forecast: 96-slot weather forecast DataFrame
energy_prices: 96-slot price vector ($/kWh)
crop_value_base: Base crop value ($/hectare/season)
gdd_cumulative: Growing degree days to date
daily_budget_kWh: Max allowed energy sacrifice
cost_move: Penalty per degree of motor movement
Returns:
Daily plan dict with angle sequence and explainability tags.
"""
stage = get_phenological_stage(gdd_cumulative)
crop_multiplier = STAGE_CROP_MULTIPLIER[stage]
zone_weights = ZONE_CROP_WEIGHTS.get(
"veraison" if stage == "veraison" else
"post_harvest" if stage == "post_harvest" else
"pre_veraison")
# Pre-compute state space
n_slots = len(weather_forecast)
candidates = self._build_candidate_angles(weather_forecast)
utility, energy, gate_locked = self._precompute(
weather_forecast, candidates, energy_prices,
crop_value_base, crop_multiplier, zone_weights)
# DP forward pass with budget constraint
trajectory = self._dp_optimize(
utility, energy, gate_locked, candidates,
daily_budget_kWh, cost_move)
# Generate explainability tags
plan = self._build_plan(date, trajectory, weather_forecast,
utility, energy, daily_budget_kWh)
return plan
```
---
## File Summary
| File | Purpose |
|---|---|
| `src/energy_budget.py` | Hierarchical Year→Month→Week→Day→Slot budget planner |
| `src/tradeoff_engine.py` | InterventionGate + minimum-dose offset search |
| `src/command_arbiter.py` | Fallback logic, hysteresis, priority stack |
| `src/safety_rails.py` | FvCB vs ML divergence validation |
| `src/spectral_aggregator.py` | CWSI/NDVI/PRI batch preprocessing |
| `src/astronomical_tracker.py` | pvlib single-axis fallback |
| `src/operational_modes.py` | Weather protection + harvesting |
| `src/roi_service.py` | Budget utilization + LER reporting |
| `src/data_schema.py` | Telemetry table definitions |
| `src/day_ahead_planner.py` | DP trajectory optimizer + agronomic value weighting |
| `scripts/import_layout.py` | Spatial asset registration |
| `scripts/run_control_simulation.py` | End-to-end historical replay |
| `src/licor_ingest.py` | LiCor data parsing, validation, and storage |
| `src/calibration_pipeline.py` | Parameter fitting + model retraining + versioned deployment |
| `scripts/upload_licor.py` | CLI script for batch LiCor upload |
| Modified File | Changes |
|---|---|
| `src/farquhar_model.py` | `calc_photosynthesis_semillon()` → returns (A, state, shading_helps) |
| `src/solar_geometry.py` | `fruiting_zone_shadow()`, `evaluate_candidate_offsets()` |
| `src/canopy_photosynthesis.py` | Expose `fruiting_zone_A`, `fruiting_zone_par`, `top_canopy_A` |
| `config/settings.py` | No-shade windows, fruiting zone params, budget config, candidate offsets |
| `app.py` | Budget Planner, Control Replay, Fruiting Zone Monitor, ROI tabs |
---
### Phase 8: LiCor Ground-Truth Calibration Pipeline
#### 8.1 Background
Manual LiCor measurements (LI-6400/6800 portable gas-exchange systems) provide the highest-fidelity ground-truth photosynthesis data available — direct leaf-level A, gs, Ci, and transpiration measured under controlled chamber conditions. These readings serve as the gold standard for calibrating and validating the Farquhar model and the ML prediction pipeline.
#### 8.2 Module: `src/licor_ingest.py`
Dedicated ingestion module for LiCor calibration datasets:
```python
class LiCorIngest:
"""Ingest, validate, and store manual LiCor gas-exchange measurements.
Accepts Excel/CSV exports from LI-6400/6800 instruments.
Maps LiCor columns (Photo, Cond, Ci, Trmmol, Tleaf, PARi, etc.)
to internal schema and stores as ground-truth reference data.
"""
LICOR_COLUMN_MAP = {
"Photo": "A_measured", # µmol CO₂ m⁻² s⁻¹
"Cond": "gs_measured", # mol H₂O m⁻² s⁻¹
"Ci": "Ci_measured", # µmol mol⁻¹
"Trmmol": "E_measured", # mmol H₂O m⁻² s⁻¹
"Tleaf": "Tleaf_measured", # °C
"PARi": "PAR_measured", # µmol m⁻² s⁻¹
"VpdL": "VPD_measured", # kPa
}
def ingest(self, file_path: str, metadata: dict) -> pd.DataFrame:
"""Parse LiCor export, validate ranges, attach metadata
(date, vine ID, zone, operator), return clean DataFrame."""
...
def validate(self, df: pd.DataFrame) -> dict:
"""Range checks: A in [-5, 35], PAR in [0, 2500], Tleaf in [10, 55].
Flag outliers and instrument errors (e.g., negative gs)."""
...
def store(self, df: pd.DataFrame, dest_dir: str):
"""Append to ground-truth archive (Parquet partitioned by date)."""
...
```
#### 8.3 API Endpoint for Calibration Upload
A dedicated endpoint (FastAPI or Streamlit file-upload widget) that:
1. **Accepts** LiCor datasets (CSV/Excel upload with metadata: date, vine ID, canopy zone, operator notes)
2. **Validates** data integrity and physiological plausibility
3. **Triggers** a re-training / fine-tuning pipeline:
- Recalibrate Farquhar model parameters (Vcmax25, Jmax25, Rd) by fitting to LiCor A-Ci curves
- Fine-tune ML regressors (Stage 2) using LiCor A as ground-truth labels instead of Farquhar-derived A
- Optionally fine-tune Chronos-2 LoRA adapter on the updated label set
4. **Versions** the updated model artifacts (model registry with timestamp, LiCor dataset hash, metrics delta)
5. **Promotes** the new model version to production only if validation metrics improve (gated deployment)
```python
class CalibrationPipeline:
"""End-to-end: LiCor upload → parameter fitting → model retraining → versioned deployment."""
def run(self, licor_df: pd.DataFrame, current_model_version: str) -> dict:
"""
1. Fit Farquhar params to A-Ci curves → updated Vcmax25, Jmax25
2. Regenerate Stage 1 labels using calibrated Farquhar
3. Retrain Stage 2 ML models on calibrated labels
4. Evaluate on held-out LiCor validation set
5. If RMSE improves ≥ 5%: promote new version
Else: keep current version, log comparison
"""
...
```
#### 8.4 Files
| File | Purpose |
|---|---|
| `src/licor_ingest.py` | LiCor data parsing, validation, and storage |
| `src/calibration_pipeline.py` | Parameter fitting + model retraining + versioned deployment |
| `scripts/upload_licor.py` | CLI script for batch LiCor upload |
---
---
## Phase 10: Reliable Advisor — Anti-Hallucination Guardrails
### Problem
The Gemini chatbot (Phase 9/9.5) hallucinated: it could fabricate sensor readings,
recommend shading that violates biology rules, and answer data questions from
training knowledge instead of calling tools. No grounding verification existed.
### Architecture
```
User Query
↓
QueryClassifier (regex) ← Step 2: data / knowledge / greeting
↓
Gemini Pass 1
↓
Tool call missing + data query? ← Step 2: re-prompt to force tool use
↓
_dispatch_tool()
↓
tag_tool_result() ← Step 4: source label + data age + freshness warning
↓
Gemini Pass 2 (with source citation instructions)
↓
validate_response() ← Step 3: deterministic rule checks (block / warn)
↓
estimate_confidence() ← Step 7: high / medium / low / insufficient_data
↓
ChatResponse ← Step 1: structured schema with metadata
↓
UI: render confidence badge, sources, caveats
```
### Components
| Component | File | Role |
|-----------|------|------|
| `QueryClassifier` | `src/chatbot/guardrails.py` | Regex-based query routing: forces tool calls for data questions |
| `ResponseValidator` | `src/chatbot/guardrails.py` | Deterministic post-response check against biology rules |
| `estimate_confidence` | `src/chatbot/guardrails.py` | Data freshness → confidence level mapping |
| `tag_tool_result` | `src/chatbot/guardrails.py` | Adds `_source`, `_data_age_minutes`, `_freshness_warning` to tool results |
| `ChatResponse` | `src/chatbot/vineyard_chatbot.py` | Extended with `confidence`, `sources`, `caveats`, `rule_violations` |
| `_render_grounding_metadata` | `ui/tab_advisor.py` | Renders confidence badge, source citations, caveats in UI |
### Validation rules enforced
| Rule | Severity | Trigger |
|------|----------|---------|
| `no_shade_before_10` | **Block** | Response recommends shading before 10:00 |
| `no_shade_in_may` | **Block** | Response recommends shading in May without citing extreme conditions |
| `temperature_transition` | Warn | Response recommends shading below 28°C |
| `no_leaves_no_shade_problem` | Warn | Response recommends shading during dormancy |
| `no_shading_must_explain` | Warn | Response says "don't shade" without giving a biological reason |
Block = response text overridden; Warn = caveat appended.
### Confidence mapping
| Condition | Confidence |
|-----------|------------|
| Tool succeeded, data < 30 min | **High** |
| Tool succeeded, data 30–120 min | **Medium** |
| Tool succeeded, data > 120 min | **Low** |
| No tool called | **Low** |
| Tool failed | **Insufficient data** |
| Computed result (FvCB, sim) | **High** |
### Remaining steps (future phases)
- **Step 5 — RAG knowledge base**: Move biology rules from prompt stuffing to searchable documents; inject only relevant rules per query
- **Step 6 — Conversation memory**: Sliding context window with session summary; pin critical context (vine state, active alerts)
- **Step 8 — Guardrail test suite**: Automated adversarial prompts run on every prompt/model change (5 QA tests implemented, expand to 20+)
- **Step 9 — Dual-channel advisory**: Separate information mode (blue, factual) from advisory mode (amber, recommendations with reasoning chain)
- **Step 10 — Feedback loop**: Thumbs up/down, "flag as incorrect" button, weekly failure-mode review
---
## Dependency Order
```
Phase 1 (Config + Schema)
│
▼
Phase 2 (FvCB Semillon + Fruiting Zone Shadow + Safety Rails)
│
▼
Phase 3 (InterventionGate + MinDose Engine + Budget + Arbiter) ← CORE
│
▼
Phase 3.5 (Agronomic Weighting + Day-Ahead DP Planner)
│
▼
Phase 4 (Operational Modes)
│
▼
Phase 5 (ROI Service)
│
▼
Phase 6 (Dashboard v2)
│
▼
Phase 7 (Integration Testing + Budget Guarantee Validation)
│
▼
Phase 8 (LiCor Ground-Truth Calibration Pipeline)
│
▼
Phase 9/9.5 (Vineyard Advisor Chatbot + Data Integration)
│
▼
Phase 10 (Reliable Advisor — Anti-Hallucination Guardrails)
```
# SolarWine 2.0 — Implementation Plan
## Executive Summary
Evolve the existing two-stage photosynthesis prediction pipeline into a biology-driven agrivoltaic control system. The core principle: **full astronomical tracking is the default — the panels follow the sun and generate maximum energy 100% of the time**. Anti-tracking (tilting away from the sun to shade the vine) is a rare, targeted intervention used ONLY when the vine's fruiting zone is at measurable risk of heat or radiation damage, and ONLY at the minimum dose necessary to protect yield.
The user sets a hard ceiling on annual energy sacrifice (e.g., 5%). The system distributes that budget across Year → Month → Week → Day → 15-min slots, concentrating almost all of it in July–August midday windows when heat stress actually threatens grape quality. Margins at each level allow the system to respond to unexpected stress events while guaranteeing the energy ceiling is never breached.
---
## Biological Design Principles
These principles, drawn from viticultural research for *Vitis vinifera* in hot-arid climates, constrain every decision the algorithm makes:
### Principle 1: Morning light is sacred — never shade mornings
Morning radiation (sunrise to ~10:00) is the most valuable light the vine receives:
- **Stomata are maximally open** in early morning (low VPD, cool air). This is when gas exchange is most efficient and photosynthetic carbon loading peaks.
- **Spring mornings (May–June)** are critical for flowering, fruit set, and early berry development. Shading during this period directly reduces yield potential.
- The vine "charges" its carbohydrate reserves in the morning to survive afternoon heat stress. Reducing morning PAR weakens the vine's ability to cope later in the day.
**Hard constraint**: Shading is PROHIBITED before 10:00 local time, regardless of any other signal.
### Principle 2: The fruiting zone matters more than the canopy top
In a VSP (Vertical Shoot Positioning) trellis, the grape clusters hang in the **mid-canopy zone** (~0.4–0.8m height, zone index 1 in our 3-zone ShadowModel at center height ~0.6m). This zone determines grape quality:
- **Berry color** (anthocyanins) requires moderate light exposure — but direct midday sun in the Negev causes sunburn (berry surface > 35°C).
- **Sugar accumulation** (Brix) depends on canopy photosynthesis above, not direct fruit irradiance.
- **Flavor compounds** (terpenes, phenolics) develop best under filtered/diffuse light in the fruiting zone, not harsh direct radiation.
**Optimization target**: When intervention IS needed, choose the minimum tilt angle that shades the **fruiting zone** specifically while keeping the **upper canopy** (zone index 2, where 50% of leaf area lives) in full sun for photosynthesis.
### Principle 3: Shade only when it actually helps — minimum effective dose
Shading is not inherently beneficial. Below 30°C (RuBP-limited state), the vine is light-limited and shading HURTS both energy generation AND photosynthesis. The system should only intervene when:
- The vine crosses into **Rubisco-limited state** (Tleaf > 30°C for Semillon)
- CWSI indicates real water stress (> 0.4), meaning stomata are closing due to heat/VPD
- The FvCB model confirms that reducing PAR at the current temperature would actually improve or maintain A (not reduce it)
And the intervention must be the **smallest angle offset** that brings the fruiting zone below the damage threshold. Once the stress signal drops, return immediately to full tracking.
### Principle 4: Most of the season needs zero intervention
In a typical Sde Boker growing season:
- **May**: Canopy developing, temps moderate (25–32°C). Almost no stress events. Budget needed: ~0%.
- **June**: Temps rising, occasional heat spikes. Rare intervention. Budget needed: ~5%.
- **July**: Peak heat (35–42°C), daily midday stress. This is where 40–45% of the budget goes.
- **August**: Sustained heat, fruit ripening. Critical sunburn risk. Budget needed: 40–45%.
- **September**: Declining heat, pre-harvest. Occasional late heat waves. Budget needed: ~10%.
The budget is heavily concentrated in ~4 hours/day (11:00–15:00) during ~8 weeks (mid-July to mid-August). Outside those windows, the tracker runs at full astronomical tracking with zero sacrifice.
---
## What Already Exists (Baseline v1)
| Component | Module | Status |
|---|---|---|
| Farquhar/Greer-Weedon FvCB model | `src/farquhar_model.py` | Done — Vcmax/Jmax temp response, CWSI, stomatal conductance |
| Shadow/solar geometry | `src/solar_geometry.py` | Done — ShadowModel, 3-zone vertical grid (bottom/mid/top), ray-tracing, PAR distribution |
| Canopy-level photosynthesis | `src/canopy_photosynthesis.py` | Done — zone-weighted Farquhar, LAI weights [0.15, 0.35, 0.50] bottom→top |
| Tilt-angle simulation | `src/tracker_optimizer.py` | Done — cosine-factor PAR scaling, stress heatmap |
| IMS weather client | `src/ims_client.py` | Done — fetch/cache/resample IMS station 43 |
| ML predictor (Stage 2) | `src/predictor.py` | Done — LR, DT, RF, GBR, XGBoost |
| Time-series forecasting | `src/ts_predictor.py` | Done — multi-horizon lag-based |
| Preprocessor | `src/preprocessor.py` | Done — merge, time features, temporal split |
| Sensor data loader | `src/sensor_data_loader.py` | Done |
| Streamlit dashboard | `app.py` | Done — 6 tabs, branded UI |
| Site/panel config | `config/settings.py` | Done — panel 1.13m wide at 2.05m height, canopy 1.2m VSP, 3.0m row spacing |
### Key existing geometry (from ShadowModel)
```
Vertical zones (n_vertical=3):
Zone 0 (bottom): center ~0.2m — trunk/cordons — LAI weight 15%
Zone 1 (mid): center ~0.6m — FRUITING ZONE — LAI weight 35%
Zone 2 (top): center ~1.0m — main leaf canopy — LAI weight 50%
Panel: 1.13m wide, mounted at 2.05m, row spacing 3.0m
Canopy: 1.2m tall, 0.6m wide, VSP trellis
```
---
## Gap Analysis
### New Components
1. **EnergyBudgetPlanner** — Hierarchical budget with no-shade windows and fruiting-zone targeting
2. **TradeoffEngine** — Budget-constrained, minimum-dose optimization
3. **Command Arbiter** — Fallback logic, hysteresis, mode priority
4. **Safety Rails** — FvCB vs ML divergence validation
5. **Spectral Aggregator** — CWSI/NDVI/PRI from sensor data
6. **ROI Reporting Service** — Budget utilization, LER
7. **Weather Protection / Harvest Modes** — Override protocols
8. **Data Schema** — Telemetry storage definitions
### Existing Components to Upgrade
1. **FarquharModel** — Add 30°C Semillon transition returning limiting state