Both flags were silently falling through to default on any unknown value
(UHI_MODEL=Landsat, THRESHOLD_MODE=perzone, or any case/typo mistake
in an HF Space secret). Now each reader checks against the allowed set
and logs a WARNING naming the bad value + the valid options before
returning the default. Happy path is unchanged.
UHI_MODEL must be one of {synthetic, lst}
THRESHOLD_MODE must be one of {global, zone_specific}
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Simplify: reuse shared WBGT+percentile utils, batch DB writes, memoize reads
Code-reuse cleanup:
- src/pricing/zone_thresholds.py: drop local _calculate_wbgt + _percentile
helpers; use src.indexing.heat_index.calculate_wbgt and numpy.percentile.
- src/downscaling/uhi_lst_model.py: promote _load_zone_deltas to public
load_zone_deltas (with underscore-prefix alias retained); add module-level
cache so the features JSON is read once per process instead of per call.
- src/downscaling/__init__.py: factory now imports load_zone_deltas by its
public name (removing the underscore-prefix reach-across-modules).
Efficiency cleanup:
- src/pipeline.py::_step_predict: hoist compute_zone_thresholds out of the
per-zone loop (previously invoked on every get_zone_thresholds call via
both forecast + fallback paths, ~30 JSON reads per run).
- src/pricing/zone_thresholds.py: add in-process _IN_MEMORY_CACHE keyed on
UHI_MODEL so repeat get_zone_thresholds calls within a run skip disk.
- src/pricing/zone_thresholds.py::persist_to_neon: replace per-zone
cur.execute loo…
Adds a zone_thresholds Neon table populated by the pipeline and consumed
by the zones serverless endpoint, so the Dashboard can show per-zone
alert/payout thresholds when zone-specific mode is active instead of
hardcoded city-wide defaults.
- src/database/schema.py: new zone_thresholds DDL (zone_id PK + alert/
payout thresholds + uhi_model + threshold_mode + computed_at).
- src/pricing/zone_thresholds.py: new persist_to_neon() helper that
UPSERTs current thresholds; no-op when THRESHOLD_MODE=global.
- src/pipeline.py: calls persist_to_neon at the start of _step_predict
so every pipeline run pushes current thresholds to Neon (dependent on
the active UHI_MODEL and THRESHOLD_MODE env vars).
- frontend/api/zones.ts: LEFT JOINs zone_thresholds and exposes
alert_threshold_c + payout_threshold_c + threshold_uhi_model +
threshold_mode. Risk-level bucketing now uses per-zone thresholds
when available, falling back to 35/38°C city-wide defaults.
- frontend/src/lib/api.ts: adds thr…
UHILSTCorrector: defer to synthetic UHICorrector for LST-uncovered zones
Fixes premium-inflation artifact for zones without LST coverage
(DAR-KIN, DAR-VIN): correct_temperature's XGB fallback was predicting
deltas in the opposite direction (-0.73°C for KIN) from what
get_zone_uhi_range returned (+1.0°C from synthetic UHI_RANGES['formal']).
Under THRESHOLD_MODE=zone_specific, compute_zone_thresholds iterates
the historical WBGT series with correct_temperature applied per day —
so the XGB's negative delta produced a cool distribution with low P90
(KIN alert = 33.49°C). Then burn_for_zone applied synthetic +1.0°C UHI
to that same history but compared against the cold-biased P90 —
inflating trigger frequency dramatically (KIN premium $195 vs. peer
formal zones MIK $43, MAS $46).
The CV test already showed the XGB fallback can't generalize (test MAE
2.8°C), so replacing it with the synthetic UHICorrector for uncovered
zones is more principled and keeps both code paths in agreement. With
the fix, DAR-KIN's alert threshold recomputes to 35.35°C (up from
33.49°C)…
Under THRESHOLD_MODE=zone_specific, each zone gets its own (alert_c,
payout_c) calibrated to the P90/P97 of that zone's 20-year ERA5-Land ×
UHI-corrected WBGT distribution. This matches the actuarial pattern
used by ARC, CCRIF, and SEWA heat pilot, and fixes the "Jangwani $0
premium" problem that surfaced under UHI_MODEL=lst with the global
35.1°C threshold — zones get equitable ~10% alert / ~3% payout trigger
frequency regardless of absolute temperature distribution.
Under UHI_MODEL=lst + THRESHOLD_MODE=zone_specific:
Jangwani: alert 31.79°C / payout 32.54°C (vs global 35.1/36.0 it
never reached, producing $0 premium)
Tandale: alert 36.10°C / payout 36.93°C (hotter than global, LST
confirmed this zone really is a heat hotspot)
Vingunguti: alert 36.73°C / payout 37.55°C (hottest zone)
Default remains THRESHOLD_MODE=global so production behavior is
unchanged until the HF Space secret is set.
- src/pricing/zone_thresholds.py: new module. Computes per-zon…
Fix FEATURES_PATH resolution for Docker (COPY to /app) vs local
Validates the existing literature-calibrated UHICorrector against Landsat
8/9 Collection 2 Level 2 surface temperatures at 15 Dar zone centroids
(2020-2024, 707 cloud-masked samples across 13 zones). Finding: the
synthetic model systematically overpredicts UHI at valley/coastal Dar —
Jangwani LST hot-season anomaly −5.5°C vs synthetic model +3.8°C implied
delta; Msasani LST −5.9°C vs synthetic +1.8°C. Surface-temperature rank
order (Tandale +3.6°C hottest; Jangwani and Msasani coolest) contradicts
the "settlement_type=informal → high UHI" heuristic.
This commit ships:
- src/downscaling/uhi_lst_model.py: new UHILSTCorrector with the same
correct_temperature() interface. Direct per-zone LST lookup (zone
hot-season anomaly × Voogt & Oke 2003 midpoint LST_TO_AIR_SCALE=0.5)
with XGBoost fallback for zones outside LST coverage (DAR-KIN, DAR-VIN).
- src/downscaling/__init__.py: factory + get_zone_uhi_range() helper.
UHI_MODEL env var picks synthetic or lst at instance time; default
…
Fix TOA accumulation window: 6h → 1h to match ERA5/GraphCast
Same fix as graphcast-benchmark c75e327 (same root cause). Our first
GFS smoke test on the benchmark Space diverged to 60°+ temperatures;
GraphCast's published normalization stats show it was trained on a
1-hour TOA accumulation (mean 1.07e6 J/m², stddev 1.44e6) even though
the model itself steps at 6h. Default changed to 1h; new test asserts
our analytical values land in the GraphCast stats band.
No CRE production behaviour change on the existing ERA5t path;
relevant only when NWP_INIT_SOURCE=gfs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pipeline: one-shot reconnect on DB stale connection
GraphCast inference takes ~80s during which nothing touches Neon. By
the time the predict step tries to write predictions + trigger_events,
the pool's TCP connections have been torn down by Neon's idle timeout
and the first write raises "SSL connection has been closed
unexpectedly". The previous behaviour disabled DB writes for the rest
of the run, so a successful GraphCast run still left the predictions
table populated with the previous run's fallback values — masking the
successful forecast entirely.
Both _db_write and _db_write_sync now call _refresh_conn() on the
first exception and retry once. Only if the retry also fails do we
disable DB writes. _refresh_conn() tests with SELECT 1 and swaps in
a fresh connection from the pool, so this handles the common Neon
idle-suspend case without masking real failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
graphcast: handle static vars without a time dim (GFS path)
The CRE GFS run got past the TOA forcing check and failed at
graphcast_inference.py:339 when .sel(time=t1) was called on a static
variable that had no time dim. ERA5 stores geopotential_at_surface
and land_sea_mask with a spurious time axis (same values at every
step). Our GFS static cache — produced by static_vars.py — correctly
strips that axis, so the static vars are genuinely 2D.
Guard the selection: only .sel(time=...) when the DataArray actually
has a time dim. Drop a lingering time *coordinate* too — .sel strips
the dim but can leave a scalar coord behind.
Fixes: KeyError: 'time' is not a valid dimension or coordinate for
Dataset with dimensions {'latitude': 721, 'longitude': 1440}
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gfs: analytical TOA forcing + pipeline traceback logging
The first CRE GFS smoke fell back to observed_fallback for all 15 zones.
Root cause: GraphCast's task_config forces toa_incident_solar_radiation
on every forecast step, but GFS pgrb2.0p25 doesn't publish it — cfgrib
returns nothing, graphcast.data_utils.extract_inputs_targets_forcings
raises, and the pipeline catches the exception and falls back silently.
This commit:
- adds compute_toa_incident_solar_radiation() that integrates
S0 * max(0, cos zenith) over each 6h window. 12 substeps ≈ 30-min
quadrature, units J/m^2 to match ERA5.
- attaches TOA to full_ds after the per-timestep concat in
fetch_gfs_as_era5 so it's present at every input + forecast step.
- logs per-timestep a count of loaded surface + pressure-level vars
so the next live run will tell us which vars actually made it in.
- switches the pipeline's fallback warning to log.exception so the
traceback lands in container logs — next fall-back, we'll see why.
- 5 new unit tests pin shape, non-negativity, night=0, noo…
Fix GFS shortName regression: use ecCodes names, not cfgrib names
Mirror of the WA2 fix — same bug, same change, synced to stay in lockstep
until init_sources is extracted to a shared package.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add GFS init source as alternative to ERA5T (Phase 2 wiring)
Mirrors the same plumbing shipped in Weather AI 2. CRE's GraphCast
currently inits from ARCO ERA5T at today-5, which means its "5-day
forecast" actually covers today-5 through today-1 — the past week,
not the next week. Trigger decisions confirm past heat events instead
of warning before, which misaligns with the alert-SMS product intent.
GFS has ~3h lag instead of 5 days, enabling forward-looking forecasts
over today → today+5.
Behind a config toggle:
* NWP_INIT_SOURCE=gfs — GFS instead of ERA5T
* NWP_TARGET_DATE_OVERRIDE=YYYY-MM-DD — pin init date for
validation runs against the LMB insurance benchmark panel
Defaults leave existing ERA5T behavior unchanged. Only
src/init_sources/ + tests/test_init_sources/ + two one-line call-site
swaps in graphcast_inference/pipeline touch production code.
56 new unit tests, all pass. Opt-in live GFS test skipped unless
GFS_LIVE_TEST=1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These were left behind after the earlier XGBoost heat-wave predictor
cut (93f565b) and the move to empirical burn analysis for pricing.
Nothing in src/pipeline.py or src/api.py imports them; the eval tests
they ship with couldn't pass either (neural_pricer_dar.pt never
existed on disk, so test_neural_model_loads fails by construction).
Removed:
- src/prediction/heat_forecast.py (HeatWavePredictor)
- src/prediction/lstm_model.py (LSTM trainer + CITY_THRESHOLDS)
- src/pricing/neural_actuarial.py (neural pricer; replaced by burn analysis)
- src/notification/sender.py + __init__.py (notify step was cut in a
prior pipeline pass; no one imports this anymore)
- scripts/train_neural_pricer.py, train_on_era5.py, train_on_nasa_power.py,
train_lstm.py, backtest_pricing.py (all drove the dead paths)
- tests/eval_heat_predictor.py, eval_neural_pricer.py (evaluated dead code)
- models/heat_predictor_xgb.json, trigger_head_retrained.pt
- (untracked) models/heat_lstm.pt, lstm_norm.json,
scripts…
NASA POWER ingest: drop all-missing days instead of inserting empty rows
NASA POWER's publication lag is stochastic -- usually 2 days, sometimes 3.
Before this fix, asking NASA for today-2 when it hadn't published yet
returned -999 for every field; _safe_val filtered -999 to None, but the
parser still appended the DailyReading. That all-NULL row got upserted
into daily_readings, and then _step_downscale hit the pipeline's
hardcoded 30.0 C / 65% humidity fallback paths (pipeline.py:447, 470) to
compute heat_indices -- writing a fake, UHI-plus-30 WBGT row to Neon.
Every Vercel endpoint that does `ORDER BY date DESC LIMIT 1` (zones,
indices, coverage-recommendation, disbursements) would then serve the
fake row as "current conditions" for the dashboard -- e.g. DAR-JAN
showing wbgt=35.6 when the latest real reading (04-18) said 37.5.
Fix: in _parse_response, skip any day where all six measurement fields
came back missing. Partial days (common -- temp_max published a day
before humidity sometimes) still flow through. Now daily_readings never
has all-NULL rows, …
predictions ON CONFLICT: overwrite xgb/lstm/confidence/ensemble too
After cutting HeatWavePredictor, pipeline started writing NULL for
trigger_probability_7d on conflict. But the ON CONFLICT clause only
listed 5 columns for update -- xgb_probability, lstm_probability,
prediction_confidence, and ensemble_method were left out, so every
existing predictions row kept the last XGBoost-era values forever
(e.g. DAR-JAN on 04-21 was still carrying xgb_probability=0.0403
from a pre-cut run).
Expanded UPDATE SET to overwrite all of those plus predicted_at=NOW()
so next run fully heals the stale fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cut XGBoost/LSTM heat predictor from pipeline and demo
Parallels the MI MOS cut. HeatWavePredictor (XGBoost classifier +
ensembled LSTM) was trained on 2 years of pure-synthetic AR(1) noise
using CITY_CLIMATE seasonal curves (heat_forecast.py:331, lstm auto-
retrain at :530). It wrote trigger_probability_7d / prediction_confidence
/ xgb_probability / lstm_probability into the predictions table, but
nothing downstream used those numbers for a decision -- GraphCast drives
trigger firing, burn analysis drives pricing. Classic orphan layer.
Changes:
- Drop HeatWavePredictor import + self._predictor in pipeline.py
- Drop the predictor.predict() call inside _step_predict; the step now
only does GraphCast/observed trigger logic and burn-analysis pricing
- predictions row: NULL for trigger_probability_7d, prediction_confidence,
xgb_probability, lstm_probability. model_tier and ensemble_method
retained (they describe the active forecast source, not the dropped
ensemble). annual_cost_per_worker / payout_factor / learned_frequency
keep com…
Refresh Neon connection at the start of each pipeline run
Same bug family as the stale-conn fix from earlier in the session, but
for the longer-lived _db_conn that's set up once in lifespan and reused
across many pipeline runs. _refresh_conn only ran during __init__, so
the first DB write of a run that fired hours after the last one would
hit a Neon-autosuspended socket.
Concrete failure mode (observed on run fcb5cf78):
- _db_conn created fresh in lifespan, first run writes land fine
- Space idle for 15 min, Neon autosuspends the conn
- Next trigger: start_pipeline_run throws on conn.execute
- _db_write catches it and sets self.db=None for the rest of the run
- Pipeline reports status=ok because the step logic doesn't check DB
outcomes, but Neon has zero new rows for the run (not even the
pipeline_runs row) and the HF status page proudly shows "done"
Fix: before each run, call _db_conn._refresh_conn() which does a
SELECT 1 and reconnects from the pool if the socket's dead. If the
pool itself is also toast, recreate _db_conn via init_db(…
heat_indices + healed_readings ON CONFLICT: overwrite all derived fields
Run 54cb7520 (first post-fix pipeline run) surfaced a sibling of the
daily_readings bug: every heat_indices row was frozen in time. The
pipeline wrote fresh rows with real NASA data but ON CONFLICT only
updated heat_risk_score + risk_level -- neither of which the pipeline
writes (both passed as None at pipeline.py:483-484). So the INSERT
was a silent no-op for the computed fields.
Result: the live dashboard was serving grid_temp_c=30.0 hardcoded-
fallback values left over from the pre-NASA-fix runs, despite having
accurate NASA data in daily_readings.
Fix: expand UPDATE SET to overwrite all derived fields on conflict
(grid_temp_c, uhi_delta_c, corrected_temp_c, wbgt_c, heat_index_c,
heat_risk_score, risk_level, consecutive_hot_days, computed_at=NOW()).
Unlike daily_readings (source data -- use COALESCE to preserve any
good reading), heat_indices is fully derived, so the latest
computation always wins.
Same pattern applied to healed_readings, which had a narrower but
identical bug: O…
Fix real Claude cost bug: token double-count, revert Haiku swap
Neon was reporting \$4.99/run but the actual Sonnet spend was closer to
\$0.25/run. Root cause: HealingAgent's _heal_sub_batch assigns the
cumulative batch total_tokens to EVERY assessment it returns
(healer.py:696). _step_heal then did
`sum(a.tokens_used for a in claude_result.assessments)`, multiplying
the real token count by the number of flagged readings (~20x).
Fix: expose the real batch total as HealedData.tokens_used and read
that field directly instead of summing per-assessment values. No change
to heal behaviour -- only the accounting.
Also reverts the Haiku swap from the previous commit: the swap was
reducing a phantom cost. Sonnet 4.6 is the right model for the
agentic healer (5-tool investigation loop, needs good reasoning over
structured inputs) and for the weekly reviewer. Revert keeps both on
Sonnet and restores the original 0.005/1000 cost factor (correct for
Sonnet blended, not Haiku).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pipeline hardening: Haiku, dead code cleanup, consec fix, ERA5-lag init
Bundles the remaining CRE diagnostic fixes that all touch pipeline.py.
(a) Claude cost: run 67b30124 cost \$4.99 -- 50x the CLAUDE.md target of
~\$0.10/wk. HealingAgent used claude-sonnet-4-6 with a 4-round tool-
use loop at 8192 max_tokens, and the weekly reviewer also used
Sonnet. Switched both to Haiku 4.5 (claude-haiku-4-5-20251001).
Updated blended-cost estimates to \$2/M tokens.
(b) Dead notify step: _step_notify was never called from run() -- the
pipeline dispatches to _step_review at step 6 -- but its method,
its create_sender import, and the insert_notification /
insert_explanation CRUD imports were still wired in. 0 rows in
explanations and notifications tables confirmed. Removed the dead
method + imports. delivery_channel kwarg stays for API compat.
(c) consecutive_days bug: pipeline.py stored
sum(1 for w in reversed(zone_wbgt) if w > THRESH), which is total
days above, not the max consecutive run. Historical trigger_events
…
Prewarm GraphCast at FastAPI startup to avoid cold-start fallback
Mirrors the MI Chronos prewarm pattern. load_model() was called lazily
on the first _step_predict, which on a freshly-rebuilt HF Space took
~30-120s (downloading the 1.4GB checkpoint from GCS plus normalization
stats). That puts the pipeline's first weekly trigger at risk of the
same silent-fallback path that bit MI before PR #5.
Fix: spawn a daemon thread in the lifespan context that calls
load_model() at startup. JAX compile + GCS download happen in the
background while FastAPI comes up; by the time someone hits
/api/pipeline/trigger the model is resident. Failures log and move on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NASA POWER ingest: heal NULL fields, request precip, respect 2-day lag
Diagnostic on run 67b30124 showed 81% of humidity/temp_mean/temp_min/wind
were NULL and 100% of precip_mm was NULL across 315 rows since 2026-04-01.
Three distinct bugs, all in the ingest path:
1. PRECTOTCORR was in the parser but not requested from NASA POWER, so
every precip row came back empty. Added it to NASA_POWER_PARAMS.
2. ON CONFLICT (zone_id, date) in insert_daily_reading + batch variant
only updated temp_max_c and data_quality. If the first insert for a
(zone, date) had NULL humidity (NASA not yet published when the zone
was first ingested) the NULL stuck forever -- subsequent runs
returning full data couldn't overwrite it. Confirmed live: NASA
returned full data for 04-15..04-17 but Neon still showed only
temp_max. Switched to per-field COALESCE(EXCLUDED.x, old.x) so any
later run can fill in fields that were previously null. data_quality
now GREATEST of old+new.
3. ingest_all_zones fetched through today, bypassing nasa_power's
_default_date…
_refresh_conn now retries up to 3 times after SELECT 1 fails. The
previous single-retry path could hand back a second dead connection
because SimpleConnectionPool initializes minconn=2 fresh conns at
startup which typically go stale together after Neon's ~5-min idle
timeout — if the first cached conn was dead, the second often was too.
Matches the Weather AI 2 fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fix pipeline failures: api.py import, NASA POWER resilience, CPU torch
- api.py: Replace NeuralActuarialPricer import with BurnAnalysisPricer
- pipeline.py: Relax ingest completeness check — NASA POWER lags ~7 days,
treat any zone with readings as usable instead of requiring >50% quality
- Dockerfile: Restore CPU torch (sentence-transformers/RAG depends on it)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clean up merge conflict markers in neural_actuarial.py (reference code)
Coverage page:
- Cost section now renders Annual program cost, Cost per worker, and
3-year total in matching 44px Source Serif blocks (previously the
Annual cost was a 56px custom headline and the other two were default
MetricCards, creating an awkward size asymmetry).
- "Where it lands" table rebuilt: dropped the "Total / worker" column
(redundant with fast cash + insurance per worker), added a "Total to
neighborhood" column showing aggregate $ flowing to each triggered
neighborhood this week (per_worker × enrolled_workers), added a
"Workers" column, and renamed the zone header to "Neighborhood".
Reads as: "Kinondoni → $138,600 this week to 7,700 workers".
DPI-style worker pilot (the climate-risk equivalent of Market Intel's
Lakshmi/Kumar/Meena and Weather AI's 2k DPI farmer registry):
- New `workers` table in Neon (src/database/schema.py) with
worker_id / name / name_swahili / nida_id / phone / zone_id /
occupation / age / years_outdoor / household_size / mobile_…
Pipeline was claiming to be adaptable but hardcoded Dar references sat
inside the "DO NOT CHANGE" files. A forker would hit an empty zones list,
a missing era5 file, and the weekly-review prompt would still say "for
Dar es Salaam". Now config.py is the only place that needs editing.
- config.py: add PRIMARY_CITY, PRIMARY_CITY_SLUG, slug_for() helper;
PRIMARY_CITY in ZONES runtime check so a bad edit fails fast
- src/pipeline.py: DAR_ZONES to ACTIVE_ZONES derived from PRIMARY_CITY;
ERA5 path and weekly-review prompt are now config-driven
- scripts/fetch_era5land.py (renamed from _dar): config-driven bbox and
filename via --city; rewrote extract_zone_data to extract_all_zones
(opens each NetCDF file once per month instead of once per zone, 15x
fewer opens for a 20-year fetch)
- scripts/train_neural_pricer.py: fix buggy "Dar es Salaam" to "dar" slug
that would have broken ERA5 file resolution for any fork; raise WBGT
fallback 30.0 to 35.0 for tropical defaults
- tests/eval_…
Add AI trigger explanations behind EXPLANATION_MODE flag
Wire TriggerExplainer into step_explain so explanation generation
can be switched from the default free template path to a Claude-
Haiku-4.5 path via EXPLANATION_MODE=ai. Default stays "template"
so nothing changes at runtime without an opt-in.
- config.py: read EXPLANATION_MODE env var (default "template")
- src/explanation/explainer.py: switch TriggerExplainer default
model from Sonnet to Haiku 4.5 (advisory rewriting is a
template-fill task, not reasoning)
- src/pipeline.py: dispatch Template vs Trigger in _step_explain
based on the flag; report mode and cost in StepResult details
- frontend/src/pages/Pipeline.tsx: add ScalingCostPanel showing
three tiers (current/pilot/state-wide) on the Architecture tab
The design point the panel makes: LLM cost is decoupled from
worker population because explanations fire at trigger events
(a handful per week), not per worker. Pilot at 106K workers
costs the same ~$0.05/wk as the demo; AI explanations on add
less than a dollar even at 1…
Harden serverless DB checks, CORS, cursor lifecycle, and healing JSON parse
Replace the process.env.DATABASE_URL! bang assertion across every
Vercel serverless function with an explicit check that returns 500,
removing a silent crash path. Make src/api.py CORS origins configurable
via ALLOWED_ORIGINS. Fix a cursor leak in PgConnection by closing the
previously-returned cursor before opening the next one, and add a
bulk insert_daily_readings_batch() helper using executemany. Catch
JSONDecodeError on Claude healing responses and return an empty
assessment list rather than crashing mid-heal. Update CLAUDE.md to
the new extreme-heat-insurance.jeff-levine.com custom domain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix $0.00 cost: pass total_cost_usd to finish_pipeline_run
Alert tier (cash, philanthropy-funded): 2+ consecutive days above 33°C WBGT
~12 events/year, $2-5 per event, model scales by vulnerability
Payout tier (insurance, underwritten): 5+ consecutive days + peak > 36°C
~3 events/year, $7-20 per event, model scales by settlement type
Funding decomposition per worker per year:
Worker contribution: capped at $3 (affordable for informal daily wage)
Philanthropy: 45% (covers cash tier + vulnerability gap)
Insurer: 40% (covers insurance tier)
Model outputs: alert_prob, payout_prob, alert_severity, payout_severity
All learned end-to-end from climate data + zone characteristics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Write neural trigger_prob + pricing to predictions table
Pipeline now persists neural model outputs (trigger_prob, payout_factor,
annual_cost_per_worker, learned_frequency) to Neon predictions table.
Vercel coverage-recommendation reads these for real pricing data.
Added columns to predictions: annual_cost_per_worker, payout_factor,
learned_frequency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Neural model now decides triggers AND payouts end-to-end
The model outputs trigger_prob (is zone in danger NOW?) and
payout_factor (vulnerability-weighted severity). No hardcoded
thresholds — the model learns from WHO 33°C WBGT physiological
danger level + settlement type + outdoor exposure.
Vulnerable populations (informal settlements) get higher payouts
because the model learns their payout_factor is higher (0.97 vs
0.93 for formal). Same threshold for everyone — equal protection.
Seasonal ERA5 padding: pipeline uses same calendar period from
prior year to build 90-day context window for the Chronos encoder.
Prices: $94-319/worker/year. Triggers: model-driven, not static.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Neural compound triggers: model confirms risk + current conditions
Trigger fires when BOTH conditions met:
1. Neural model says zone has real risk (λ > 3 events/year)
2. Current conditions show active heat event (3+ consecutive days
above settlement-specific WBGT threshold)
ERA5-Land data pads short NASA POWER history to 90 days for the
Chronos encoder (loaded once at pipeline init, ~0ms per zone).
Tested: 9/15 Dar zones trigger. Informal zones don't trigger because
their 35°C threshold is higher — they only fire in extreme events.
Mixed/formal trigger at lower thresholds (33°C/32°C).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trigger = 3+ consecutive days above WBGT threshold (not single days).
Thresholds calibrated per settlement type so each gets ~11-13 events/yr:
informal: >35°C, mixed: >33°C, formal: >32°C, commercial: >34°C
Prices now $100-295/worker/year (was $3,600). Frequency ~15 events/yr
(was 200). δ_NN corrections are small (-1% to -12%). Pipeline days_back
reduced to 14.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: async CRUD wrappers around sync psycopg2 blocked the event
loop, causing the pipeline to hang on HF Spaces.
Fix: everything is sync now.
- PgConnection uses SimpleConnectionPool with _refresh_conn() for Neon cold starts
- All 21 CRUD functions are plain def (no async)
- Pipeline run() is plain def (no async)
- Only httpx ingestion is async, wrapped in asyncio.run() for that one call
- init_db() returns PgConnection or None
- Lifespan seeds zones synchronously at startup
Tested locally with real Neon DB: 14s, 15 zones, DB writes succeed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skip DB writes in pipeline — psycopg2 blocks async event loop
DB init now runs in background task — doesn't block app startup.
Fixed AttributeError in downscale step (reading.date is a string, not datetime).
Pipeline runs in 2.6s locally with 15 zones, 15 triggers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
asyncpg's connection pool breaks with Neon auto-suspend. psycopg2
with fresh connections per query handles Neon cold-start reliably.
Same pattern that works in Weather AI 2.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix _pool references in CRUD functions for poolless connection manager
All 21 'if db._pool:' guards replaced with 'if db.is_postgres and not db._memory:'
to match the new per-query connection approach.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace asyncpg pool with fresh connections per query
Neon auto-suspends after inactivity, killing pooled connections.
Fresh connections per query let Neon wake automatically on connect.
For a weekly pipeline writing ~100 rows, a pool is unnecessary overhead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restructure pipeline: rule-based heal, template alerts, one Claude review
New architecture (target: ~$0.02/run instead of $3+):
- Heal: rule-based for all readings, Claude only for flagged anomalies (0-1 calls)
- Explain: template-based alerts (free, instant)
- Review: ONE Claude call at the end to validate pricing and write
weekly recommendation in English + Swahili
HF Space status page: vertical pipeline tracker with step dots,
run button, and last-run result. Auto-refreshes every 5s.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cut ERA5-Land from live pipeline, fix zone count to 15 Dar zones
ERA5-Land is training data only (ends 2024). Live pipeline goes
straight to NASA POWER for current conditions. Frontend zone caption
updated to reflect 15 Dar es Salaam zones.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
