jantheman/heating-grid-forecast-benchmark

Heating Grid Probabilistic Forecasting Benchmark

Day-ahead probabilistic heat-load forecasting for a district heating grid in Memmingen (Bavaria, DE): one shared multi-output model for all 27 end users, 96 steps ahead at 15-minute resolution, each output as 10 quantiles plus a parametric Johnson SU distribution.

Benchmark: seasonal-naive (empirical quantiles) · XGBoost (pooled quantile regression) · LSTM (no graph) · GNN (grid topology). The LSTM/GNN pair isolates what the topology contributes.

Data

jantheman/heating-grid-forecast-data

• Meter data: 2024-12-31 23:00 → 2026-03-27 22:45 UTC, 27 consumers × 43,296 15-min slots. Target: kWh per interval (power kW × 0.25 h; the total_energy counter is cumulative MWh).
• Timestamps verified as true UTC (spring-forward complete, zero duplicate rows). One shared 2 h collection gap on 2025-10-26 00:00–01:45 UTC, bridged by interpolation with an observation-mask feature.
• Cleaning: power_quality != 1 → missing (41,950 rows), negatives → missing, per-endpoint winsorization at q=0.999. True zeros kept (≈28% of rows).
• Long outages left as NaN + loss mask (CONS_022 91% missing in train, CONS_002 19% in test) — interpolation across seasons would be fabrication.
• Exogenous: Open-Meteo ERA5 hourly weather for Memmingen (interpolated to 15 min), Bavarian holidays, calendar, day-ahead price (ffill; ~46% NaN).
• Splits by time: train ≤ 2026-01-15 (36,389 slots), val to 2026-02-28 (4,224), test = March 2026 (2,683 slots = 27 day-ahead origins).

Graph

From Memmingen_pressure.yaml (the digitized DXF plan): 15 junctions + 27 consumers = 42 nodes, 14 pipes (length_m, diameter_mm as edge features) + service connections. Consumer→junction assignment: V_k = CONS_k directly (owner-confirmed); see graph/consumer_mapping.csv in the data repo.

Package

heatingforecast/
  data.py          load/clean/pivot, gap bridging, exog features, splits
  graph.py         42-node graph from YAML + consumer mapping
  distributions.py JohnsonSU (torch) with closed-form quantiles/mean
  metrics.py       pinball, quantile-CRPS (Bracher-style), interval coverage

Quantile levels: 0.05–0.95 in steps of 0.10. Validated against the real data (validate_core.py): monotone quantiles, closed-form median/mean match Monte Carlo, splits and masks as described above.

Results (test = March 2026, 26 day-ahead origins, ~65.8k observed pairs, kWh per 15-min)

Model	Pinball (kWh)	CRPS (kWh)	Cov. @ 0.90	Cov. @ 0.50
GNN (42-node grid graph)	3.42	—	0.908	0.508
LSTM (same model, no graph)	3.39	—	0.897	0.480
Seasonal naive (7-day lag + residual quantiles)	4.33	0.87	0.891	0.426
XGBoost quantile, level target	8.58	1.72	0.531	—
XGBoost quantile, residual target (y − y₇d)	9.04	1.81	0.484	—

Neural-model metrics are converted from z-scaled training units via the per-consumer fit-window sd (see make_comparison.py; full per-consumer breakdowns in results/*.json).

Findings:

• Both neural models beat weekly persistence by ~21% (3.4 vs 4.3 kWh) and are excellently calibrated (GNN 0.908 vs nominal 0.90).
• GNN ≈ LSTM on pinball; the graph's value shows in calibration — the GNN is closer to nominal at every interval width (0.712 vs 0.687 @ 0.70, 0.508 vs 0.480 @ 0.50). At 15 nodes / 14 pipes the topology adds little point-forecast information, but it regularizes the uncertainty.
• The GNN halves the worst-consumer error vs the naive where it matters: CONS_022 (91% missing in train) drops from 28.2 (naive) to 20.9 kWh, and CONS_003 from 13.8 to 12.8 — the shared model pools strength from neighbours and weather where a per-consumer or naive model cannot.
• Both XGBoost variants lose to persistence by ~2× and under-cover badly (0.53 / 0.48 at nominal 0.90 — quantile crossing). The residual-target variant did not help: the failure is the model class at this resolution, not the target parameterization.
• Per-consumer pinball spans 0.02 (CONS_009, mostly off) to ~21 kWh (CONS_022) — the largest consumers dominate the aggregate.

Status

Benchmark complete. All five models trained and evaluated on the same protocol; results in results/. Core modules validated (validate_core.py, validate_epoch.py).