Konuktor92's picture
Add full training code for model v1 (same files as PR #1)
7e6579d verified
Raw History Blame Contribute Delete
4.24 kB
# TrendeX "expected greenness" model, protocol v1.
# Locked BEFORE the first model fit. Every output records the git commit of this file.
# Changing anything below after results are seen = a new version (model_v2.yaml), never an edit.
name: expected-greenness
version: v1
question: >
How much of each cell's summer greenness anomaly is explained by weather (precipitation, temperature,
snow, soil moisture) and terrain alone? Residuals = change not explained by weather (never called a cause).
data:
cube: TrendeX90/naryn-cube # cells.parquet + monthly/
cells: in_oblast and mask_reason is null # pasture candidates in the 5 districts
months: [5, 6, 7, 8, 9]
years: [2001, 2025] # 2000 dropped: IMERG starts 2000-06, no Oct-Apr precipitation
target_rows: ndvi_ok == true # pixel reliability 0-1 only; others are not imputed
target:
name: ndvi_anomaly
definition: ndvi - clim(cell, month)
climatology: >
mean NDVI of the same cell and month over the TRAINING years of the current split, excluding the row's own
year (leave-one-year-out inside training years). Never uses test years.
features:
static: [elev_mean, elev_std, slope_mean, steep_share, aspect_sin, aspect_cos, tpi, grass_frac, barren_frac, lc_type1]
calendar: [month]
climatology: [clim_ndvi]
weather_anomalies: # anomaly = value - full-period (2001-2025) mean for the same cell and month
- precip_m # this month, mm
- precip_3m # this + 2 previous months
- precip_winter # Oct (previous year) .. Apr sum
- lst_day_m
- lst_night_m
- lst_day_prev # previous month
- snow_spring # Mar-May mean snow fraction
- snow_m
- soil_m # GLDAS 0-10 cm soil moisture, this month
- swe_apr # GLDAS snow water equivalent, April
weather_climatology_note: >
Weather anomalies use a full-period climatology. These are model inputs, not the target, so this leaks no
target information; the target climatology is fold-aware (see above).
excluded_on_purpose: [year, lat, lon, x, y, any NDVI lag]
splits:
temporal_test: {train: [2001, 2019], test: [2020, 2025]}
spatial_test: {kind: GroupKFold, folds: 5, group: superblock_33x33_cells} # all years, unseen areas
cross_fit_for_residuals: {kind: contiguous_year_blocks, blocks: [[2001, 2005], [2006, 2010], [2011, 2015], [2016, 2020], [2021, 2025]]}
inner_validation_for_tuning: {train: [2001, 2015], valid: [2016, 2019]} # never touches 2020-2025
baselines:
B0_climatology: anomaly = 0
B1_ridge: sklearn Ridge on the same standardized features, alpha chosen on the inner validation split
model:
kind: xgboost
params_fixed: {tree_method: hist, device: cuda, objective: "reg:squarederror"}
tuning: {sampler: optuna_tpe, max_minutes: 25, max_trials: 40, metric: rmse_inner_valid}
seeds: [0, 1, 2, 3, 4]
metrics:
primary: skill_vs_ridge = 1 - MSE_model / MSE_ridge
others: [rmse, mae, skill_vs_climatology, anomaly_correlation, tercile_balanced_accuracy]
uncertainty:
temporal: bootstrap over test years (2000 resamples)
spatial: bootstrap over superblocks (2000 resamples)
report_per: [cell_month, block_summer_mean, district_summer_mean]
gate:
rule: ship xgboost only if the 95% CI lower bound of skill_vs_ridge > 0 on BOTH temporal and spatial tests
otherwise: ship ridge as the expected-greenness model and say so on the Methods page
intervals:
method: split conformal on cross-fitted residuals of training years, per month
levels: [0.1, 0.9]
check: empirical coverage on the temporal test years
residual_trends:
series: summer (Jun-Aug) mean of out-of-fold residuals per cell and year
test: trend.py trend_1d (Sen slope + Mann-Kendall, Hamed-Rao) - copied unchanged from the backend
multiple_testing: Benjamini-Hochberg across cells, q < 0.05
min_years: 18
placebo: shuffle years within each cell, 200 permutations, mk_vectorized
wording: "change not explained by precipitation, temperature, snow and soil moisture"
cross_check:
aqua_vs_terra: per-block summer NDVI trend 2003-2025 from MYD13A3 vs MOD13A3; report sign agreement