Two coordinated fixes:
1. ChronosXGBoostForecaster.train() now sets self.model_used = "chronos2"
when _init_chronos() succeeds. The previous refactor set model_used =
"xgboost" from the XGBoost training step and never updated it when
Chronos loaded, so the "intent" reported to the pipeline was wrong
whenever the train path ran (which happens on the first run after
every HF Space rebuild — no joblib on disk → load() raises → train()).
2. pipeline.py recaptures model_type = model.model_used AFTER predict(),
not before. The model.model_used attribute is updated by predict()
based on which branch actually ran; capturing before predict() records
intent, not reality. This guarantees the step_results detail and the
price_forecasts.model_type column reflect the forecast path the
farmer-facing outputs actually came from.
Observed behavior before the fix: forecast step ran in 2.8s (Chronos
range, not the ~9s XGBoost-standalone range), Chronos clearly loaded
from …
HF Spaces rebuilds wipe /home/appuser/.cache/huggingface even though
the Dockerfile pre-bakes amazon/chronos-bolt-tiny there, forcing a
re-download on the first trigger after each rebuild. Cold-start
download takes ~80-90s, exceeding the prior 60s timeout and falling
through to XGBoost for the first run. Subsequent triggers load
from disk cache in <1s.
Default timeout now 180s — fits the observed cold-start with headroom.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove orphaned MOS layer — Chronos-2 is the forecasting model (#2)
The XGBoost MOS "bias correction" sitting on top of Chronos-2 wasn't
earning its keep:
- The monthly retrain (scripts/retrain_mos.py) wrote to models/mos_*.json,
which the live pipeline never loaded — it reads models/price_model.joblib
via ChronosXGBoostForecaster.load(). The retrain has been a silent no-op
since it was written.
- The in-pipeline _train_mos path trained on synthetic data every run
(generate_training_data), against residuals that were structurally OK
but contributed no real accumulated learning. Fresh model per run, 20
samples minimum, no eval gate.
- The retrain script itself had three correctness bugs (cartesian join,
current_price = actual label leak, no promotion gate) — all moot now
since the script is deleted.
Chronos-2 is the honest ML story: a foundation model producing zero-shot
probabilistic forecasts with quantile-native confidence intervals. The
XGBoost standalone model remains as a fallback when Chronos fails to
load or a pair has insuffic…
Fix stale comment: bolt-base → bolt-tiny in model selection docs
iterrows() yields index labels but iloc expects positional indices.
When the DataFrame index is non-sequential (after filtering), iloc[[i]]
goes out of bounds. Changed to loc[[i]] in all prediction paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch to chronos-bolt-tiny, add visible pipeline logging, 60s timeout
- bolt-tiny (8MB) loads in seconds on CPU, bolt-base (700MB) was hanging
- Add print() logging in scheduler so pipeline progress shows in HF logs
- Reduce Chronos load timeout from 300s to 60s for faster fallback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move model pre-download AFTER pip install so huggingface_hub is
available. Restores bolt-base (not tiny). The Docker build downloads
the 700MB model once; subsequent pipeline runs load from disk.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch to chronos-bolt-tiny (8MB) for fast CPU loading
chronos-bolt-base (700MB) was hanging on download during pipeline runs.
bolt-tiny is 8MB, loads in seconds, still provides neural foundation
model forecasts with XGBoost MOS correction on top.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents pipeline from hanging indefinitely when downloading the
Chronos-2 model on a cold HF Space start. Falls back to XGBoost
if load exceeds 300s.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete Claude agents, Chronos-2 forecasting, and frontend polish
P0: SPA catch-all returns 404 not None, recommendation_to_dict includes
farmer_id/name/tamil, demo sell options include drive_time_min/storage_cost.
P1: Pipeline forecasts use current_price_rs (matching frontend), model
metrics flattened (r2 not nested under metrics), seasonal_index added to
forecasts, StatusBadge handles active/partial, Forecast chart uses real
CI values not hardcoded scaling, division-by-zero guard added.
P2: Package name health→market-intelligence, health-domain artifacts in
docstrings/comments cleaned, Google Maps reference removed (uses haversine),
deprecated datetime.utcnow→now(timezone.utc), confidence clamped to 0.10+,
dist_bak deleted.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace health supply chain optimizer with market intelligence agent
Complete rewrite from health domain to post-harvest market intelligence
for Tamil Nadu smallholder farmers. Same 6-step pipeline architecture
(ingest → extract → reconcile → forecast → optimize → recommend),
new domain: conflicting mandi price data → reconciled prices →
sell-now-or-wait recommendations with credit readiness assessment.
Backend: 15 real Tamil Nadu mandis, 10 commodities, real Agmarknet API
(data.gov.in), eNAM scraper, NASA POWER weather, XGBoost price forecaster,
Claude agents with rule-based fallbacks, FAISS+BM25 RAG.
Frontend: 5 pages (Markets, Forecast, Sell Advisor, Pipeline, Data),
amber/saffron theme, Tamil Nadu map, Joyride tour, credit readiness UI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix Chronos integration bugs and efficiency issues from /simplify
- Fix silent bug: column names _facility_id/_drug_id (underscore prefix)
in training data — Chronos predictions were silently empty
- Defer `import torch` from module level to _ensure_loaded/predict_batch
— avoids 1-3s PyTorch load on every process start
- Add torch.no_grad() context for inference
- Use module-level singleton for ChronosBoltForecaster in pipeline
— avoids reloading 9M-param model on every pipeline run
- Remove hardcoded Chronos metadata from api.py, use class directly
- Fix Dockerfile: mkdir models/ instead of COPY (models are gitignored)
- Update tests for underscore-prefixed column names + add plain-column test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Amazon Chronos-Bolt neural foundation model for demand forecasting
Integrates Chronos-Bolt-Tiny (9M params, T5 transformer pre-trained on
100B+ time series) as a zero-shot ensemble partner with XGBoost. Point
forecasts use 65/35 weighted average; prediction intervals take the
wider bounds for conservative coverage.
Key additions:
- src/forecasting/chronos_model.py: ChronosBoltForecaster wrapper +
ensemble_predictions + build_series_from_training_data
- Pipeline Step 4: Chronos runs as Layer 2.5 between XGBoost and
residual correction, ensembles into forecast dicts
- API: /api/model-info now surfaces chronos_bolt in ml_stack
- Frontend: Chronos hero card in Demand "How It Works" tab
- 10 new tests (all passing, 54 total)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Simplify: fix bugs, remove duplication, use public APIs
- Remove dead `or True` filter in anomaly detector score()
- Remove dead REPORTING_QUALITY_ENC from detector.py
- Add public metrics/feature_importances properties to XGBoostDemandModel
- Add predict_batch() for vectorized predictions
- Fix residual model to use public predict_batch() instead of _model internals
- Reuse training_df in pipeline instead of calling build_training_data twice
- Add exception handling for residual model training failures
- Replace per-item anomaly scoring loop with batch score_batch()
- Use real facility/drug data for anomaly features instead of hardcoded values
- Gate init_db() with flag so DDL only runs once
- Replace duplicate LEAD_TIME_MAP with import from config.LEAD_TIMES
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Expand XGBoost demand model from 13 to 20 features (RMSE 5.8→3.3, R²=0.998)
- Add residual correction model (MOS pattern: model-correcting-model)
- Add isolation forest anomaly detection on consumption patterns
- Upgrade RAG from TF-IDF to sentence-transformers (all-MiniLM-L6-v2)
- Expand knowledge base from 33 to 101 chunks across 15 categories
- Add Neon PostgreSQL persistence layer (5 tables)
- Add evaluation suite (44 pytest tests)
- Wire new components into pipeline and API
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full-stack AI pipeline for health commodity forecasting and procurement optimization.
Backend: FastAPI + Claude agents for extraction, healing, reconciliation, demand forecasting, and procurement.
Frontend: React + Vite + Tailwind dashboard with guided tour.
CI: GitHub Actions for backend/frontend validation + weekly pipeline trigger.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
