The 60s→180s timeout bump (PR #4) helped the slow-download failure mode
but didn't help when Chronos fails via a quick exception (network/HF
Hub error). Observed today: a post-rebuild trigger finished its forecast
step in 9.1s on XGBoost — the timeout didn't fire, Chronos had
errored out fast and fallen through to the fallback.
More reliable fix: kick off Chronos .load() in a daemon thread via a
FastAPI lifespan hook at Space startup. By the time the first pipeline
trigger arrives (GH Action does wake → 90s sleep → health check →
trigger, so ~60-90s minimum after container start), Chronos is either
already loaded or actively loading with its cache being populated. The
subsequent pipeline-side load() hits the disk cache in <1s regardless
of whether prewarm finished.
If prewarm itself fails for any reason, the pipeline's existing
fallback chain (Chronos → XGBoost → seasonal baseline) still covers
the first trigger.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Remove orphaned MOS layer — Chronos-2 is the forecasting model (#2)
The XGBoost MOS "bias correction" sitting on top of Chronos-2 wasn't
earning its keep:
- The monthly retrain (scripts/retrain_mos.py) wrote to models/mos_*.json,
which the live pipeline never loaded — it reads models/price_model.joblib
via ChronosXGBoostForecaster.load(). The retrain has been a silent no-op
since it was written.
- The in-pipeline _train_mos path trained on synthetic data every run
(generate_training_data), against residuals that were structurally OK
but contributed no real accumulated learning. Fresh model per run, 20
samples minimum, no eval gate.
- The retrain script itself had three correctness bugs (cartesian join,
current_price = actual label leak, no promotion gate) — all moot now
since the script is deleted.
Chronos-2 is the honest ML story: a foundation model producing zero-shot
probabilistic forecasts with quantile-native confidence intervals. The
XGBoost standalone model remains as a fallback when Chronos fails to
load or a pair has insuffic…
Makes the recommendation agent region-parameterized and renames the
India-specific output field recommendation_ta -> recommendation_local
(+ sibling local_language_code: 'ta' | 'sw').
- REGION_CONFIG dict keyed by region with 10 fields (region_name,
currency_symbol, market_type, local_language_name/code,
phone_country_code, local_greeting_example, typical_crops, etc.)
- SYSTEM_PROMPT + TRANSLATION_PROMPT parameterize via .format()
- Claude models unchanged: Sonnet for reasoning, Haiku for translation
- FarmerRecommendation dataclass: recommendation_ta -> recommendation_local
- Rule-based fallback is region-aware (currency, market terminology)
- Delivery SMS: phone country code + language-tagged local text
(+254..., 'SW:' prefix under Kenya; +91..., 'TA:' under India)
- src/api.py demo endpoint emits the new fields
- src/db.py: DB-schema-gap comment at SellRecommendation persistence
(local text lives in full_data JSONB; no dedicated column change)
Co-Authored-By: Claude Opus 4…
Replace print logging with structured logger; update Vercel custom domain
Convert all print(..., flush=True) calls in the pipeline scheduler and
retrain endpoint to logger.info/error so step status, pipeline results,
and failure traces flow through the standard logging config instead of
raw stdout. Update CLAUDE.md to the new crop-pricing.jeff-levine.com
custom domain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Step 7: SMS delivery for sell recommendations
- scripts/retrain_mos.py: pulls prediction-vs-actual pairs from Neon,
builds training features, retrains XGBoost MOS correction models
- POST /api/pipeline/retrain-mos endpoint for triggering from GH Action
- Weekly GH Action retrains on 4th Monday of each month (or manual)
- Model improves over time as more real Agmarknet data accumulates
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add live pipeline tracker to HF Space status page
- /api/pipeline/status endpoint with step-level progress
- HTML status page now shows pipeline steps with live polling
- Green dots for completed, amber pulse for active, gray for pending
- Run Pipeline button with auto-refresh
- Pipeline progress callback from each step to scheduler
Also: chronos-bolt-tiny, 60s timeout, print() logging for HF logs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete Claude agents, Chronos-2 forecasting, and frontend polish
P0: SPA catch-all returns 404 not None, recommendation_to_dict includes
farmer_id/name/tamil, demo sell options include drive_time_min/storage_cost.
P1: Pipeline forecasts use current_price_rs (matching frontend), model
metrics flattened (r2 not nested under metrics), seasonal_index added to
forecasts, StatusBadge handles active/partial, Forecast chart uses real
CI values not hardcoded scaling, division-by-zero guard added.
P2: Package name health→market-intelligence, health-domain artifacts in
docstrings/comments cleaned, Google Maps reference removed (uses haversine),
deprecated datetime.utcnow→now(timezone.utc), confidence clamped to 0.10+,
dist_bak deleted.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace health supply chain optimizer with market intelligence agent
Complete rewrite from health domain to post-harvest market intelligence
for Tamil Nadu smallholder farmers. Same 6-step pipeline architecture
(ingest → extract → reconcile → forecast → optimize → recommend),
new domain: conflicting mandi price data → reconciled prices →
sell-now-or-wait recommendations with credit readiness assessment.
Backend: 15 real Tamil Nadu mandis, 10 commodities, real Agmarknet API
(data.gov.in), eNAM scraper, NASA POWER weather, XGBoost price forecaster,
Claude agents with rule-based fallbacks, FAISS+BM25 RAG.
Frontend: 5 pages (Markets, Forecast, Sell Advisor, Pipeline, Data),
amber/saffron theme, Tamil Nadu map, Joyride tour, credit readiness UI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix Chronos integration bugs and efficiency issues from /simplify
- Fix silent bug: column names _facility_id/_drug_id (underscore prefix)
in training data — Chronos predictions were silently empty
- Defer `import torch` from module level to _ensure_loaded/predict_batch
— avoids 1-3s PyTorch load on every process start
- Add torch.no_grad() context for inference
- Use module-level singleton for ChronosBoltForecaster in pipeline
— avoids reloading 9M-param model on every pipeline run
- Remove hardcoded Chronos metadata from api.py, use class directly
- Fix Dockerfile: mkdir models/ instead of COPY (models are gitignored)
- Update tests for underscore-prefixed column names + add plain-column test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Amazon Chronos-Bolt neural foundation model for demand forecasting
Integrates Chronos-Bolt-Tiny (9M params, T5 transformer pre-trained on
100B+ time series) as a zero-shot ensemble partner with XGBoost. Point
forecasts use 65/35 weighted average; prediction intervals take the
wider bounds for conservative coverage.
Key additions:
- src/forecasting/chronos_model.py: ChronosBoltForecaster wrapper +
ensemble_predictions + build_series_from_training_data
- Pipeline Step 4: Chronos runs as Layer 2.5 between XGBoost and
residual correction, ensembles into forecast dicts
- API: /api/model-info now surfaces chronos_bolt in ml_stack
- Frontend: Chronos hero card in Demand "How It Works" tab
- 10 new tests (all passing, 54 total)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows facilities, medicines tracked, stockout risks, data reliability,
active stockout risk table, and API endpoint directory — matching the
Climate Risk Engine's status page pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Expand XGBoost demand model from 13 to 20 features (RMSE 5.8→3.3, R²=0.998)
- Add residual correction model (MOS pattern: model-correcting-model)
- Add isolation forest anomaly detection on consumption patterns
- Upgrade RAG from TF-IDF to sentence-transformers (all-MiniLM-L6-v2)
- Expand knowledge base from 33 to 101 chunks across 15 categories
- Add Neon PostgreSQL persistence layer (5 tables)
- Add evaluation suite (44 pytest tests)
- Wire new components into pipeline and API
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Unify 4 endpoints to use _get() helper instead of inline store/demo branching
- Add _source() helper replacing 9 duplicated ternary expressions
- Rename store.procurement_plan -> procurement_plans (list) to match demo data key
- Add thread-safe double-checked locking on demo cache
- Remove setdefault() calls that mutated shared cache on every request
- _get() now accepts default param for dict-typed fields
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add README.md with HF Space YAML frontmatter (docker SDK, port 7860)
- Add .dockerignore for cleaner builds
- Update Dockerfile: non-root user, healthcheck, full requirements
- Add requirements-dashboard.txt (lightweight, unused for now)
- Fix _demo -> _get_demo() bug in procurement/model/pipeline endpoints
- Restrict CORS to specific HF Space and Vercel origins
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Full-stack AI pipeline for health commodity forecasting and procurement optimization.
Backend: FastAPI + Claude agents for extraction, healing, reconciliation, demand forecasting, and procurement.
Frontend: React + Vite + Tailwind dashboard with guided tour.
CI: GitHub Actions for backend/frontend validation + weekly pipeline trigger.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
