Spaces:
Running
Running
Commit History
16 days ago
-
Retrain ML models through 2026-10-27 (was 2026-07-24)
Khanna, Videh Rakesh Rakesh +1: Claude Sonnet 5.5Holdout 2026-05-27 to 2026-10-27: INTRADAY dir 81%, 1D 45%, 3D 40%; q10/q90 coverage 87-91% (target ~90%). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>66fc4a7
-
Fix news fetching and reliability regressions
Khanna, Videh Rakesh Rakesh21db26a -
Predict AI forecast for recently listed stocks instead of refusing below 200 bars
Khanna, Videh Rakesh Rakesh +1: Claude Sonnet 5.5predict_stock_v2 hard-refused anything under 200 bars, so recent IPOs (e.g. RENTOMOJO, 17 bars) got no AI prediction even though the ML model had been relaxed. Refuse only below a 15-bar floor; between 15 and 199 bars predict as a degraded call: confidence is capped to LOW and the result carries low_history/history_bars. get_ml_feature_score's early return now includes the "upgraded" key the caller indexes (it raised KeyError for short history). ml_predictor MIN_BARS lowered 20 -> 15 to match. The watchlist payload passes low_history through and the TF cell shows a "New listing" badge; stale "after 200 bars" messages reworded. Cache buster bumped. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>6887c6c
-
Predict for low-history tickers instead of hard-refusing below 205 bars
Khanna, Videh Rakesh Rakesh +1: Claude Sonnet 5ml_predictor previously refused any prediction below 205 bars (the model's training minimum), even though every short-window feature is already computable and neutral-defaulted well before that. Add a degraded tier (20-204 bars): predict using the existing model, force confidence to LOW, and flag it, instead of returning ml_unavailable. Below 20 bars still refuses as unusable. Also lower the dead 26-bar guard in compute_features() to the new 20-bar floor, and fix the frontend's stale "awaiting 26 daily bars" message to report real bar counts from the API instead of a hardcoded, drifted number. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>5703a07
-
Retrain ML models through 2026-07-24 (was 2026-07-10)
Khanna, Videh Rakesh Rakesh +1: Claude Sonnet 4.6351k rows · 2158 tickers · 2022-05-12 → 2026-07-24 Train ≤ 2026-02-19 · holdout 2026-02-24 → 2026-07-24 (36k rows) INTRADAY: dir_acc=82% (↑ from 81.5%), q90 coverage=88% 1D: dir_acc=44%, q90 coverage=90% 3D: dir_acc=41%, q90 coverage=91% Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>68dc5e0
-
Fix ML watchlist range refresh and improve IPO/model hygiene
Khanna, Videh Rakesh Rakesh +1: Claude Sonnet 4.6- Force ?refresh=1 on ML forced-fetches so server recomputes live_price instead of returning a 60s-cached response; ₹ targets now visibly update every 5 min during market hours - Add dedicated "🤖 ML as-of HH:MM:SS IST" timestamp on each ML row so the ML fetch time is distinct from the AI cell's "Updated" clock above it - Update % range tooltip to clarify it is session-static (daily model features from yesterday's close); ₹ range tooltip says it updates live - Raise ML hard-abort from 26 → 205 bars (matching dataset.py training floor) with a descriptive IPO/short-history error message; previously the model silently extrapolated outside its training distribution - Fix _start_intraday_refresh_scheduler docstring: "every 15 min" → "3 min" - Add SHORT-trade API tests (open, reject invalid direction, close P&L) - Bump cache buster to v=20260902d Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>b85a206
-
Fix ML/intraday predictions getting stuck forever on Yahoo hangs
Khanna, Videh Rakesh Rakeshget_intraday_bars() serialized all callers on a single global lock and called yf.download() with no timeout. yf.download has no built-in timeout, so one stalled Yahoo connection wedged the lock forever, freezing intraday ORB/VWAP context (every AI INTRADAY forecast) and the /api/ml-predict today_high fetch for every ticker for the rest of the process's life - exactly the '\''ML gets stuck and never loads'\'' symptom (reproduced: 9 concurrent /api/ml-predict calls all timed out >120s). - intraday_live.get_intraday_bars: bound the lock wait (20s) and the actual yf.download() call (15s, via a worker thread) so a stall degrades to None instead of hanging every future caller forever. - ml_predictor/infer.py _indices(): same unbounded yf.download() risk for Nifty/VIX; now routed through data_sources._yf_download_timed (15s bound), the same pattern already used elsewhere in the codebase.c055932
-
Real direction-accuracy/P&L measurement + BULLISH trigger fixes
Khanna, Videh Rakesh Rakesh +1: Claude Sonnet 5The backtest only ever graded whether price touched a predicted band (target_hit_for_tf), which a deliberately low near-bound wins regardless of whether a direction call has any real edge. Added a standing real DirAcc + net P&L report (TABLE 6, research/backtest.py) and used it to find and fix three BULLISH triggers that were worse than a coin flip (T4, T5, T7 in ai_forecast.py), plus a relative-strength-vs-Nifty guard that blocks BULLISH on stocks structurally underperforming the market (mirrors ml_predictor's already-validated excess-of-Nifty labels). Net result on a 200-row diverse backtest: 1D BULLISH direction accuracy improved 54.2% -> 56.4% on 12 of 15 tickers; 3 known-hard stocks (HDFCBANK, TITAN, SUNPHARMA) remain a resistant drag after extensive targeted iteration - accepted as a genuine ceiling, not a fixable bug. Also includes the earlier, separately-validated INTRADAY/1D NEUTRAL-band widening (1D target_hit_for_tf 83.3% -> 88.9%) and a real bug fix: the Ollama fallback's…5342112
-
AI range: containment mode + Target-below-range fix + ML un-floor
Khanna, Videh Rakesh Rakesh- AI_RANGE_MODE (default 'containment'): INTRADAY/1D use a wide prediction interval (low reachable near bound) so the move lands in-range ~82% (backtested). AI_RANGE_MODE=target restores the tight ~50%-hit band. - Fix 'Target below range' bug: predictor_core keeps the AI sub-object's band in lockstep with ret_lo/ret_hi for all directional calls, and app._reanchor_targets_to_live recomputes the midpoint from ret_lo/ret_hi so the Target always sits inside the shown range (also self-heals caches). - Remove the >=1% floor from the ML sub-row: it collapsed the [q10,q90] interval's containment ~26pts; the honest wide interval hits ~87%. - Research: price_hit_research.py (reachability ceiling — a >=1% target maxes ~45-58% hit) and blend_backtest.py (AI-when-confident blend: LLM emits HIGH ~1% of the time and its tighter ranges hit LESS -> no gain).dd0818f -
Intraday: floor the WHOLE range to >=1% (not just best case)
Khanna, Videh Rakesh RakeshPreviously only the far/best-case bound was floored to 1%, so ranges like 0.30%-1.00% (midpoint 0.65%) still showed sub-1% moves. Now both the near and far bounds are floored to >=1% magnitude, keeping a small spread, so the entire predicted intraday move is at least 1%. The far bound stays volatility-scaled. Added INTRADAY_MIN_SPREAD_PCT (env, 0.1). Synced across ai_forecast, predictor_core and ml_predictor/infer.2a7068f
-
Fix wide NEUTRAL bands, live/pre-market target re-anchoring, stop-loss guard; add ₹ range to ML row
Khanna, Videh Rakesh Rakesh- NEUTRAL/1D range-only bands now flat ±1% (falsifiable) across ai_forecast, predictor_core, ml_predictor — no more unbettable ±5% ATR bands - Re-anchor cached targets to live/pre-market price in watchlist + top picks so ranges aren't already-surpassed - Server-side stop-loss/target sanity guard: prevent LONG stop above entry (instant stop-out) on actual-entry drift - Top picks now expose current_price - ML row shows ₹ price range mirroring the AI, derived from the same tightened band - Bump UI cache-buster 20260728h -> 20260729a217447c
-
Force 1D range-only, volatility-scaled ML caps, sector-mode Top Picks, self-learning throttle + collapsible panel
Khanna, Videh Rakesh Rakesh- predictor_core/ml_predictor: force 1D predictions to range-only NEUTRAL (ATR-scaled ±3.5%) in both AI and ML paths; 1D next-day direction ceiling ~74% made directional targets unreachable. - ml_predictor/infer.py: replace fixed ML range caps with volatility-scaled caps (mult x stock ATR%, clamped floor/ceiling); ML_DISABLE_RANGE_CAP toggle. Add low-history guard (<250 bars -> confidence downgrade + low_history flag) for IPOs like BLUESTONE. A/B backtest confirms cap ON == cap OFF on normal-vol stocks (pure outlier guard). - sector_pulse/top5_picker: sector-volatility-driven Top Picks candidate pool with AI-only selection (TOP5_SECTOR_MODE); get_sector_volatility + constituents. - self_learning/app.py: throttle expensive LLM calibration to every SELF_LEARN_LLM_INTERVAL_DAYS (default 7); reuse prior notes between refreshes while cheap target/bucket stats + learnings.json still refresh each validation; manual Improve AI forces refresh. - frontend: collapsible Self-Learning Insights pane…17b15cc -
Intraday 1% minimum: floor best-case to 1% (AI+ML+core) instead of skipping sub-1% calls — intraday routinely clears 1%
Khanna, Videh Rakesh Rakesh7a66a03 -
Intraday: 1% min-move gate (AI+ML), pre-market preview, momentum override, top-pick INTRADAY auto-refresh; retire 3D/5D from live view+API
Khanna, Videh Rakesh Rakeshac1382f
-
ML: add indicator + Monte Carlo features, retrain model
Khanna, Videh Rakesh RakeshAdd 12 extra point-in-time features (Bollinger bandwidth/squeeze, Keltner & Donchian channel position, Stochastic %K, CCI, MFI, Williams %R, realized vol, and Monte-Carlo bootstrap hit-probability/excursion estimates), now the production default (opt-out ML_EXTRA_FEATURES=0). Shared helpers in features.py keep live inference and the training augmenter identical (no train/serve skew). Validated on the 5-month OOS holdout (research/ml_backtest.py, 38,376 rows): INTRADAY direction 71->82%, price-target hit 64->84%, est-high MAE 1.54->1.07; 1D/3D per-trade P&L +0.86->+1.02 / +1.29->+1.51 (PF 1.74->1.93 / 1.79->1.96), no regression. Retrained the committed 49-feature model. Adds research/augment_features.py (vectorized CSV augmenter reusing the shared feature helpers).c789230 -
ML confidence + honest relative-direction display
Khanna, Videh Rakesh Rakesh- Per-TF conf_hi: INTRADAY HIGH=0.72 (~85% dir acc, OOS 25k); 1D/3D HIGH disabled (conf_hi=1.01) since 75-85% is unreachable at those horizons (max proba ~0.63). - Record excess_labels in manifest; infer.py emits per-TF dir_basis. - UI: 1D/3D shown as OUTPERFORM/UNDERPERFORM (vs Nifty), not absolute BULLISH/BEARISH; ML/AI 'split' softened to 'different axes' for relative-vs-absolute calls. - Cache-buster v=20260723b.dffb9e9
-
Fix ML unavailable on HF Spaces: research/ excluded from image
Khanna, Videh Rakesh Rakeshml_predictor/infer.py imported research.range_model in the NEUTRAL branch, but .dockerignore excluded research/ from the HF image -> ModuleNotFoundError -> /api/ml-predict 500 -> generic 'ML unavailable' on every card. - infer.py: add resilient _neutral_range() with inline fallback (never crash) - .dockerignore: ship research/range_model.py (pure prod dependency)b615c1e -
Fix ML n/a: dedupe duplicate ticker columns in OHLCV cache
Khanna, Videh Rakesh RakeshRoot cause of '🤖 ML n/a' on 1D/3D for many stocks: 556 of 3658 OHLCV cache rows (all 2y period — the horizon ml_predictor fetches) were pickled with the ticker column present twice (one sparse ~17-row column + the full series). In infer.py, sc[ticker].dropna() intersected the duplicates and collapsed history to ~17 bars → false 'insufficient history' → available:false. AI was unaffected because predictor_core fetches 1y. - data_sources._dedupe_cols(): merge duplicate columns row-wise (first non-null wins). Applied in _load_sql_cache (repairs legacy rows on read — fixes all 556 live) and _save_sql_cache (prevents re-persisting duplicates; rows self-heal on their next daily refresh). - ml_predictor/infer._series(): defensive guard so a duplicate column can never shrink history via dropna() again. Verified: TATASTEEL/ACE/ALEMBICLTD/ALKYLAMINE/AFCONS now return valid 1D/3D.e615107 -
Fix AI conflicts: persistent retry (no terminal 'AI unavailable'), consistent direction/reasoning, live-anchored 1D/3D targets + gap note
Khanna, Videh Rakesh Rakesh- AI outages are now always retryable (never terminal ai_unavailable); frontend keeps retrying per-cell until a provider frees up. ML shown independently, no ML substitution. - Reconcile guardrail overrides: reasoning + should_buy now match the final (post-guardrail) direction, ending BULLISH-badge/neutral-reasoning contradictions. - Re-anchor 1D/3D entry+targets to live price when available (matches AI-forecast anchor); add 'gapped past target' card note when live price already ran past the close-anchored move.f7bbcd1 -
ML model retrain + research updates + news/llm tweaks
Khanna, Videh Rakesh Rakesh- Retrain ml_predictor quantile/direction models + manifest - dataset/features/infer/train updates - Add research/ml_intraday_backtest.py + results; update ml selection/watchlist evals - llm_client and news_sentiment adjustments8af7f2e
-
Fix empty universe + full-market top picks; drop dead NSE price APIs
Khanna, Videh Rakesh RakeshUniverse (universe.py): - Source the WHOLE NSE market from NSE's official EQUITY_L.csv (~2,062 EQ stocks, all cap tiers incl. mid/small-caps) instead of Yahoo's top-500 screener, which is blocked on HF datacenter IPs (returned 0 -> empty universe -> top picks stuck 'Analysing...'). Chain: EQUITY_L -> Nifty-500 CSV -> Yahoo screener -> static fallback (nse_fallback_universe.py). - Persist cache to /data on HF (survives restarts). Top picks (top5_picker.py): - Scan draws from the whole market, capped at 700/run, ordered cache-first (data_sources.cached_tickers) + rotating cold tail so mid/small-caps are included and the market is swept over ~3 runs. Phase 1 is signals-only. Data sources (data_sources.py, app.py): - Remove dead NSE OHLCV API, NSE live-quote, and jugaad NSELive (all HTTP 403 bot-blocked even locally) to stop wasting a warmup+request per fetch. - OHLCV order simplified to yfinance-first. - Fix _is_price_fresh (was always-False via dead probes -> stop-loss ch…b0d9ebf