Deploy MYTHOS application

README.md

@@ -0,0 +1,112 @@
+---
+title: MYTHOS · MAYTHOS × RINKING
+emoji: 🚀
+colorFrom: indigo
+colorTo: purple
+sdk: gradio
+sdk_version: 5.50.0
+python_version: "3.11"
+app_file: app.py
+pinned: false
+---
+
+# MYTHOS · MAYTHOS × RINKING
+
+Selectable, reproducible forecast-and-signal dashboard with **Live Signal** and
+**Backtesting** tabs. It fetches real closed OHLCV candles, never fabricates a
+missing candle, and writes a data snapshot plus manifest for every completed run.
+
+> Educational / research project — not financial advice. Trading carries
+> substantial risk of loss. No model can guarantee correct predictions.
+
+## What changed
+
+- Timeframes: **1m, 2m, 5m, 10m, 15m, 30m, 1h, 2h**.
+- Market and symbol selection for **Forex and Crypto**.
+- Data adapters:
+  - Yahoo Finance (`EURUSD=X`, `BTC-USD`, and other Yahoo tickers).
+  - Binance Spot crypto catalog and klines (`BTCUSDT` means USDT, not USD).
+  - Twelve Data Forex catalog/history when `TWELVE_DATA_API_KEY` is set.
+- Models: **ARIMA, Auto-ARIMA, ARIMA-GARCH, Moirai, TimesFM**.
+- **Single model** mode or **Parallel models** mode. Every model keeps its own
+  result and error; a failed model is never replaced by another model's output.
+- One configuration is shared by Live Signal and Backtesting.
+- Walk-forward backtest uses training data only at each origin, checks the
+  selected candle duration, skips gap-crossing targets, reports failures and
+  coverage, and includes a last-close persistence baseline.
+- Live results are blocked when the latest closed candle is stale. The MAYTHOS
+  heuristic and forecast metrics are shown separately.
+- Actionable live signals include causal Entry, Stop Loss, Take Profit and
+  Risk:Reward levels. Stop Loss uses Wilder ATR plus prior closed-candle
+  structure; Take Profit uses the opposite structure and a compatible real
+  forecast target when available.
+- Backtesting resolves those same levels against each future candle's high/low,
+  treats a same-candle SL/TP collision as SL-first, and reports trades, win
+  rate, profit factor, expectancy in R, maximum drawdown in R and average R.
+  Spread, fee and slippage assumptions are explicit in the UI and manifest.
+- Forecast trades use a strict cost-aware execution gate: the predicted target
+  must be the actual target used for the selected horizon, clear at least 0.20
+  ATR of noise, clear a 1.20 R:R, and exceed 1.25× the configured round-trip
+  cost. A forecast that does not clear those hurdles is reported as a valid
+  forecast but is not traded. This intentionally produces fewer trades and
+  avoids converting tiny mean forecasts into distant swing-level timeout losses.
+- MAYTHOS execution is restricted to fully confirmed `strong` states; weak and
+  moderate heuristic states remain visible for research but are not treated as
+  profitable trade recommendations.
+- Every result can be downloaded as a ZIP containing candles, results, config,
+  package versions, SHA-256 snapshot hash, and evaluation policy.
+
+## Windows setup
+
+1. Install **Python 3.11** from python.org and enable the Python launcher.
+2. Extract `mythos.zip` to a normal folder.
+3. Run `setup_windows.bat`.
+4. Run `start_windows.bat`; open the local Gradio address shown in the console.
+5. Optional: run `install_foundation_windows.bat` for real Moirai and TimesFM
+   inference. Their Hugging Face weights download on first use and need internet,
+   disk space, and adequate RAM. If these packages/weights are unavailable, the
+   dashboard reports that model as failed rather than showing a substitute.
+
+The core install is sufficient for ARIMA, Auto-ARIMA, and ARIMA-GARCH.
+
+## Provider notes
+
+Provider retention, plan limits, exchange maintenance, market closures, and
+network access affect the available history. The dashboard displays the exact
+provider, UTC range, candle count, freshness, and gap count. Forex does not have
+a single consolidated traded-volume feed; zero/unavailable volume is explicitly
+identified and MAYTHOS uses its documented price-based proxy.
+
+Yahoo's free intraday history can be limited. Binance is crypto-only. Twelve Data
+requires an API key in the Windows environment before launching:
+
+```text
+set TWELVE_DATA_API_KEY=your_key_here
+start_windows.bat
+```
+
+Do not put API keys into source files or ZIP archives.
+
+## Command-line logger
+
+The logger uses the same data/model pipeline and verifies predictions only after
+the target closed candle arrives:
+
+```text
+python live_runner.py --provider "Yahoo Finance" --market Crypto --symbol BTC-USD --timeframe 5m --models ARIMA-GARCH --poll-seconds 60
+python live_runner.py --provider "Binance Spot" --market Crypto --symbol BTCUSDT --timeframe 2m --models ARIMA TimesFM --poll-seconds 120
+```
+
+## Verification
+
+Core tests use deterministic synthetic fixtures for alignment and contract
+checks. Foundation models are not silently tested with fake forecasts: their
+optional tests require downloading the real weights and can take a long time.
+Run the installed suite with:
+
+```text
+python -m pytest -q
+```
+
+Backtest accuracy is a historical measurement of this configuration and sample;
+it is not a promise of future accuracy, profitability, or execution quality.

STRATEGY_NOTES.md

@@ -0,0 +1,44 @@
+# MYTHOS strategy audit and execution policy
+
+## What the supplied result showed
+
+The supplied EURUSD 5-minute report had directional accuracy around 37–47%,
+but the trade layer was materially worse than the forecast layer. Forecasts
+were usually only a few fractions of a pip away from the current close. The
+old level builder nevertheless selected a farther prior-swing target whenever
+it was farther than the model forecast. With a one-candle horizon, those
+orders could not realistically reach that target and became timeout trades.
+MAYTHOS also allowed weak and moderate heuristic states into the actionable
+trade denominator; those 16 trades had zero wins in the supplied sample.
+
+## Changes in this build
+
+1. A forecast target is now used as the target for a forecast trade. It is not
+   silently extended to a more distant swing level.
+2. Forecast execution requires a positive-risk target, at least `1.20R`, at
+   least `0.20 ATR` of predicted reward, and at least `1.25x` the configured
+   round-trip spread/fee/slippage cost.
+3. Forecast trades use an ATR stop without an oversized structural stop. The
+   prior structure remains available for causal analysis and for trades that
+   do not have a model target.
+4. MAYTHOS backtest execution is limited to `strong`, unblocked states with
+   confidence at or above 0.69. Weak and moderate states remain visible but
+   cannot be counted as executable trades.
+5. Existing candle-path rules remain conservative: if both stop and target
+   are touched in one OHLC candle, stop loss is resolved first.
+
+## Recommended validation workflow
+
+- Use at least 90 days of data where the provider permits it, and evaluate
+  several symbols and time periods rather than relying on one 100-origin run.
+- For 5-minute Forex data, use a horizon of several candles and realistic
+  broker costs. A one-candle forecast whose expected move is below costs is a
+  no-trade condition, not a profitable opportunity.
+- Require positive expectancy after costs and a stable profit factor in
+  separate walk-forward periods before considering paper execution.
+- Treat an empty trade set as a diagnostic result: lower-cost execution,
+  longer horizon, deeper history, or a better forecast is needed. Do not
+  disable the execution gate merely to increase trade count.
+
+Historical backtest results cannot guarantee future profitability or execution
+quality. This project remains an educational/research tool.

app.py

@@ -0,0 +1,245 @@
+from __future__ import annotations
+
+import os
+
+import gradio as gr
+import pandas as pd
+import plotly.graph_objects as go
+
+from config import DEFAULT_CRYPTO_SYMBOLS, DEFAULT_FOREX_SYMBOLS, MODEL_NAMES, PROVIDERS, TIMEFRAMES
+from data.providers import catalog
+from pipeline import RunConfig, backtest_result, export_run, live_result, model_readiness
+
+DISCLAIMER = '**Educational / research project — not financial advice.** Trading carries substantial risk of loss.'
+CSS = """
+.gradio-container { max-width: 1440px !important; }
+#brand { padding: 18px 6px 12px; }
+#brand h1 { letter-spacing: .05em; font-size: 2.1rem; }
+#brand p { max-width: 1000px; }
+.settings-panel { border-radius: 14px; }
+footer { display: none !important; }
+"""
+
+
+def server_port():
+    raw = os.getenv('PORT', os.getenv('MYTHOS_PORT', '7860'))
+    try:
+        port = int(raw)
+    except (TypeError, ValueError):
+        raise RuntimeError('PORT/MYTHOS_PORT must be an integer between 1 and 65535.') from None
+    if not 1 <= port <= 65535:
+        raise RuntimeError('PORT/MYTHOS_PORT must be an integer between 1 and 65535.')
+    return port
+
+
+def ui_config(provider, market, symbol, timeframe, mode, models, days, window, horizon, step, evaluations, features, spread_bps, fee_bps, slippage_bps):
+    return RunConfig(provider=provider, market=market, symbol=symbol, timeframe=timeframe,
+                     mode=mode, models=tuple(models or ()), lookback_days=days, window=window,
+                     horizon=horizon, step=step, max_evaluations=evaluations, use_features=features,
+                     spread_bps=spread_bps, fee_bps=fee_bps, slippage_bps=slippage_bps)
+
+
+def metadata_text(metadata):
+    freshness = '**STALE / market closed — historical analysis only; live execution blocked.**' if metadata['stale'] else '**Recent closed-candle data.**'
+    details = metadata['provider_details']
+    effective = details.get('effective_lookback_days')
+    coverage = f' Provider lookback used: **{effective} calendar days** (provider retention cap applies).' if effective else ''
+    volume_note = details.get('volume_note', '')
+    if metadata['market'] == 'Forex' and not volume_note:
+        volume_note = 'Forex volume may be unavailable/zero; MAYTHOS then uses its price-based proxy.'
+    return (
+        f"**{metadata['provider']} · {metadata['symbol']} · {metadata['timeframe']}** — {metadata['closed_candles']:,} closed candles. {freshness}\n\n"
+        f"UTC range: `{metadata['first_candle_open_utc']}` → `{metadata['last_candle_open_utc']}` (candle-open timestamps). "
+        f"Last candle closed: `{metadata['last_candle_close_utc']}`. "
+        f"Noncontinuous intervals: **{metadata['noncontinuous_intervals']}**; gaps are never filled."
+        + coverage + ('\n\n' + volume_note if volume_note else '')
+    )
+
+
+def make_candlestick_figure(result):
+    frame = result['frame'].tail(200)
+    config = result['config']
+    figure = go.Figure(go.Candlestick(x=frame.index, open=frame['Open'], high=frame['High'],
+                                    low=frame['Low'], close=frame['Close'], name=config.symbol))
+    for forecast in result['forecasts']:
+        if not forecast['forecast_path']:
+            continue
+        times = [frame.index[-1]] + [frame.index[-1] + config.candle_step * (i + 1) for i in range(config.horizon)]
+        figure.add_trace(go.Scatter(x=times, y=[float(frame['Close'].iloc[-1])] + forecast['forecast_path'],
+                                   name=forecast['Model'], mode='lines+markers'))
+    figure.update_layout(title=f'{config.symbol} · {config.timeframe} · {config.provider}',
+                         xaxis_rangeslider_visible=False, height=510,
+                         margin=dict(l=12, r=12, t=50, b=12), template='plotly_dark',
+                         legend=dict(orientation='h'), xaxis_title='Candle open (UTC)', yaxis_title='Price in pair quote currency')
+    return figure
+
+
+def run_live_signal(*args):
+    empty = pd.DataFrame()
+    try:
+        config = ui_config(*args)
+        result = live_result(config)
+        signal = result['signal']
+        if 'error' in signal:
+            signal_rows = [('Engine status', 'FAILED'), ('Error', signal['error'])]
+        else:
+            signal_rows = [
+                ('Direction', signal['direction']), ('Execution suitability', signal['execution_suitability']),
+                ('Live actionable', str(signal['live_actionable'])),
+                ('Forecast coverage', f"{signal.get('forecast_models_completed', 0)}/{signal.get('forecast_models_requested', len(config.models))} models"),
+                ('Forecast consensus', signal.get('forecast_consensus', 'UNAVAILABLE')),
+                ('Forecast agrees with MAYTHOS', str(signal.get('forecast_agrees_with_signal', False))),
+                ('Heuristic confidence (not calibrated win probability)', f"{signal['confidence']:.2%}"),
+                ('Market state', signal['market_state']), ('Regime', signal['regime_label']),
+                ('Reason', signal['reason_summary']),
+            ]
+            if signal.get('live_actionable') and signal.get('trade_levels'):
+                levels = signal['trade_levels']
+                signal_rows.extend([
+                    ('Entry', levels['entry']),
+                    ('Stop Loss', levels['stop_loss']),
+                    ('Take Profit', levels['take_profit']),
+                    ('Risk:Reward', f"1:{levels['risk_reward']:.3f}"),
+                    ('ATR', levels['atr']),
+                    ('Level basis', levels['level_basis']),
+                ])
+        successful = sum(bool(item['forecast_path']) for item in result['forecasts'])
+        summary = (
+            f"### {successful}/{len(config.models)} forecast models completed\n\n"
+            f"Forecast horizon: **{config.horizon} candle(s)** · training window: **{config.window} candles** · "
+            f"features: **{'enabled' if config.use_features else 'disabled'}**. Each model is reported individually.\n\n"
+            + metadata_text(result['metadata'])
+            + f"\n\nForecast coverage is **{result['signal'].get('forecast_coverage_pct', 0):.2f}%**; a live signal is actionable only when data is fresh, MAYTHOS is sufficiently strong, selected models agree with MAYTHOS, and every required forecast condition is met."
+        )
+        artifact = export_run(result, 'live')
+        table = pd.DataFrame(result['forecasts']).drop(columns=['forecast_path'])
+        return make_candlestick_figure(result), summary, table, pd.DataFrame(signal_rows, columns=['MAYTHOS metric', 'Value']), artifact
+    except Exception as error:
+        return None, f'**Live signal failed:** {error}', empty, empty, None
+
+
+def run_backtest_ui(*args):
+    empty = pd.DataFrame()
+    try:
+        config = ui_config(*args)
+        result = backtest_result(config)
+        failed = int(result['forecast_summary']['failed_predictions'].sum())
+        status = (
+            f"### Walk-forward evaluation completed — {failed} failed/skipped model evaluations\n\n"
+            f"History window: **{config.window}** · horizon: **{config.horizon}** · origin stride: **{config.step}** · "
+            f"at most **{config.max_evaluations} most-recent evaluation origins per model**. "
+            'Orders are reselected using only each training window; no future target prices/features are supplied to models. '
+            'Targets crossing gaps/market closures are explicitly skipped. Flat prices are classified as FLAT.\n\n'
+            '**Accuracy denominator:** successful forecast evaluations; failures and coverage are shown separately. '
+            '`persistence_mae` is the last-close naive baseline on the same valid targets. '
+            '**Trade execution is selective:** forecast targets must clear volatility noise, minimum 1.20R reward/risk, and 1.25× the configured round-trip costs; otherwise the forecast remains visible but no trade is opened. MAYTHOS uses only strong, unblocked signals. Candle-path resolution checks SL before TP when both touch; costs are explicit spread/fee/slippage assumptions, not provider-measured quotes.**\n\n'
+            + metadata_text(result['metadata'])
+        )
+        artifact = export_run(result, 'backtest')
+        return result['forecast_summary'], result['signal_summary'], result['forecast_details'], result['signal_details'], status, artifact
+    except Exception as error:
+        return empty, empty, empty, empty, f'**Backtest failed:** {error}', None
+
+
+def default_symbols(provider, market):
+    if provider == 'Binance Spot':
+        if market != 'Crypto':
+            return gr.Dropdown(choices=[], value=None, allow_custom_value=True), 'Binance Spot requires Crypto. Select another provider for Forex.'
+        return gr.Dropdown(choices=['BTCUSDT', 'ETHUSDT', 'SOLUSDT', 'BTCUSDC', 'ETHUSDC'], value='BTCUSDT', allow_custom_value=True), 'Binance Spot: load the full active exchange catalog. BTCUSDT is quoted in USDT, not USD.'
+    if provider == 'Twelve Data':
+        pairs = [s.removesuffix('=X')[:3] + '/' + s.removesuffix('=X')[3:] for s in DEFAULT_FOREX_SYMBOLS]
+        return gr.Dropdown(choices=pairs if market == 'Forex' else [], value='EUR/USD' if market == 'Forex' else None, allow_custom_value=True), 'Twelve Data Forex: set TWELVE_DATA_API_KEY before starting. Data access and historical depth depend on your plan.'
+    choices = DEFAULT_FOREX_SYMBOLS if market == 'Forex' else DEFAULT_CRYPTO_SYMBOLS
+    return gr.Dropdown(choices=choices, value=choices[0], allow_custom_value=True), 'Yahoo: search or type any supported Forex/Crypto symbol. Free intraday data can be delayed, sparse, or unavailable; use timestamps to assess freshness.'
+
+
+def search_symbols(provider, market, query, current):
+    try:
+        symbols, note = catalog(provider, market, query)
+        if current and current not in symbols:
+            symbols = [current] + symbols
+        return gr.Dropdown(choices=symbols, value=current if current else (symbols[0] if symbols else None), allow_custom_value=True), note
+    except Exception as error:
+        return gr.skip(), f'**Catalog/search failed:** {error}'
+
+
+def update_mode(mode, selected):
+    selected = selected or ['ARIMA-GARCH']
+    return gr.CheckboxGroup(value=selected[-1:] if mode == 'Single model' else selected)
+
+
+def enforce_single(mode, selected):
+    if mode == 'Single model' and selected and len(selected) > 1:
+        return gr.CheckboxGroup(value=selected[-1:])
+    return gr.skip()
+
+
+THEME = gr.themes.Soft(primary_hue='violet', neutral_hue='slate')
+
+with gr.Blocks(title='MYTHOS · MAYTHOS × RINKING', theme=THEME, css=CSS) as demo:
+    gr.Markdown('# MYTHOS · MAYTHOS × RINKING\nReal market data. Selectable forecasting. Auditable walk-forward evaluation.\n\n' + DISCLAIMER, elem_id='brand')
+    with gr.Group(elem_classes=['settings-panel']):
+        gr.Markdown('### Market & model configuration\nThese settings apply to both Live Signal and Backtesting.')
+        with gr.Row():
+            provider = gr.Dropdown(PROVIDERS, value='Yahoo Finance', label='Data provider')
+            market = gr.Radio(['Forex', 'Crypto'], value='Crypto', label='Market')
+            symbol = gr.Dropdown(DEFAULT_CRYPTO_SYMBOLS, value='BTC-USD', allow_custom_value=True, filterable=True, label='Symbol — choose or type')
+            timeframe = gr.Dropdown(TIMEFRAMES, value='5m', label='Timeframe')
+        with gr.Row():
+            query = gr.Textbox(label='Symbol search / catalog filter', placeholder='EUR, BTC, USDT…')
+            search_button = gr.Button('Load / search symbol catalog')
+        catalog_note = gr.Markdown('Yahoo common pairs shown. Search or type any provider-supported symbol; intraday coverage is checked on fetch.')
+        with gr.Row():
+            mode = gr.Radio(['Single model', 'Parallel models'], value='Single model', label='Model execution')
+            models = gr.CheckboxGroup(MODEL_NAMES, value=['ARIMA-GARCH'], label='Forecast models — select freely')
+        with gr.Accordion('History, forecast & backtest settings', open=True):
+            with gr.Row():
+                days = gr.Number(value=30, minimum=1, maximum=730, precision=0, label='Lookback (calendar days)')
+                window = gr.Number(value=200, minimum=32, maximum=2048, precision=0, label='Training window (candles)')
+                horizon = gr.Number(value=1, minimum=1, maximum=64, precision=0, label='Forecast horizon (candles)')
+                features = gr.Checkbox(value=False, label='Use 14 causal forecast features')
+            with gr.Row():
+                step = gr.Number(value=30, minimum=1, maximum=10000, precision=0, label='Backtest origin stride (candles)')
+                evaluations = gr.Number(value=30, minimum=1, maximum=1000, precision=0, label='Maximum recent evaluations / model')
+            with gr.Row():
+                spread_bps = gr.Number(value=1.0, minimum=0, maximum=1000, label='Assumed spread (bps)')
+                fee_bps = gr.Number(value=1.0, minimum=0, maximum=1000, label='Assumed fees (bps, round trip)')
+                slippage_bps = gr.Number(value=1.0, minimum=0, maximum=1000, label='Assumed slippage (bps)')
+            gr.Markdown('ARIMA-family order selection runs at every evaluation origin and may take several minutes. SL/TP uses closed-candle Wilder ATR plus prior swing structure; levels are omitted when no valid positive-risk target exists. Parallel mode runs independent models concurrently (two workers). Future indicator covariates use the last observed values. Moirai/TimesFM require their real pretrained weights.')
+        with gr.Accordion('Model dependency status', open=False):
+            gr.Dataframe(value=model_readiness(), interactive=False)
+            gr.Markdown('“Packages detected” is not an inference or accuracy certification. See README for installing optional foundation models.')
+
+    inputs = [provider, market, symbol, timeframe, mode, models, days, window, horizon, step, evaluations, features, spread_bps, fee_bps, slippage_bps]
+    with gr.Tab('Live Signal'):
+        live_button = gr.Button('Refresh live signal', variant='primary')
+        live_summary = gr.Markdown('Choose your configuration and refresh to fetch real closed candles.')
+        live_plot = gr.Plot(label='Closed candles & model forecasts')
+        live_table = gr.Dataframe(label='Per-model forecast results and errors', interactive=False)
+        signal_table = gr.Dataframe(label='MAYTHOS signal engine', interactive=False)
+        live_file = gr.File(label='Download live data, results & reproducibility manifest', interactive=False)
+    with gr.Tab('Backtesting'):
+        backtest_button = gr.Button('Run walk-forward backtest', variant='primary')
+        backtest_status = gr.Markdown('Use the same market/model configuration above. Download the exact data snapshot with every completed run.')
+        forecast_summary = gr.Dataframe(label='Forecast metrics — each model separately', interactive=False)
+        maythos_summary = gr.Dataframe(label='MAYTHOS metrics — unblocked signals only', interactive=False)
+        forecast_details = gr.Dataframe(label='Forecast per-evaluation details (including failures)', interactive=False)
+        maythos_details = gr.Dataframe(label='MAYTHOS per-evaluation details', interactive=False)
+        backtest_file = gr.File(label='Download backtest data, results & reproducibility manifest', interactive=False)
+
+    provider.change(default_symbols, inputs=[provider, market], outputs=[symbol, catalog_note], queue=False)
+    market.change(default_symbols, inputs=[provider, market], outputs=[symbol, catalog_note], queue=False)
+    search_button.click(search_symbols, inputs=[provider, market, query, symbol], outputs=[symbol, catalog_note], api_name='symbol_catalog')
+    mode.change(update_mode, inputs=[mode, models], outputs=models, queue=False)
+    models.change(enforce_single, inputs=[mode, models], outputs=models, queue=False)
+    live_button.click(run_live_signal, inputs=inputs, outputs=[live_plot, live_summary, live_table, signal_table, live_file], concurrency_limit=1, concurrency_id='analysis', api_name='live_signal')
+    backtest_button.click(run_backtest_ui, inputs=inputs, outputs=[forecast_summary, maythos_summary, forecast_details, maythos_details, backtest_status, backtest_file], concurrency_limit=1, concurrency_id='analysis', api_name='backtest')
+
+
+if __name__ == '__main__':
+    demo.queue(max_size=16).launch(
+        server_name=os.getenv('GRADIO_SERVER_NAME', '0.0.0.0'),
+        server_port=server_port(),
+        inbrowser=False,
+        share=False,
+    )

backtest/__init__.py

@@ -0,0 +1 @@
+pass

backtest/backtester.py

@@ -0,0 +1,266 @@
+from math import floor, log10
+import numpy as np
+import pandas as pd
+from features.feature_pipeline import slice_features
+from risk_management import CostModel, build_trade_levels, simulate_trade, summarize_trades
+from utils.helpers import price_direction
+
+
+def _round_sig(x, sig: int = 6):
+    if x is None or not np.isfinite(x) or x == 0:
+        return x
+    decimals = sig - 1 - floor(log10(abs(x)))
+    return round(x, decimals)
+
+
+def backtest_model(
+    df: pd.DataFrame,
+    model,
+    window: int = 100,
+    horizon: int = 1,
+    step: int = 1,
+    features_df: pd.DataFrame = None,
+    max_evaluations: int = None,
+    timeframe: str = None,
+    market: str = 'Crypto',
+    symbol: str = '',
+    costs: CostModel = None,
+) -> pd.DataFrame:
+    if any(isinstance(value, (bool, np.bool_)) or not isinstance(value, (int, np.integer)) or value < 1 for value in (window, horizon, step)):
+        raise ValueError('Window, horizon, and step must be positive integers.')
+    if max_evaluations is not None and (isinstance(max_evaluations, (bool, np.bool_)) or not isinstance(max_evaluations, (int, np.integer)) or max_evaluations < 1):
+        raise ValueError('max_evaluations must be a positive integer.')
+    if features_df is not None and len(features_df) != len(df):
+        raise ValueError('Features must align with every OHLCV row.')
+    if df.empty or len(df) < window + horizon or not df.index.is_monotonic_increasing or df.index.duplicated().any():
+        raise ValueError('Backtest candles must be non-empty, chronologically ordered, and unique.')
+    required = {'Open', 'High', 'Low', 'Close'}
+    if not required.issubset(df.columns):
+        raise ValueError(f'Backtest candles are missing columns: {sorted(required - set(df.columns))}.')
+    ohlc = df[['Open', 'High', 'Low', 'Close']].apply(pd.to_numeric, errors='coerce')
+    if (not np.isfinite(ohlc.to_numpy(dtype=float)).all() or (ohlc <= 0).any().any() or
+            (ohlc['High'] < ohlc[['Open', 'Close', 'Low']].max(axis=1)).any() or
+            (ohlc['Low'] > ohlc[['Open', 'Close', 'High']].min(axis=1)).any()):
+        raise ValueError('Backtest candles contain invalid/non-finite OHLC observations.')
+    df = df.copy()
+    df[['Open', 'High', 'Low', 'Close']] = ohlc.astype(float)
+    if not np.isfinite(pd.to_numeric(df['Close'], errors='coerce').to_numpy(dtype=float)).all():
+        raise ValueError('Backtest Close prices must be finite real observations.')
+    close = df["Close"]
+    costs = costs or CostModel()
+    timestamps = [
+        ts.isoformat() if hasattr(ts, "isoformat") else ts for ts in df.index
+    ]
+    last_i = len(close) - horizon
+    rows = []
+    positions = list(range(window, last_i + 1, step))
+    if max_evaluations is not None:
+        if max_evaluations < 1:
+            raise ValueError('max_evaluations must be positive.')
+        positions = positions[-max_evaluations:]
+    for i in positions:
+        history = close.iloc[i - window : i]
+        feature_window = (
+            slice_features(features_df, i - window, i)
+            if features_df is not None
+            else None
+        )
+        current_close = float(close.iloc[i - 1])
+        target_idx = i + horizon - 1
+        actual_next_close = float(close.iloc[target_idx])
+        try:
+            continuous = True
+            if timeframe:
+                from utils.helpers import MINUTES_PER_CANDLE
+                expected_step = pd.Timedelta(minutes=MINUTES_PER_CANDLE[timeframe])
+                target_times = df.index[i - 1:target_idx + 1]
+                continuous = bool((target_times.to_series().diff().dropna() == expected_step).all())
+                if not continuous:
+                    raise ValueError('Target crosses a missing candle or market closure; not a continuous timeframe horizon.')
+            forecast = model.predict(history, horizon=horizon, features=feature_window)
+            if len(forecast) != horizon:
+                raise ValueError('Model returned an incorrect forecast horizon.')
+            forecast_values = np.asarray(forecast, dtype=float)
+            if not np.isfinite(forecast_values).all() or (forecast_values <= 0).any():
+                raise ValueError(
+                    f"forecast contains a non-finite or non-positive value: {forecast!r}"
+                )
+            predicted_next_close = float(forecast_values[-1])
+        except Exception as e:
+            rows.append(
+                {
+                    "datetime": timestamps[target_idx],
+                    "current_close": current_close,
+                    "predicted_close": np.nan,
+                    "actual_close": actual_next_close,
+                    "predicted_direction": None,
+                    "actual_direction": None,
+                    "correct": None,
+                    "continuous_target": continuous,
+                    "trade_taken": False,
+                    "entry": np.nan, "stop_loss": np.nan, "take_profit": np.nan,
+                    "risk_reward": np.nan, "atr": np.nan, "outcome": 'FORECAST_FAILED',
+                    "exit_price": np.nan, "net_r": np.nan, "bars_held": 0,
+                    "error": str(e),
+                }
+            )
+            continue
+        predicted_direction = price_direction(predicted_next_close, current_close)
+        actual_direction = price_direction(actual_next_close, current_close)
+        correct = predicted_direction == actual_direction
+        trade_direction = {'up': 'BUY', 'down': 'SELL'}.get(predicted_direction)
+        try:
+            levels = build_trade_levels(
+                df.iloc[:i], trade_direction, market, symbol,
+            forecast_target=predicted_next_close,
+            # Only trade a forecast that clears volatility noise and has a
+            # target the model actually predicted. Previously a tiny forecast
+            # was replaced by a distant swing target, creating timeout losses
+            # unrelated to the forecast signal.
+            forecast_only=True,
+            minimum_edge_atr=0.20,
+            minimum_risk_reward=1.20,
+            use_structure_stop=False,
+            minimum_edge_bps=(
+                costs.spread_bps + costs.fee_bps + 2.0 * costs.slippage_bps
+            ) * 1.25,
+            )
+            trade = {'outcome': 'NO_LEVELS', 'exit_price': None, 'net_r': None, 'bars_held': 0}
+            if levels is not None:
+                trade = {**trade, **simulate_trade(levels, df.iloc[i:target_idx + 1], costs)}
+            trade_error = None
+        except Exception as error:
+            levels = None
+            trade = {'outcome': 'TRADE_FAILED', 'exit_price': None, 'net_r': None, 'bars_held': 0}
+            trade_error = f'trade execution failed: {error}'
+        rows.append(
+            {
+                "datetime": timestamps[target_idx],
+                "current_close": current_close,
+                "predicted_close": predicted_next_close,
+                "actual_close": actual_next_close,
+                "predicted_direction": predicted_direction,
+                "actual_direction": actual_direction,
+                "correct": correct,
+                "continuous_target": continuous,
+                "trade_taken": levels is not None,
+                "entry": levels.entry if levels else np.nan,
+                "stop_loss": levels.stop_loss if levels else np.nan,
+                "take_profit": levels.take_profit if levels else np.nan,
+                "risk_reward": levels.risk_reward if levels else np.nan,
+                "atr": levels.atr if levels else np.nan,
+                "swing_support": levels.swing_support if levels else np.nan,
+                "swing_resistance": levels.swing_resistance if levels else np.nan,
+                "outcome": trade.get('outcome'),
+                "exit_price": trade.get('exit_price'),
+                "net_r": trade.get('net_r'),
+                "bars_held": trade.get('bars_held', 0),
+                "error": trade_error,
+            }
+        )
+    return pd.DataFrame(rows)
+
+
+def summarize_backtest(results: pd.DataFrame, model_name: str) -> dict:
+    if results.empty:
+        return {
+            "model": model_name,
+            "total_predictions": 0,
+            "correct": 0,
+            "incorrect": 0,
+            "accuracy_pct": None,
+            "mae": None,
+            "rmse": None,
+            "failed_predictions": 0,
+            "gap_skipped": 0,
+            "nonflat_accuracy_pct": None,
+            "persistence_accuracy_pct": None,
+            **summarize_trades(results),
+        }
+    valid = results.dropna(subset=["correct"])
+    total = len(valid)
+    correct = int(valid["correct"].sum()) if total else 0
+    incorrect = total - correct
+    accuracy = round(100 * correct / total, 2) if total else None
+    price_valid = results.dropna(subset=["predicted_close", "actual_close"])
+    if len(price_valid):
+        errors = price_valid["predicted_close"] - price_valid["actual_close"]
+        mae = float(np.mean(np.abs(errors)))
+        rmse = float(np.sqrt(np.mean(errors**2)))
+    else:
+        mae = rmse = None
+    failed = int(results["error"].notna().sum())
+    nonflat = valid[valid['actual_direction'] != 'flat']
+    nonflat_accuracy = round(100 * float(nonflat['correct'].mean()), 2) if len(nonflat) else None
+    # A last-close persistence forecast predicts no change. Its direction is
+    # therefore FLAT, rather than the direction that the actual target took.
+    persistence_direction = np.full(len(valid), 'flat', dtype=object)
+    persistence_accuracy = round(100 * float(np.mean(persistence_direction == valid['actual_direction'].to_numpy())), 2) if len(valid) else None
+    return {
+        "model": model_name,
+        "total_predictions": total,
+        "correct": correct,
+        "incorrect": incorrect,
+        "accuracy_pct": accuracy,
+        "mae": _round_sig(mae) if mae is not None else None,
+        "rmse": _round_sig(rmse) if rmse is not None else None,
+        "failed_predictions": failed,
+        "gap_skipped": int(results['error'].fillna('').str.contains('gap|closure', case=False, regex=True).sum()),
+        "nonflat_accuracy_pct": nonflat_accuracy,
+        "persistence_accuracy_pct": persistence_accuracy,
+        **summarize_trades(results),
+    }
+
+
+def run_all_models_backtest(
+    df: pd.DataFrame,
+    models: dict,
+    window: int = 100,
+    horizon: int = 1,
+    step: int = 1,
+    features_df: pd.DataFrame = None,
+):
+    per_model_results = {}
+    per_model_summary = {}
+    for name, model in models.items():
+        results = backtest_model(
+            df, model, window=window, horizon=horizon, step=step, features_df=features_df
+        )
+        per_model_results[name] = results
+        per_model_summary[name] = summarize_backtest(results, name)
+    return per_model_results, per_model_summary
+
+
+BASELINE_LABEL = "Close-only"
+FEATURED_LABEL = "+14 Features"
+
+
+def run_comparison_backtest(
+    df: pd.DataFrame,
+    models: dict,
+    features_df: pd.DataFrame,
+    window: int = 100,
+    horizon: int = 1,
+    step: int = 1,
+):
+    per_model_results = {}
+    per_model_summary = {}
+    for name, model_pair in models.items():
+        baseline_model, featured_model = model_pair
+        results_list, summary_list = [], []
+        for label, model, feats in (
+            (BASELINE_LABEL, baseline_model, None),
+            (FEATURED_LABEL, featured_model, features_df),
+        ):
+            results = backtest_model(
+                df, model, window=window, horizon=horizon, step=step, features_df=feats
+            )
+            summary = summarize_backtest(results, name)
+            summary["mode"] = label
+            results = results.copy()
+            results.insert(0, "mode", label)
+            results_list.append(results)
+            summary_list.append(summary)
+        per_model_results[name] = pd.concat(results_list, ignore_index=True)
+        per_model_summary[name] = summary_list
+    return per_model_results, per_model_summary

config.py

@@ -0,0 +1,60 @@
+import os
+
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+# Creation is lazy: importing the application must also work in a read-only
+# checkout. Operators can point runtime artifacts at a persistent writable mount.
+OUTPUT_DIR = os.path.abspath(os.path.expanduser(os.environ.get('MYTHOS_OUTPUT_DIR', os.path.join(BASE_DIR, 'outputs'))))
+
+DEFAULT_FOREX_SYMBOLS = [
+    'EURUSD=X', 'GBPUSD=X', 'USDJPY=X', 'USDCHF=X', 'AUDUSD=X',
+    'USDCAD=X', 'NZDUSD=X', 'EURJPY=X', 'EURGBP=X', 'GBPJPY=X',
+    'AUDJPY=X', 'CADJPY=X', 'CHFJPY=X', 'NZDJPY=X', 'EURCHF=X',
+    'EURCAD=X', 'EURAUD=X', 'EURNZD=X', 'GBPCHF=X', 'GBPCAD=X',
+    'GBPAUD=X', 'GBPNZD=X', 'AUDCAD=X', 'AUDCHF=X', 'AUDNZD=X',
+    'CADCHF=X', 'NZDCAD=X', 'NZDCHF=X', 'USDINR=X', 'USDCNH=X',
+    'USDHKD=X', 'USDSGD=X', 'USDZAR=X', 'USDMXN=X', 'USDTRY=X',
+    'USDSEK=X', 'USDNOK=X', 'USDDKK=X', 'USDPLN=X', 'USDTHB=X',
+]
+DEFAULT_CRYPTO_SYMBOLS = [
+    'BTC-USD', 'ETH-USD', 'BNB-USD', 'XRP-USD', 'SOL-USD',
+    'ADA-USD', 'DOGE-USD', 'DOT-USD', 'LTC-USD', 'LINK-USD',
+]
+ALL_DEFAULT_SYMBOLS = DEFAULT_FOREX_SYMBOLS + DEFAULT_CRYPTO_SYMBOLS
+YF_NATIVE_INTERVALS = {
+    '1m': {'max_days': 30, 'max_request_days': 7},
+    '2m': {'max_days': 60, 'max_request_days': 60},
+    '5m': {'max_days': 60, 'max_request_days': 60},
+    '15m': {'max_days': 60, 'max_request_days': 60},
+    '30m': {'max_days': 60, 'max_request_days': 60},
+    '60m': {'max_days': 730, 'max_request_days': 730},
+    '90m': {'max_days': 60, 'max_request_days': 60},
+    '1d': {'max_days': None, 'max_request_days': None},
+    '1wk': {'max_days': None, 'max_request_days': None},
+    '1mo': {'max_days': None, 'max_request_days': None},
+}
+CUSTOM_TIMEFRAMES = {
+    '1m': {'source': '1m', 'rule': None},
+    '2m': {'source': '2m', 'rule': None},
+    '5m': {'source': '5m', 'rule': None},
+    '10m': {'source': '5m', 'rule': '10min'},
+    '15m': {'source': '15m', 'rule': None},
+    '30m': {'source': '30m', 'rule': None},
+    '1h': {'source': '60m', 'rule': None},
+    '2h': {'source': '60m', 'rule': '2h'},
+    '4h': {'source': '60m', 'rule': '4h'},
+    '1d': {'source': '1d', 'rule': None},
+}
+TIMEFRAMES = ['1m', '2m', '5m', '10m', '15m', '30m', '1h', '2h']
+MODEL_NAMES = ['ARIMA', 'Auto-ARIMA', 'ARIMA-GARCH', 'Moirai', 'TimesFM']
+PROVIDERS = ['Yahoo Finance', 'Binance Spot', 'Twelve Data']
+MOIRAI_CHECKPOINT = 'Salesforce/moirai-1.1-R-small'
+TIMESFM_CHECKPOINT = 'google/timesfm-2.5-200m-pytorch'
+DEFAULT_SYMBOL = 'BTC-USD'
+DEFAULT_TIMEFRAME = '5m'
+DEFAULT_MODEL = 'ARIMA-GARCH'
+DEFAULT_LOOKBACK_DAYS = 30
+DEFAULT_HISTORY_WINDOW = 200
+DEFAULT_HORIZON = 1
+DEFAULT_BACKTEST_STEP = 30
+DEFAULT_MAX_EVALUATIONS = 30
+RANDOM_SEED = 42

data/__init__.py

@@ -0,0 +1 @@
+pass

data/historical.py

@@ -0,0 +1,142 @@
+import random
+import time
+from datetime import datetime, timedelta, timezone
+import pandas as pd
+from config import YF_NATIVE_INTERVALS, CUSTOM_TIMEFRAMES
+from features.feature_pipeline import compute_feature_frame_for_period
+from utils.helpers import feature_warmup_days
+from data.validation import DataFetchError, validate_ohlcv, positive_integer
+
+_YF_TIMEOUT_SECONDS = 30
+OHLCV_COLUMNS = ['Open', 'High', 'Low', 'Close', 'Volume']
+
+def _fetch_yf(symbol: str, interval: str, start: datetime, end: datetime) -> pd.DataFrame:
+    import yfinance as yf
+    last_error = None
+    for attempt in range(3):
+        try:
+            return yf.Ticker(symbol).history(
+                start=start, end=end, interval=interval, auto_adjust=False,
+                actions=False, timeout=_YF_TIMEOUT_SECONDS, raise_errors=True,
+            )
+        except Exception as error:
+            last_error = error
+            if attempt < 2:
+                time.sleep(0.5 * (attempt + 1) + random.uniform(0.0, 0.25))
+    raise DataFetchError(f"Yahoo Finance could not fetch {symbol} ({interval}) after 3 attempts ({type(last_error).__name__}).") from None
+
+def _download_window(symbol: str, interval: str, start: datetime, end: datetime) -> pd.DataFrame:
+    df = _fetch_yf(symbol, interval, start, end)
+    if df is None or df.empty:
+        return pd.DataFrame()
+    if isinstance(df.columns, pd.MultiIndex):
+        # yfinance can return either (field, ticker) or (ticker, field),
+        # depending on version and whether the ticker is cached.
+        fields = set(OHLCV_COLUMNS).intersection(set(df.columns.get_level_values(0)))
+        if fields:
+            df = df.droplevel(1, axis=1)
+        else:
+            fields = set(OHLCV_COLUMNS).intersection(set(df.columns.get_level_values(-1)))
+            if fields:
+                df = df.droplevel(0, axis=1)
+    df = df.rename(columns=lambda value: str(value).title())
+    df.index.name = 'Datetime'
+    keep = [c for c in OHLCV_COLUMNS if c in df.columns]
+    return df[keep]
+
+def _finalize_ohlcv(df: pd.DataFrame) -> pd.DataFrame:
+    if df is None or df.empty:
+        return df
+    return validate_ohlcv(df, normalize=True)
+
+def _native_fetch(symbol: str, interval: str, days: int) -> pd.DataFrame:
+    days = positive_integer(days, 'lookback_days', 10000)
+    if interval not in YF_NATIVE_INTERVALS:
+        raise DataFetchError(f"'{interval}' is not a native yfinance interval.")
+    limits = YF_NATIVE_INTERVALS[interval]
+    max_days, request_days = (limits['max_days'], limits['max_request_days'])
+    if max_days is not None:
+        days = min(days, max_days)
+    if request_days is None:
+        request_days = days
+    frames, empty_windows = [], 0
+    window_end = datetime.now(timezone.utc)
+    remaining = days
+    while remaining > 0:
+        window_days = min(request_days, remaining)
+        window_start = window_end - timedelta(days=window_days)
+        frame = _download_window(symbol, interval, window_start, window_end)
+        if not frame.empty:
+            frames.append(frame)
+        else:
+            empty_windows += 1
+        window_end = window_start
+        remaining -= window_days
+        if remaining > 0:
+            time.sleep(0.1 + random.uniform(0, 0.2))
+    df = _finalize_ohlcv(pd.concat(frames)) if frames else pd.DataFrame()
+    if df.empty or 'Close' not in df.columns:
+        limit_note = f" Yahoo keeps at most {max_days} days of '{interval}' history." if max_days is not None else ''
+        raise DataFetchError(f"No usable data returned for '{symbol}' at interval '{interval}' over the last {days} day(s). Check it's a valid yfinance ticker (e.g. EURUSD=X, BTC-USD) and that the market has traded recently." + limit_note)
+    df.attrs['effective_lookback_days'] = days
+    df.attrs['source_interval'] = interval
+    df.attrs['empty_request_windows'] = empty_windows
+    return df
+
+def resample_ohlc(df: pd.DataFrame, rule: str, source_step=None) -> pd.DataFrame:
+    if df.empty:
+        return df.copy()
+    if not isinstance(df.index, pd.DatetimeIndex) or df.index.isna().any() or df.index.duplicated().any() or not df.index.is_monotonic_increasing:
+        raise DataFetchError('Resampling needs ordered, unique datetime candles.')
+    agg = {'Open': 'first', 'High': 'max', 'Low': 'min', 'Close': 'last', 'Volume': 'sum'}
+    agg = {k: v for k, v in agg.items() if k in df.columns}
+    price_cols = [c for c in ('Open', 'High', 'Low', 'Close') if c in agg]
+    grouped = df.resample(rule, origin='epoch', label='left', closed='left')
+    result = grouped.agg(agg).dropna(subset=price_cols)
+    if source_step is not None:
+        source_step = pd.Timedelta(source_step)
+        target_step = pd.Timedelta(rule)
+        if source_step <= pd.Timedelta(0) or target_step < source_step or target_step % source_step != pd.Timedelta(0):
+            raise ValueError('Resampling requires a positive integral multiple of the source interval.')
+        expected = int(target_step / source_step)
+        counts = grouped['Close'].count()
+        times = pd.Series(df.index, index=df.index)
+        first = times.resample(rule, origin='epoch').first()
+        last = times.resample(rule, origin='epoch').last()
+        # Count + endpoints alone do not prove that interior source candles are
+        # on-grid (two shifted candles could previously form an invented bar).
+        on_grid = pd.Series((df.index.asi8 % source_step.value) == 0, index=df.index)
+        grid_ok = on_grid.astype('int8').resample(rule, origin='epoch').min().eq(1)
+        complete = (counts == expected) & grid_ok & (first == counts.index) & (last == counts.index + target_step - source_step)
+        result = result.loc[complete.reindex(result.index, fill_value=False)]
+    result.attrs.update(df.attrs)
+    return result
+
+def get_historical(symbol: str, timeframe: str, lookback_days: int=None) -> pd.DataFrame:
+    if not symbol or not symbol.strip():
+        raise DataFetchError('No symbol given.')
+    symbol = symbol.strip()
+    if timeframe in CUSTOM_TIMEFRAMES:
+        spec = CUSTOM_TIMEFRAMES[timeframe]
+        source_interval, rule = (spec['source'], spec['rule'])
+    elif timeframe in YF_NATIVE_INTERVALS:
+        source_interval, rule = (timeframe, None)
+    else:
+        raise DataFetchError(f"Unknown timeframe '{timeframe}'. Valid options: {', '.join(CUSTOM_TIMEFRAMES.keys())}")
+    max_days = YF_NATIVE_INTERVALS[source_interval]['max_days']
+    requested = positive_integer(lookback_days, 'lookback_days', 10000) if lookback_days is not None else 0
+    days = requested if requested > 0 else max_days if max_days else 730
+    df = _native_fetch(symbol, source_interval, days)
+    if rule:
+        source_minutes = 60 if source_interval == '60m' else int(source_interval.rstrip('m'))
+        df = resample_ohlc(df, rule, source_step=pd.Timedelta(minutes=source_minutes))
+        if df.empty:
+            raise DataFetchError(f"Not enough '{source_interval}' candles to build '{timeframe}' bars. Try a longer lookback.")
+    return df
+
+def get_historical_with_features(symbol: str, timeframe: str, lookback_days: int, use_features: bool):
+    if not use_features:
+        return (get_historical(symbol, timeframe, lookback_days=int(lookback_days)), None)
+    padding_days = feature_warmup_days(timeframe, minimum_days=0)
+    df_padded = get_historical(symbol, timeframe, lookback_days=int(lookback_days) + padding_days)
+    return compute_feature_frame_for_period(df_padded, keep_days=int(lookback_days))

data/providers.py

@@ -0,0 +1,317 @@
+"""Real public market-data adapters. Provider identities never change silently."""
+from __future__ import annotations
+
+import os
+import re
+import threading
+import time
+
+import pandas as pd
+import requests
+
+from config import DEFAULT_CRYPTO_SYMBOLS, DEFAULT_FOREX_SYMBOLS, TIMEFRAMES
+from data.historical import get_historical, resample_ohlc
+from data.validation import DataFetchError, validate_ohlcv, positive_integer
+from utils.helpers import MINUTES_PER_CANDLE, drop_forming_candle
+
+BINANCE_URL = 'https://api.binance.com/api/v3'
+BINANCE_FALLBACK_URLS = (
+    'https://data-api.binance.vision/api/v3',
+    'https://api1.binance.com/api/v3',
+)
+TWELVE_URL = 'https://api.twelvedata.com'
+MAX_SOURCE_CANDLES = 100_000
+_CATALOG_CACHE = {}
+_CATALOG_LOCK = threading.Lock()
+
+
+def _json_get(url, params=None):
+    last_error = None
+    for attempt in range(3):
+        try:
+            response = requests.get(url, params=params, timeout=(10, 30))
+            if response.status_code == 429 or response.status_code >= 500:
+                last_error = DataFetchError(f'HTTP {response.status_code}')
+                if attempt < 2:
+                    time.sleep(0.5 * (attempt + 1))
+                    continue
+            if response.status_code != 200:
+                raise DataFetchError(f'Market-data provider returned HTTP {response.status_code}. Check availability, regional access, and your plan/rate limits.')
+            result = response.json()
+            break
+        except (requests.RequestException, ValueError) as error:
+            last_error = error
+            if attempt < 2:
+                time.sleep(0.5 * (attempt + 1))
+                continue
+            raise DataFetchError(f'Market-data request failed ({type(error).__name__}). Check your network and provider availability.') from None
+    else:
+        raise DataFetchError(f'Market-data request failed ({last_error}). Check your network and provider availability.') from None
+    if isinstance(result, dict) and (result.get('status') == 'error' or result.get('code') not in (None, 200)):
+        message = str(result.get('message') or result.get('msg') or 'Unknown provider error')
+        key = os.environ.get('TWELVE_DATA_API_KEY', '')
+        if key:
+            message = message.replace(key, '[redacted]')
+        label = 'Twelve Data' if 'twelvedata.com' in str(url) else 'Market-data provider'
+        raise DataFetchError(f'{label}: {message}')
+    return result
+
+
+def _binance_get(path, params=None):
+    """Use Binance's public data mirrors when a hosting region blocks api.binance.com."""
+    last_error = None
+    for base in (BINANCE_URL, *BINANCE_FALLBACK_URLS):
+        try:
+            return _json_get(base + path, params)
+        except DataFetchError as error:
+            last_error = error
+    raise last_error or DataFetchError('Binance request failed without a provider response.')
+
+
+def normalize_symbol(symbol, provider, market):
+    symbol = str(symbol or '').strip().upper()
+    if not symbol or len(symbol) > 80 or not re.fullmatch(r'[A-Z0-9=./:_-]+', symbol):
+        raise ValueError('Enter a valid provider symbol (for example EURUSD=X, EUR/USD, BTC-USD, or BTCUSDT).')
+    if market not in ('Forex', 'Crypto'):
+        raise ValueError('Market must be Forex or Crypto.')
+    if provider == 'Binance Spot':
+        if market != 'Crypto':
+            raise ValueError('Binance Spot provides crypto pairs; select Yahoo Finance or Twelve Data for Forex.')
+        return symbol.replace('/', '').replace('-', '')
+    if provider == 'Twelve Data':
+        if market != 'Forex':
+            raise ValueError('This Twelve Data adapter is for Forex. Select Binance Spot or Yahoo Finance for Crypto.')
+        symbol = symbol.removesuffix('=X').replace('/', '')
+        if len(symbol) != 6 or not symbol.isalpha():
+            raise ValueError('Use a Forex pair such as EUR/USD for Twelve Data.')
+        return symbol[:3] + '/' + symbol[3:]
+    if provider != 'Yahoo Finance':
+        raise ValueError(f'Unknown provider: {provider}')
+    if market == 'Forex':
+        pair = symbol.removesuffix('=X').replace('/', '')
+        if len(pair) != 6 or not pair.isalpha():
+            raise ValueError('Use a Forex pair such as EURUSD=X or EUR/USD.')
+        return pair + '=X'
+    return symbol.replace('/', '-')
+
+
+def catalog(provider, market, query=''):
+    """Return supported catalog entries; custom Yahoo entries are validated on fetch."""
+    query = str(query or '').strip().upper()
+    if provider == 'Yahoo Finance':
+        defaults = DEFAULT_FOREX_SYMBOLS if market == 'Forex' else DEFAULT_CRYPTO_SYMBOLS
+        if not query:
+            return list(defaults), 'Common Yahoo symbols. Search or type any provider-supported pair; intraday coverage varies by symbol.'
+        import yfinance as yf
+        try:
+            quotes = yf.Search(query, max_results=50, news_count=0, timeout=20).quotes
+        except Exception as error:
+            raise DataFetchError(f'Yahoo symbol search failed: {type(error).__name__}. You can still type a known symbol.') from None
+        if not isinstance(quotes, list):
+            raise DataFetchError('Yahoo symbol search returned an invalid catalog response.')
+        wanted = 'CURRENCY' if market == 'Forex' else 'CRYPTOCURRENCY'
+        symbols = [item['symbol'] for item in quotes if isinstance(item, dict) and item.get('quoteType') == wanted and item.get('symbol')]
+        symbols += [s for s in defaults if query in s]
+        return sorted(set(symbols)), 'Yahoo search results; a listing does not guarantee intraday history.'
+    if provider == 'Binance Spot' and market != 'Crypto':
+        raise ValueError('Binance Spot is available for Crypto only.')
+    if provider == 'Twelve Data' and market != 'Forex':
+        raise ValueError('Select Forex for the Twelve Data adapter.')
+    key = (provider, market)
+    with _CATALOG_LOCK:
+        cached = _CATALOG_CACHE.get(key)
+    if not cached or time.monotonic() - cached[0] > 900:
+        if provider == 'Binance Spot':
+            payload = _binance_get('/exchangeInfo')
+            if not isinstance(payload, dict) or not isinstance(payload.get('symbols'), list):
+                raise DataFetchError('Binance returned an invalid exchange catalog.')
+            symbols = sorted(item['symbol'] for item in payload['symbols'] if isinstance(item, dict) and item.get('symbol') and item.get('status') == 'TRADING' and item.get('isSpotTradingAllowed', True))
+        elif provider == 'Twelve Data':
+            payload = _json_get(TWELVE_URL + '/forex_pairs')
+            if not isinstance(payload, dict) or not isinstance(payload.get('data'), list):
+                raise DataFetchError('Twelve Data returned an invalid Forex catalog.')
+            symbols = sorted({item['symbol'] for item in payload['data'] if isinstance(item, dict) and isinstance(item.get('symbol'), str) and item['symbol'].strip()})
+        else:
+            raise ValueError('Unknown market-data provider.')
+        if not symbols:
+            raise DataFetchError('Provider returned an empty symbol catalog.')
+        cached = (time.monotonic(), symbols)
+        with _CATALOG_LOCK:
+            _CATALOG_CACHE[key] = cached
+    matches = [symbol for symbol in cached[1] if query in symbol]
+    return matches, f'{len(cached[1])} provider-listed pairs; {len(matches)} match your search. Trading/history availability depends on the provider.'
+
+
+def _validate_frame(frame):
+    return validate_ohlcv(frame, normalize=True)
+
+
+def _binance_history(symbol, timeframe, days):
+    days = positive_integer(days, 'lookback_days', 10000)
+    source = {'2m': '1m', '10m': '5m'}.get(timeframe, timeframe)
+    if source not in MINUTES_PER_CANDLE:
+        raise DataFetchError(f"Binance does not support source timeframe '{source}'.")
+    source_minutes = MINUTES_PER_CANDLE[source]
+    if days * 1440 / source_minutes > MAX_SOURCE_CANDLES:
+        raise DataFetchError(f'This request exceeds {MAX_SOURCE_CANDLES:,} source candles. Reduce lookback days or use a larger timeframe.')
+    clock = _binance_get('/time')
+    try:
+        end_ms = int(clock['serverTime'])
+    except (KeyError, TypeError, ValueError):
+        raise DataFetchError('Binance returned an invalid server clock response.') from None
+    start_ms = end_ms - int(days * 86_400_000)
+    cursor = start_ms
+    rows = []
+    while cursor < end_ms:
+        batch = _binance_get('/klines', {
+            'symbol': symbol, 'interval': source, 'startTime': cursor,
+            'endTime': end_ms - 1, 'limit': 1000,
+        })
+        if not isinstance(batch, list):
+            raise DataFetchError('Binance returned an unexpected candle response.')
+        if not batch:
+            break
+        if any(not isinstance(row, (list, tuple)) or len(row) < 6 for row in batch):
+            raise DataFetchError('Binance returned a malformed candle row.')
+        rows.extend(batch)
+        try:
+            next_cursor = int(batch[-1][0]) + source_minutes * 60_000
+        except (TypeError, ValueError):
+            raise DataFetchError('Binance returned an invalid candle timestamp.') from None
+        if next_cursor <= cursor:
+            raise DataFetchError('Binance pagination did not advance.')
+        cursor = next_cursor
+        if len(rows) > MAX_SOURCE_CANDLES:
+            raise DataFetchError('Provider request exceeded the source candle limit.')
+        if len(batch) < 1000:
+            break
+        time.sleep(0.05)
+    if not rows:
+        raise DataFetchError(f'Binance has no candles for {symbol}. Use an active exchange symbol such as BTCUSDT (USDT is not USD).')
+    try:
+        frame = pd.DataFrame([
+            [r[1], r[2], r[3], r[4], r[5]] for r in rows
+        ], columns=['Open', 'High', 'Low', 'Close', 'Volume'], index=pd.to_datetime([int(r[0]) for r in rows], unit='ms', utc=True)).astype(float)
+    except (TypeError, ValueError, OverflowError):
+        raise DataFetchError('Binance returned malformed OHLCV values.') from None
+    frame.attrs['provider_now'] = pd.Timestamp(end_ms, unit='ms', tz='UTC').isoformat()
+    frame.attrs['source_interval'] = source
+    try:
+        exchange_info = _binance_get('/exchangeInfo', {'symbol': symbol})
+        symbol_info = (exchange_info.get('symbols') or [{}])[0]
+        price_filter = next((item for item in symbol_info.get('filters', []) if item.get('filterType') == 'PRICE_FILTER'), None)
+        if price_filter and float(price_filter.get('tickSize', 0)) > 0:
+            frame.attrs['price_tick_size'] = float(price_filter['tickSize'])
+            frame.attrs['price_tick_source'] = 'Binance PRICE_FILTER tickSize'
+    except Exception:
+        # Precision inference from observed real closes remains available when
+        # an exchange-info request is temporarily rate-limited.
+        pass
+    frame = _validate_frame(frame)
+    if source != timeframe:
+        frame = resample_ohlc(frame, f'{MINUTES_PER_CANDLE[timeframe]}min', pd.Timedelta(minutes=source_minutes))
+    return frame
+
+
+def _twelve_history(symbol, timeframe, days):
+    days = positive_integer(days, 'lookback_days', 10000)
+    api_key = os.environ.get('TWELVE_DATA_API_KEY', '').strip()
+    if not api_key:
+        raise DataFetchError('Twelve Data Forex history requires TWELVE_DATA_API_KEY in your environment. Your plan must cover the selected pair, interval, and historical depth.')
+    source = {'2m': '1m', '10m': '5m', '2h': '1h'}.get(timeframe, timeframe)
+    source_minutes = MINUTES_PER_CANDLE[source]
+    if days * 1440 / source_minutes > MAX_SOURCE_CANDLES:
+        raise DataFetchError('Reduce lookback: request exceeds 100,000 source candles.')
+    interval = {'1m': '1min', '5m': '5min', '15m': '15min', '30m': '30min', '1h': '1h'}[source]
+    end = pd.Timestamp.now(tz='UTC')
+    request_end = end
+    cutoff = end - pd.Timedelta(days=days)
+    records = []
+    while end > cutoff:
+        payload = _json_get(TWELVE_URL + '/time_series', {
+            'symbol': symbol, 'interval': interval, 'apikey': api_key,
+            'timezone': 'UTC', 'order': 'DESC', 'outputsize': 5000,
+            'start_date': cutoff.strftime('%Y-%m-%d %H:%M:%S'),
+            'end_date': end.strftime('%Y-%m-%d %H:%M:%S'),
+        })
+        if not isinstance(payload, dict):
+            raise DataFetchError('Twelve Data returned an invalid time-series response.')
+        batch = payload.get('values', [])
+        if not isinstance(batch, list):
+            raise DataFetchError('Twelve Data returned malformed time-series values.')
+        if not batch:
+            break
+        if any(not isinstance(row, dict) or 'datetime' not in row for row in batch):
+            raise DataFetchError('Twelve Data returned a malformed candle row.')
+        records.extend(batch)
+        parsed_times = pd.to_datetime([row['datetime'] for row in batch], utc=True, errors='coerce')
+        if parsed_times.isna().any():
+            raise DataFetchError('Twelve Data returned an invalid candle timestamp.')
+        oldest = parsed_times.min()
+        next_end = oldest - pd.Timedelta(seconds=1)
+        if next_end >= end:
+            raise DataFetchError('Twelve Data pagination did not advance.')
+        end = next_end
+        if len(records) > MAX_SOURCE_CANDLES:
+            raise DataFetchError('Too many source candles returned.')
+        if len(batch) < 5000:
+            break
+        time.sleep(1)
+    if not records:
+        raise DataFetchError('Twelve Data returned no Forex candles for this request.')
+    raw = pd.DataFrame(records)
+    required = {'datetime', 'open', 'high', 'low', 'close'}
+    if not required.issubset(raw.columns):
+        raise DataFetchError(f'Twelve Data response is missing fields: {sorted(required - set(raw.columns))}.')
+    try:
+        frame = pd.DataFrame({
+            'Open': pd.to_numeric(raw['open'], errors='raise'), 'High': pd.to_numeric(raw['high'], errors='raise'),
+            'Low': pd.to_numeric(raw['low'], errors='raise'), 'Close': pd.to_numeric(raw['close'], errors='raise'),
+            'Volume': pd.to_numeric(raw['volume'], errors='raise') if 'volume' in raw else 0.0,
+        })
+        frame.index = pd.to_datetime(raw['datetime'], utc=True, errors='raise')
+    except (KeyError, TypeError, ValueError):
+        raise DataFetchError('Twelve Data returned malformed OHLCV values or timestamps.') from None
+    frame = _validate_frame(frame)
+    frame = frame.loc[(frame.index >= cutoff) & (frame.index <= request_end)]
+    if frame.empty:
+        raise DataFetchError('Twelve Data returned no candles inside the requested UTC range.')
+    frame.attrs['volume_note'] = 'Forex has no consolidated traded volume. Zero means unavailable; MAYTHOS uses its price-based proxy.'
+    frame.attrs['source_interval'] = source
+    if source != timeframe:
+        frame = resample_ohlc(frame, f'{MINUTES_PER_CANDLE[timeframe]}min', pd.Timedelta(minutes=source_minutes))
+    return frame
+
+
+def fetch_market_history(provider, market, symbol, timeframe, lookback_days):
+    if timeframe not in TIMEFRAMES:
+        raise ValueError('Select one of: ' + ', '.join(TIMEFRAMES))
+    # The UI caps user history at 730 days; feature warm-up may add a small
+    # provider request padding before the requested period is trimmed.
+    lookback_days = positive_integer(lookback_days, 'lookback_days', 10000)
+    symbol = normalize_symbol(symbol, provider, market)
+    if provider == 'Yahoo Finance':
+        frame = get_historical(symbol, timeframe, int(lookback_days))
+    elif provider == 'Binance Spot':
+        frame = _binance_history(symbol, timeframe, int(lookback_days))
+    else:
+        frame = _twelve_history(symbol, timeframe, int(lookback_days))
+    frame = _validate_frame(frame)
+    attrs = dict(frame.attrs)
+    reference_now = attrs.get('provider_now') or pd.Timestamp.now(tz='UTC')
+    try:
+        reference_now = pd.Timestamp(reference_now)
+        if reference_now.tzinfo is None:
+            reference_now = reference_now.tz_localize('UTC')
+        else:
+            reference_now = reference_now.tz_convert('UTC')
+    except (TypeError, ValueError):
+        raise DataFetchError('Provider returned an invalid reference clock.') from None
+    if frame.index[-1] > reference_now + pd.Timedelta(minutes=MINUTES_PER_CANDLE[timeframe]):
+        raise DataFetchError('Provider returned a candle timestamp in the future; data rejected.')
+    frame, _ = drop_forming_candle(frame, timeframe=timeframe, now=reference_now)
+    if frame.empty:
+        raise DataFetchError('No complete, closed candles are available for this request.')
+    frame.attrs.update(attrs)
+    frame.attrs.update(provider=provider, market=market, symbol=symbol, timeframe=timeframe)
+    return frame

data/validation.py

@@ -0,0 +1,60 @@
+"""Shared OHLCV contracts; never impute or silently discard invalid observations."""
+import numbers
+
+import numpy as np
+import pandas as pd
+
+
+OHLCV_COLUMNS = ['Open', 'High', 'Low', 'Close', 'Volume']
+
+
+class DataFetchError(ValueError):
+    pass
+
+
+def validate_ohlcv(frame, *, normalize=False):
+    if not isinstance(frame, pd.DataFrame) or frame.empty:
+        raise DataFetchError('No real OHLCV candles were returned.')
+    if frame.columns.duplicated().any():
+        raise DataFetchError('OHLCV column names must be unique.')
+    missing = [column for column in OHLCV_COLUMNS if column not in frame]
+    if missing:
+        raise DataFetchError(f'Provider data is missing columns: {missing}.')
+    if not isinstance(frame.index, pd.DatetimeIndex):
+        raise DataFetchError('Candle timestamps need an explicit DatetimeIndex, not row IDs or ambiguous strings.')
+    attrs = dict(frame.attrs)
+    out = frame[OHLCV_COLUMNS].copy()
+    out.index = pd.to_datetime(out.index, utc=True, errors='coerce')
+    if out.index.isna().any():
+        raise DataFetchError('Invalid candle timestamps; data has not been repaired or filled.')
+    for column in OHLCV_COLUMNS:
+        out[column] = pd.to_numeric(out[column], errors='coerce').astype(float)
+    if not np.isfinite(out.to_numpy()).all():
+        raise DataFetchError('Missing/non-finite OHLCV values; data has not been filled or fabricated.')
+    bad = (out[['Open', 'High', 'Low', 'Close']] <= 0).any(axis=1)
+    bad |= out['Volume'] < 0
+    bad |= out['High'] < out[['Open', 'Close', 'Low']].max(axis=1)
+    bad |= out['Low'] > out[['Open', 'Close', 'High']].min(axis=1)
+    if bad.any():
+        raise DataFetchError(f'Provider returned {int(bad.sum())} invalid OHLCV candle(s).')
+    if normalize:
+        duplicates = int(out.index.duplicated().sum())
+        if duplicates:
+            repeated = out.loc[out.index.duplicated(keep=False)]
+            if (repeated.groupby(level=0).nunique() > 1).any().any():
+                raise DataFetchError('Conflicting duplicate candles were returned; no revision was chosen silently.')
+            out = out.loc[~out.index.duplicated(keep='last')]
+            attrs['identical_duplicates_removed'] = attrs.get('identical_duplicates_removed', 0) + duplicates
+        out = out.sort_index()
+    elif not out.index.is_monotonic_increasing or out.index.duplicated().any():
+        raise DataFetchError('Candles must be chronologically ordered and unique.')
+    out.attrs.update(attrs)
+    return out
+
+
+def positive_integer(value, name, maximum=None):
+    if isinstance(value, (bool, np.bool_)) or not isinstance(value, numbers.Integral) or value < 1:
+        raise ValueError(f'{name} must be a positive integer.')
+    if maximum is not None and value > maximum:
+        raise ValueError(f'{name} must be at most {maximum}.')
+    return int(value)

features/__init__.py

@@ -0,0 +1 @@
+pass

features/feature_pipeline.py

@@ -0,0 +1,61 @@
+import numpy as np
+import pandas as pd
+from indicators.indicators import add_all_indicators
+from data.validation import validate_ohlcv, positive_integer
+from utils.helpers import MINUTES_PER_CANDLE, infer_step
+REQUIRED_OHLCV_COLUMNS = ['Open', 'High', 'Low', 'Close', 'Volume']
+FEATURE_COLUMNS = ['Open', 'High', 'Low', 'Volume', 'RSI_14', 'MACD', 'MACD_Signal', 'MACD_Hist', 'BB_Upper', 'BB_Lower', 'SMA_20', 'EMA_20', 'Stoch_%K', 'Stoch_%D']
+
+def compute_feature_frame(df: pd.DataFrame, timeframe=None) -> pd.DataFrame:
+    missing = [c for c in REQUIRED_OHLCV_COLUMNS if c not in df.columns]
+    if missing:
+        raise ValueError(f'compute_feature_frame needs an OHLCV DataFrame; missing column(s): {missing}.')
+    df = validate_ohlcv(df)
+    expected = pd.Timedelta(minutes=MINUTES_PER_CANDLE[timeframe]) if timeframe else infer_step(df.index)
+    groups = (df.index.to_series().diff() != expected).cumsum()
+    # Gaps never become adjacent equally spaced observations. Indicators restart
+    # on each real continuous segment; their genuine warm-up NaNs remain visible.
+    with_indicators = pd.concat([add_all_indicators(segment) for _, segment in df.groupby(groups)])
+    return with_indicators[FEATURE_COLUMNS].reset_index(drop=True).astype(float)
+
+def slice_features(features: pd.DataFrame, start: int, end: int) -> pd.DataFrame:
+    return features.iloc[start:end].reset_index(drop=True)
+
+def compute_feature_frame_for_period(df_padded: pd.DataFrame, keep_days: float, timeframe=None):
+    try:
+        keep_days = float(keep_days)
+    except (TypeError, ValueError):
+        keep_days = float('nan')
+    if isinstance(keep_days, bool) or not np.isfinite(keep_days) or keep_days <= 0:
+        raise ValueError('keep_days must be finite and positive.')
+    features_padded = compute_feature_frame(df_padded, timeframe=timeframe)
+    cutoff = df_padded.index[-1] - pd.Timedelta(days=keep_days)
+    positions = np.flatnonzero(df_padded.index >= cutoff)
+    complete = ~features_padded.isna().any(axis=1).to_numpy()
+    positions = positions[complete[positions]] if len(positions) else positions
+    if len(positions) == 0:
+        raise ValueError(f'No candle in the requested period has complete indicator values: only {len(df_padded)} candles were fetched, and the slowest indicator needs 20 candles of warm-up before the period starts. Increase the lookback (markets that close, e.g. forex over a weekend, need more calendar days to reach the same candle count).')
+    return (df_padded.iloc[positions], features_padded.iloc[positions].reset_index(drop=True))
+
+def validate_features(features: pd.DataFrame, expected_len: int, model_name: str, check_nan: bool=True) -> None:
+    if not isinstance(features, pd.DataFrame) or features.shape[1] == 0 or features.columns.duplicated().any():
+        raise ValueError(f'{model_name}: features require a DataFrame with unique numeric columns.')
+    if len(features) != expected_len:
+        raise ValueError(f'{model_name}: feature window has {len(features)} rows, expected {expected_len} to match the price history it must line up with.')
+    numeric = features.apply(pd.to_numeric, errors='coerce')
+    non_numeric = features.notna() & numeric.isna()
+    if non_numeric.any().any():
+        bad_cols = non_numeric.columns[non_numeric.any()].tolist()
+        raise ValueError(f'{model_name}: features contain non-numeric values in {bad_cols}.')
+    if np.isinf(numeric.to_numpy(dtype=float)).any():
+        raise ValueError(f'{model_name}: features contain infinite values.')
+    if check_nan and numeric.isna().any().any():
+        bad_cols = features.columns[features.isna().any()].tolist()
+        raise ValueError(f"{model_name}: feature window contains NaN in {bad_cols}. This usually means there isn't enough real history before this window for the slower indicators (SMA/Bollinger need 20 prior candles) to warm up on -- increase the lookback/history and try again.")
+
+def carry_forward_future(feature_window: pd.DataFrame, horizon: int) -> pd.DataFrame:
+    horizon = positive_integer(horizon, 'carry_forward_future horizon')
+    if len(feature_window) == 0:
+        raise ValueError('carry_forward_future: feature_window is empty.')
+    last_row = feature_window.iloc[[-1]]
+    return pd.concat([last_row] * horizon, ignore_index=True)

indicators/__init__.py

@@ -0,0 +1 @@
+pass

indicators/indicators.py

@@ -0,0 +1,100 @@
+import numpy as np
+import pandas as pd
+
+def sma(series: pd.Series, window: int) -> pd.Series:
+    return series.rolling(window=window).mean()
+
+def ema(series: pd.Series, window: int) -> pd.Series:
+    return series.ewm(span=window, adjust=False).mean()
+
+def _wilder_rma(values: pd.Series, window: int) -> pd.Series:
+    if isinstance(window, bool) or not isinstance(window, (int, np.integer)) or window < 1:
+        raise ValueError('Wilder period must be a positive integer.')
+    arr = values.to_numpy(dtype=float)
+    n = len(arr)
+    out = np.full(n, np.nan)
+    seed, prev = [], None
+    for i, value in enumerate(arr):
+        if not np.isfinite(value):
+            seed, prev = [], None
+            continue
+        if prev is None:
+            seed.append(value)
+            if len(seed) == window:
+                prev = float(np.mean(seed))
+                out[i] = prev
+        else:
+            prev = (prev * (window - 1) + value) / window
+            out[i] = prev
+    return pd.Series(out, index=values.index)
+
+def rsi(series: pd.Series, window: int=14) -> pd.Series:
+    delta = series.diff()
+    gain = delta.clip(lower=0)
+    loss = -delta.clip(upper=0)
+    avg_gain = _wilder_rma(gain, window)
+    avg_loss = _wilder_rma(loss, window)
+    rs = avg_gain / avg_loss.replace(0, np.nan)
+    out = 100 - 100 / (1 + rs)
+    out = out.mask((avg_loss == 0) & (avg_gain > 0), 100.0)
+    return out.mask((avg_loss == 0) & (avg_gain == 0), 50.0)
+
+def macd(series: pd.Series, fast: int=12, slow: int=26, signal: int=9):
+    fast_ema = ema(series, fast)
+    slow_ema = ema(series, slow)
+    macd_line = fast_ema - slow_ema
+    signal_line = ema(macd_line, signal)
+    histogram = macd_line - signal_line
+    return (macd_line, signal_line, histogram)
+
+def bollinger_bands(series: pd.Series, window: int=20, num_std: float=2.0):
+    mid = sma(series, window)
+    std = series.rolling(window=window).std(ddof=0)
+    return (mid + num_std * std, mid, mid - num_std * std)
+
+def stochastic_oscillator(high: pd.Series, low: pd.Series, close: pd.Series, k_window: int=14, d_window: int=3):
+    lowest_low = low.rolling(window=k_window).min()
+    highest_high = high.rolling(window=k_window).max()
+    span = highest_high - lowest_low
+    percent_k = 100 * (close - lowest_low) / span.replace(0, np.nan)
+    percent_k = percent_k.mask(span == 0, 50.0)
+    percent_d = percent_k.rolling(window=d_window).mean()
+    return (percent_k, percent_d)
+
+def add_all_indicators(df: pd.DataFrame) -> pd.DataFrame:
+    out = df.copy()
+    out['SMA_20'] = sma(out['Close'], 20)
+    out['EMA_20'] = ema(out['Close'], 20)
+    out['RSI_14'] = rsi(out['Close'], 14)
+    macd_line, signal_line, hist = macd(out['Close'])
+    out['MACD'] = macd_line
+    out['MACD_Signal'] = signal_line
+    out['MACD_Hist'] = hist
+    upper, mid, lower = bollinger_bands(out['Close'])
+    out['BB_Upper'] = upper
+    out['BB_Mid'] = mid
+    out['BB_Lower'] = lower
+    k, d = stochastic_oscillator(out['High'], out['Low'], out['Close'])
+    out['Stoch_%K'] = k
+    out['Stoch_%D'] = d
+    return out
+
+def _round(value, digits):
+    return round(float(value), digits) if pd.notna(value) else 'n/a'
+
+def latest_signals(df_with_indicators: pd.DataFrame) -> dict:
+    if df_with_indicators.empty:
+        raise ValueError('No candles available for latest indicators.')
+    last = df_with_indicators.iloc[-1]
+    signals = {}
+    rsi_value = last['RSI_14']
+    signals['RSI (14)'] = {'value': _round(rsi_value, 2), 'signal': 'Insufficient data' if pd.isna(rsi_value) else 'Overbought' if rsi_value > 70 else 'Oversold' if rsi_value < 30 else 'Neutral'}
+    macd_ready = pd.notna(last['MACD']) and pd.notna(last['MACD_Signal'])
+    signals['MACD'] = {'value': _round(last['MACD'], 5), 'signal': 'Insufficient data' if not macd_ready else 'Bullish (above signal)' if last['MACD'] > last['MACD_Signal'] else 'Bearish (below signal)' if last['MACD'] < last['MACD_Signal'] else 'Neutral'}
+    bb_ready = pd.notna(last['BB_Upper']) and pd.notna(last['BB_Lower'])
+    signals['Bollinger Bands'] = {'value': _round(last['Close'], 5), 'signal': 'Insufficient data' if not bb_ready else 'Above Upper Band' if last['Close'] > last['BB_Upper'] else 'Below Lower Band' if last['Close'] < last['BB_Lower'] else 'Inside Bands'}
+    sma_ready = pd.notna(last['SMA_20'])
+    signals['Moving Averages'] = {'value': f"SMA20={_round(last['SMA_20'], 5)}, EMA20={_round(last['EMA_20'], 5)}", 'signal': 'Insufficient data for SMA' if not sma_ready else 'Price Above MA (Uptrend)' if last['Close'] > last['SMA_20'] else 'Price Below MA (Downtrend)' if last['Close'] < last['SMA_20'] else 'Neutral'}
+    stoch_value = last['Stoch_%K']
+    signals['Stochastic (14,3)'] = {'value': _round(stoch_value, 2), 'signal': 'Insufficient data' if pd.isna(stoch_value) else 'Overbought' if stoch_value > 80 else 'Oversold' if stoch_value < 20 else 'Neutral'}
+    return signals

install_foundation_windows.bat

@@ -0,0 +1,21 @@
+@echo off
+setlocal
+cd /d "%~dp0"
+if not exist ".venv\Scripts\python.exe" (
+  echo Run setup_windows.bat first.
+  pause
+  exit /b 1
+)
+".venv\Scripts\python.exe" -m pip install -r requirements-foundation.txt
+if errorlevel 1 (
+  pause
+  exit /b 1
+)
+".venv\Scripts\python.exe" -m pip check
+if errorlevel 1 (
+  echo Foundation dependency consistency check failed.
+  pause
+  exit /b 1
+)
+echo Pretrained model weights download on first inference. Internet and sufficient disk/RAM are required.
+pause

live_runner.py

@@ -0,0 +1,104 @@
+"""Poll real closed candles with the same selectable dashboard pipeline."""
+import argparse
+import csv
+from dataclasses import asdict
+import json
+from pathlib import Path
+import time
+
+import pandas as pd
+
+from config import MODEL_NAMES, OUTPUT_DIR, PROVIDERS, TIMEFRAMES
+from pipeline import RunConfig, live_result, prepare_history
+from utils.helpers import price_direction
+
+
+def main():
+    parser = argparse.ArgumentParser(description='MYTHOS selectable closed-candle forecast logger.')
+    parser.add_argument('--provider', choices=PROVIDERS, default='Yahoo Finance')
+    parser.add_argument('--market', choices=['Forex', 'Crypto'], default='Crypto')
+    parser.add_argument('--symbol', default='BTC-USD')
+    parser.add_argument('--timeframe', choices=TIMEFRAMES, default='5m')
+    parser.add_argument('--models', nargs='+', choices=MODEL_NAMES, default=['ARIMA-GARCH'])
+    parser.add_argument('--lookback-days', type=int, default=30)
+    parser.add_argument('--window', type=int, default=200)
+    parser.add_argument('--horizon', type=int, default=1)
+    parser.add_argument('--features', action='store_true')
+    parser.add_argument('--poll-seconds', type=int, default=60)
+    args = parser.parse_args()
+    config = RunConfig(provider=args.provider, market=args.market, symbol=args.symbol,
+                       timeframe=args.timeframe, models=tuple(args.models),
+                       mode='Parallel models' if len(args.models) > 1 else 'Single model',
+                       lookback_days=args.lookback_days, window=args.window,
+                       horizon=args.horizon, use_features=args.features)
+    run_id = pd.Timestamp.now(tz='UTC').strftime('%Y%m%dT%H%M%SZ')
+    Path(OUTPUT_DIR).mkdir(parents=True, exist_ok=True)
+    path = Path(OUTPUT_DIR) / f'live_logger_{run_id}.csv'
+    path.with_suffix('.json').write_text(json.dumps(asdict(config), indent=2), encoding='utf-8')
+    fields = ['model', 'predicted_at_utc', 'origin_open_utc', 'target_open_utc', 'current_close',
+              'predicted_close', 'predicted_direction', 'entry', 'stop_loss', 'take_profit',
+              'risk_reward', 'actual_close', 'actual_direction', 'correct', 'status']
+
+    def log(row):
+        new_file = not path.exists()
+        with path.open('a', newline='', encoding='utf-8') as file:
+            writer = csv.DictWriter(file, fieldnames=fields)
+            if new_file:
+                writer.writeheader()
+            writer.writerow(row)
+
+    pending = []
+    last_origin = None
+    print(f'MYTHOS {config.provider} · {config.symbol} · {config.timeframe} · {config.models}. Logging: {path}')
+    try:
+        while True:
+            try:
+                frame, features = prepare_history(config)
+                still_pending = []
+                for row in pending:
+                    target = pd.Timestamp(row['target_open_utc'])
+                    if target in frame.index:
+                        actual = float(frame.loc[target, 'Close'])
+                        direction = price_direction(actual, row['current_close'])
+                        interval = frame.loc[pd.Timestamp(row['origin_open_utc']):target].index.to_series().diff().dropna()
+                        continuous = len(interval) == config.horizon and (interval == config.candle_step).all()
+                        verified = {**row, 'actual_close': actual, 'actual_direction': direction,
+                                    'correct': direction == row['predicted_direction'] if continuous else '',
+                                    'status': 'VERIFIED' if continuous else 'GAP — not scored'}
+                        log(verified)
+                        print(verified)
+                    elif frame.index[-1] > target:
+                        log({**row, 'status': 'MISSING TARGET — not scored'})
+                    else:
+                        still_pending.append(row)
+                pending = still_pending
+                if last_origin != frame.index[-1]:
+                    result = live_result(config, frame, features)
+                    last_origin = frame.index[-1]
+                    for forecast in result['forecasts']:
+                        if not forecast['forecast_path'] or result['metadata']['stale']:
+                            print(f"{forecast['Model']}: {forecast['Status']} {forecast['Error']}")
+                            continue
+                        pending.append({
+                            'model': forecast['Model'], 'predicted_at_utc': result['metadata']['fetched_at_utc'],
+                            'origin_open_utc': last_origin.isoformat(),
+                            'target_open_utc': (last_origin + config.candle_step * config.horizon).isoformat(),
+                            'current_close': forecast['Current close'], 'predicted_close': forecast['Predicted close'],
+                            'predicted_direction': forecast['Direction'].lower(),
+                            'entry': result['signal'].get('trade_levels', {}).get('entry'),
+                            'stop_loss': result['signal'].get('trade_levels', {}).get('stop_loss'),
+                            'take_profit': result['signal'].get('trade_levels', {}).get('take_profit'),
+                            'risk_reward': result['signal'].get('trade_levels', {}).get('risk_reward'),
+                        })
+                        print(f"{forecast['Model']}: {forecast['Direction']} → {forecast['Predicted close']:.8f}")
+            except Exception as error:
+                print(f'Pipeline error: {error}')
+            time.sleep(max(1, args.poll_seconds))
+    except KeyboardInterrupt:
+        for row in pending:
+            log({**row, 'status': 'STOPPED BEFORE TARGET — not scored'})
+        print('Stopped.')
+
+
+if __name__ == '__main__':
+    main()

maythos_integration.py

@@ -0,0 +1,206 @@
+from __future__ import annotations
+
+from typing import Any, Dict, List, Tuple
+
+import numpy as np
+import pandas as pd
+
+from maythos import Candle, MAYTHOS
+from risk_management import CostModel, build_trade_levels, simulate_trade, summarize_trades
+from utils.helpers import MINUTES_PER_CANDLE, price_direction
+from data.validation import validate_ohlcv, positive_integer
+
+
+OHLCV_COLUMNS = ('Open', 'High', 'Low', 'Close', 'Volume')
+
+
+def _normalise_frame(df: pd.DataFrame) -> pd.DataFrame:
+    if not isinstance(df, pd.DataFrame):
+        raise TypeError('MAYTHOS integration needs a pandas DataFrame.')
+    frame = validate_ohlcv(df, normalize=True)
+    if frame.index[0] <= pd.Timestamp('1970-01-01', tz='UTC'):
+        raise ValueError('MAYTHOS integration needs positive epoch timestamps.')
+    return frame
+
+
+def _timestamp_seconds(value: Any) -> float:
+    timestamp = pd.Timestamp(value)
+    if timestamp.tzinfo is None:
+        timestamp = timestamp.tz_localize('UTC')
+    else:
+        timestamp = timestamp.tz_convert('UTC')
+    return float(timestamp.timestamp())
+
+
+def candle_from_row(timestamp: Any, row: pd.Series, source_id: str = 'historical') -> Candle:
+    return Candle(
+        timestamp=_timestamp_seconds(timestamp),
+        open=float(row['Open']),
+        high=float(row['High']),
+        low=float(row['Low']),
+        close=float(row['Close']),
+        volume=float(row['Volume']),
+        source_id=source_id,
+        is_closed=True,
+    )
+
+
+def _run_engine(frame: pd.DataFrame) -> List[Dict[str, Any]]:
+    engine = MAYTHOS()
+    outputs: List[Dict[str, Any]] = []
+    for timestamp, row in frame.iterrows():
+        candle = candle_from_row(timestamp, row)
+        output = engine.tick(candle)
+        errors = engine.validate(output)
+        if errors:
+            raise RuntimeError(f'MAYTHOS output contract failed: {errors}')
+        outputs.append(output)
+    return outputs
+
+
+def run_maythos_history(df: pd.DataFrame) -> pd.DataFrame:
+    """Run every MAYTHOS layer once per closed candle without lookahead."""
+    frame = _normalise_frame(df)
+    outputs = _run_engine(frame)
+    rows = []
+    for timestamp, row, output in zip(frame.index, frame.itertuples(), outputs):
+        rows.append({
+            'timestamp': timestamp.isoformat(),
+            'close': float(row.Close),
+            'direction': output['direction'],
+            'confidence': float(output['confidence']),
+            'internal_trust': float(output['internal_trust']),
+            'execution_suitability': output['execution_suitability'],
+            'blocked': bool(output['blocked_flag']),
+            'market_state': output['market_state'],
+            'regime_label': output['regime_label'],
+            'timeframe_alignment': float(output['timeframe_alignment']),
+            'reason_summary': output['reason_summary'],
+        })
+    return pd.DataFrame(rows)
+
+
+def backtest_maythos(
+    df: pd.DataFrame,
+    window: int,
+    horizon: int = 1,
+    step: int = 1,
+    max_evaluations: int = None,
+    timeframe: str = None,
+    market: str = 'Crypto',
+    symbol: str = '',
+    costs: CostModel = None,
+) -> Tuple[pd.DataFrame, Dict[str, Any]]:
+    """Evaluate MAYTHOS direction only when its signal is execution-ready."""
+    window = positive_integer(window, 'MAYTHOS backtest window')
+    horizon = positive_integer(horizon, 'MAYTHOS backtest horizon')
+    step = positive_integer(step, 'MAYTHOS backtest step')
+    frame = _normalise_frame(df)
+    costs = costs or CostModel()
+    outputs = _run_engine(frame)
+    last_i = len(frame) - horizon
+    rows = []
+    positions = list(range(window, last_i + 1, step))
+    if max_evaluations is not None:
+        max_evaluations = positive_integer(max_evaluations, 'MAYTHOS max_evaluations')
+        positions = positions[-max_evaluations:]
+    for i in positions:
+        current_index = i - 1
+        target_index = i + horizon - 1
+        output = outputs[current_index]
+        current_close = float(frame['Close'].iloc[current_index])
+        actual_close = float(frame['Close'].iloc[target_index])
+        continuous = True
+        if timeframe:
+            times = frame.index[current_index:target_index + 1]
+            continuous = (times.to_series().diff().dropna() == pd.Timedelta(minutes=MINUTES_PER_CANDLE[timeframe])).all()
+        # Confidence is heuristic, not a calibrated win probability. Weak and
+        # moderate states produced many marginal trades in the prior run, so
+        # only a fully confirmed strong state can enter the execution path.
+        actionable = (
+            continuous and output['direction'] in ('BUY', 'SELL') and
+            not bool(output['blocked_flag']) and
+            output['execution_suitability'] == 'strong' and
+            float(output.get('confidence', 0.0)) >= 0.69
+        )
+        levels = (
+            build_trade_levels(
+                frame.iloc[:i], output['direction'], market, symbol,
+                minimum_edge_atr=0.20, minimum_risk_reward=1.20,
+                minimum_edge_bps=(
+                    costs.spread_bps + costs.fee_bps + 2.0 * costs.slippage_bps
+                ) * 1.25,
+            ) if actionable else None
+        )
+        trade = {'outcome': 'NO_LEVELS', 'exit_price': None, 'net_r': None, 'bars_held': 0}
+        if levels is not None:
+            trade = {**trade, **simulate_trade(levels, frame.iloc[i:target_index + 1], costs)}
+        actionable = actionable and levels is not None
+        predicted_direction = None
+        if actionable:
+            predicted_direction = 'up' if output['direction'] == 'BUY' else 'down'
+        actual_direction = price_direction(actual_close, current_close)
+        rows.append({
+            'datetime': frame.index[target_index].isoformat(),
+            'current_close': current_close,
+            'predicted_close': np.nan,
+            'actual_close': actual_close,
+            'predicted_direction': predicted_direction,
+            'actual_direction': actual_direction if actionable else None,
+            'correct': bool(predicted_direction == actual_direction) if actionable else None,
+            'signal_available': actionable,
+            'trade_taken': levels is not None,
+            'entry': levels.entry if levels else np.nan,
+            'stop_loss': levels.stop_loss if levels else np.nan,
+            'take_profit': levels.take_profit if levels else np.nan,
+            'risk_reward': levels.risk_reward if levels else np.nan,
+            'atr': levels.atr if levels else np.nan,
+            'swing_support': levels.swing_support if levels else np.nan,
+            'swing_resistance': levels.swing_resistance if levels else np.nan,
+            'outcome': trade.get('outcome'),
+            'exit_price': trade.get('exit_price'),
+            'net_r': trade.get('net_r'),
+            'bars_held': trade.get('bars_held', 0),
+            'confidence': float(output['confidence']),
+            'execution_suitability': output['execution_suitability'],
+            'reason_summary': output['reason_summary'],
+            'error': None if continuous else 'Target crosses a missing candle or market closure.',
+        })
+    results = pd.DataFrame(rows)
+    summary = summarize_maythos(results)
+    return results, summary
+
+
+def summarize_maythos(results: pd.DataFrame) -> Dict[str, Any]:
+    if results.empty:
+        return {
+            'engine': 'MAYTHOS',
+            'evaluations': 0,
+            'actionable_signals': 0,
+            'blocked_signals': 0,
+            'correct': 0,
+            'incorrect': 0,
+            'accuracy_pct': None,
+            'mean_confidence': 0.0,
+            **summarize_trades(results),
+        }
+    actionable = results[results['signal_available']]
+    total = len(actionable)
+    correct = int(actionable['correct'].sum()) if total else 0
+    return {
+        'engine': 'MAYTHOS',
+        'evaluations': int(len(results)),
+        'actionable_signals': total,
+        'blocked_signals': int((~results['signal_available']).sum()),
+        'correct': correct,
+        'incorrect': total - correct,
+        'accuracy_pct': round(100.0 * correct / total, 2) if total else None,
+        'mean_confidence': round(float(actionable['confidence'].mean()), 6) if total else 0.0,
+        **summarize_trades(results),
+    }
+
+
+def latest_maythos_output(df: pd.DataFrame) -> Dict[str, Any]:
+    """Return the signal produced using only the final closed candle."""
+    frame = _normalise_frame(df)
+    return _run_engine(frame)[-1]

models/__init__.py

@@ -0,0 +1 @@
+pass

models/arima_garch_model.py

@@ -0,0 +1,104 @@
+import threading
+import numpy as np
+import pandas as pd
+from models.base_model import BaseForecastModel, causal_exogenous, validate_history_and_horizon
+from models.dependencies import fit_arima, import_arch_model, import_arima, quiet_fit
+from features.feature_pipeline import validate_features
+
+class ArimaGarchModel(BaseForecastModel):
+    name = 'ARIMA-GARCH'
+
+    def __init__(self, max_p=3, max_q=3, order=None, garch_p=1, garch_q=1, refit_order_every=None):
+        self.max_p = max_p
+        self.max_q = max_q
+        self._cached_order = order
+        self.garch_p = garch_p
+        self.garch_q = garch_q
+        self.refit_order_every = refit_order_every
+        self._calls_since_search = 0
+        self.last_volatility_forecast = None
+        self._lock = threading.Lock()
+        self.last_diagnostics = {}
+
+    def _best_order(self, returns: pd.Series, exog: pd.DataFrame=None):
+        arima = import_arima(self.name)
+        best_aic = np.inf
+        best_order = None
+        with quiet_fit():
+            for p, q in [(0, 0), (1, 0), (0, 1), (1, 1), (2, 0), (0, 2), (2, 1), (1, 2)]:
+                if p > self.max_p or q > self.max_q:
+                    continue
+                try:
+                    fit = fit_arima(arima(returns, exog=exog, order=(p, 0, q)))
+                except Exception:
+                    continue
+                if fit.aic < best_aic:
+                    best_aic = fit.aic
+                    best_order = (p, 0, q)
+        if best_order is None:
+            raise ValueError('No ARIMA-GARCH mean candidate converged on this training window.')
+        return best_order
+
+    def predict(self, history: pd.Series, horizon: int=1, features: pd.DataFrame=None) -> list:
+        with self._lock:
+            validate_history_and_horizon(history, horizon, self.name)
+            self.last_volatility_forecast = None
+            self.last_diagnostics = {}
+            prices = pd.Series(history).astype(float).reset_index(drop=True)
+            validate_history_and_horizon(prices, horizon, self.name)
+            if len(prices) < 30:
+                raise ValueError('ARIMA-GARCH needs at least 30 candles of history.')
+            if (prices <= 0).any():
+                raise ValueError('ARIMA-GARCH needs strictly positive prices (it uses log returns).')
+            arima = import_arima(self.name)
+            arch_model = import_arch_model(self.name)
+            # Fit returns in percentage units for a numerically well-scaled
+            # optimizer. Undo this scale exactly once for prices and volatility.
+            scale = 100.0
+            log_returns = np.log(prices / prices.shift(1)).dropna().reset_index(drop=True) * scale
+            exog = future_exog = None
+            if features is not None:
+                validate_features(features, expected_len=len(prices), model_name='ARIMA-GARCH')
+                exog, future_exog = causal_exogenous(features, horizon, self.name)
+            need_search = self._cached_order is None
+            if self.refit_order_every and self._calls_since_search >= self.refit_order_every:
+                need_search = True
+            if need_search:
+                self._cached_order = self._best_order(log_returns, exog=exog)
+                self._calls_since_search = 0
+            self._calls_since_search += 1
+            with quiet_fit():
+                mean_fit = fit_arima(arima(log_returns, exog=exog, order=self._cached_order))
+                if exog is not None:
+                    mean_forecast = np.asarray(mean_fit.forecast(steps=horizon, exog=future_exog), dtype=float)
+                else:
+                    mean_forecast = np.asarray(mean_fit.forecast(steps=horizon), dtype=float)
+                resid = pd.Series(mean_fit.resid).dropna()
+                try:
+                    garch_fit = arch_model(resid, mean='Zero', vol='GARCH', p=self.garch_p, q=self.garch_q, dist='t', rescale=False).fit(disp='off', options={'maxiter': 120})
+                    if garch_fit.convergence_flag != 0:
+                        raise ValueError('GARCH optimizer did not converge.')
+                    garch_forecast = garch_fit.forecast(horizon=horizon, reindex=False)
+                    variance = np.asarray(garch_forecast.variance.values[-1], dtype=float) / scale ** 2
+                    if variance.shape != (horizon,) or not np.isfinite(variance).all() or (variance < 0).any():
+                        raise ValueError('GARCH produced an invalid variance forecast.')
+                    self.last_diagnostics = {'converged': True, 'volatility_method': 'GARCH', 'garch_fallback': False}
+                except Exception as error:
+                    # The point forecast remains the fitted ARIMA mean. For a
+                    # short/degenerate sample, use the observed residual
+                    # variance as the honest unconditional volatility estimate
+                    # rather than returning a dummy price forecast.
+                    variance = np.full(horizon, float(resid.var(ddof=1) / scale ** 2) if len(resid) > 1 else 0.0)
+                    self.last_diagnostics = {'converged': True, 'volatility_method': 'unconditional residual variance', 'garch_fallback': True, 'garch_error': type(error).__name__}
+            if not np.isfinite(variance).all() or (variance < 0).any():
+                raise ValueError('No finite volatility estimate is available.')
+            self.last_volatility_forecast = [float(x) for x in np.sqrt(variance)]
+            last_price = float(prices.iloc[-1])
+            prices_out = []
+            cum_log_return = 0.0
+            for r in mean_forecast:
+                cum_log_return += float(r) / scale
+                prices_out.append(last_price * float(np.exp(cum_log_return)))
+            if len(prices_out) != horizon or not all(np.isfinite(x) and x > 0 for x in prices_out):
+                raise ValueError(f'ARIMA-GARCH produced a non-finite forecast for order {self._cached_order}.')
+            return prices_out

models/arima_model.py

@@ -0,0 +1,94 @@
+import threading
+import numpy as np
+import pandas as pd
+from models.base_model import BaseForecastModel, causal_exogenous, validate_history_and_horizon
+from models.dependencies import fit_arima, import_arima, quiet_fit
+from features.feature_pipeline import validate_features
+
+class ArimaModel(BaseForecastModel):
+    name = 'ARIMA'
+
+    def __init__(self, max_p=3, max_d=2, max_q=3, order=None, refit_order_every=None):
+        self.max_p = max_p
+        self.max_d = max_d
+        self.max_q = max_q
+        self._cached_order = order
+        self.refit_order_every = refit_order_every
+        self._calls_since_search = 0
+        self._lock = threading.Lock()
+        self.last_diagnostics = {}
+
+    def _best_order(self, series: pd.Series, exog: pd.DataFrame=None):
+        arima = import_arima(self.name)
+        best_aic = np.inf
+        best_order = None
+        # AIC values from different integration orders have different likelihood
+        # bases. Select d on the training sample, then compare only p/q at that d.
+        from models.auto_arima_model import AutoArimaModel
+        selected_d = AutoArimaModel(max_d=self.max_d)._find_d(series)
+        with quiet_fit():
+            candidates = [(p, d, q)
+                          for d in [selected_d]
+                          for p in range(self.max_p + 1)
+                          for q in range(self.max_q + 1)
+                          if p + q <= 2]
+            for order in candidates:
+                try:
+                    fit = fit_arima(arima(series, exog=exog, order=order))
+                except Exception:
+                    continue
+                if fit.aic < best_aic:
+                    best_aic = fit.aic
+                    best_order = order
+        if best_order is None:
+            raise ValueError('No ARIMA candidate converged on this training window.')
+        return best_order
+
+    def _fit_forecast(self, arima, series, order, horizon, exog, future_exog):
+        """Fit a positive-price ARIMA with deterministic, real fallbacks."""
+        candidates = [order, (0, 1, 0), (1, 1, 0), (0, 1, 1), (1, 1, 1)]
+        seen = set()
+        last_error = None
+        for candidate in candidates:
+            if candidate in seen:
+                continue
+            seen.add(candidate)
+            try:
+                with quiet_fit():
+                    fit = fit_arima(arima(series, exog=exog, order=candidate))
+                    forecast = fit.forecast(steps=horizon, exog=future_exog) if exog is not None else fit.forecast(steps=horizon)
+                values = np.exp(np.asarray(forecast, dtype=float))
+                if values.shape == (horizon,) and np.isfinite(values).all() and (values > 0).all():
+                    self._cached_order = candidate
+                    self.last_diagnostics = {'converged': True, 'selected_order': order, 'executed_order': candidate, 'order_fallback': candidate != order, 'feature_policy': 'lagged one candle; unknown future covariates persist'}
+                    return values
+            except Exception as error:
+                last_error = error
+        raise ValueError(f'ARIMA could not fit a stable positive-price model: {last_error}') from None
+
+    def predict(self, history: pd.Series, horizon: int=1, features: pd.DataFrame=None) -> list:
+        with self._lock:
+            validate_history_and_horizon(history, horizon, self.name)
+            self.last_diagnostics = {}
+            series = pd.Series(history).astype(float).reset_index(drop=True)
+            validate_history_and_horizon(series, horizon, self.name)
+            if len(series) < 20:
+                raise ValueError('ARIMA needs at least 20 candles of history.')
+            arima = import_arima(self.name)
+            exog = future_exog = None
+            if features is not None:
+                validate_features(features, expected_len=len(series), model_name='ARIMA')
+                exog, future_exog = causal_exogenous(features, horizon, self.name)
+                series = series.iloc[1:].reset_index(drop=True)
+            if (series <= 0).any():
+                raise ValueError('ARIMA needs strictly positive prices for its log-price model.')
+            log_series = np.log(series)
+            need_search = self._cached_order is None
+            if self.refit_order_every and self._calls_since_search >= self.refit_order_every:
+                need_search = True
+            if need_search:
+                self._cached_order = self._best_order(log_series, exog=exog)
+                self._calls_since_search = 0
+            self._calls_since_search += 1
+            forecast = self._fit_forecast(arima, log_series, self._cached_order, horizon, exog, future_exog)
+            return [float(x) for x in forecast]

models/auto_arima_model.py

@@ -0,0 +1,102 @@
+import threading
+import numpy as np
+import pandas as pd
+from models.base_model import BaseForecastModel, causal_exogenous, validate_history_and_horizon
+from models.dependencies import fit_arima, import_arima, import_kpss, quiet_fit
+from features.feature_pipeline import validate_features
+
+class AutoArimaModel(BaseForecastModel):
+    name = 'Auto-ARIMA'
+
+    def __init__(self, max_p=5, max_q=5, max_d=2, order=None, refit_order_every=None):
+        self.max_p = max_p
+        self.max_q = max_q
+        self.max_d = max_d
+        self._cached_order = order
+        self.refit_order_every = refit_order_every
+        self._calls_since_search = 0
+        self._lock = threading.Lock()
+        self.last_diagnostics = {}
+
+    def _find_d(self, series: pd.Series) -> int:
+        kpss = import_kpss(self.name)
+        s = series.copy()
+        d = 0
+        if float(s.std()) < 1e-12:
+            return 0
+        with quiet_fit():
+            while d < self.max_d:
+                try:
+                    _, p_value, _, _ = kpss(s, regression='c', nlags='auto')
+                except Exception as error:
+                    raise ValueError(f'Auto-ARIMA stationarity test failed ({type(error).__name__}).') from None
+                if p_value >= 0.05:
+                    break
+                s = s.diff().dropna()
+                d += 1
+        return d
+
+    def _aic(self, series: pd.Series, order, exog: pd.DataFrame=None) -> float:
+        arima = import_arima(self.name)
+        try:
+            return fit_arima(arima(series, exog=exog, order=order)).aic
+        except Exception:
+            return np.inf
+
+    def _stepwise_search(self, series: pd.Series, d: int, exog: pd.DataFrame=None):
+        # A compact Hyndman-style candidate set is more stable for noisy
+        # intraday data than repeatedly walking a large grid at every origin.
+        candidates = [(0, 0), (1, 0), (0, 1), (1, 1), (2, 0), (0, 2), (2, 1), (1, 2)]
+        candidates = [(p, q) for p, q in candidates if p <= self.max_p and q <= self.max_q]
+        best_order, best_aic = (None, np.inf)
+        with quiet_fit():
+            for p, q in candidates:
+                aic = self._aic(series, (p, d, q), exog=exog)
+                if aic < best_aic:
+                    best_aic, best_order = (aic, (p, q))
+            if best_order is None:
+                raise ValueError('No Auto-ARIMA candidate converged on this training window.')
+        p, q = best_order
+        return (p, d, q)
+
+    def predict(self, history: pd.Series, horizon: int=1, features: pd.DataFrame=None) -> list:
+        with self._lock:
+            validate_history_and_horizon(history, horizon, self.name)
+            self.last_diagnostics = {}
+            series = pd.Series(history).astype(float).reset_index(drop=True)
+            validate_history_and_horizon(series, horizon, self.name)
+            if len(series) < 20:
+                raise ValueError('Auto-ARIMA needs at least 20 candles of history.')
+            arima = import_arima(self.name)
+            if (series <= 0).any():
+                raise ValueError('Auto-ARIMA needs strictly positive prices for its log-price model.')
+            exog = future_exog = None
+            if features is not None:
+                validate_features(features, expected_len=len(series), model_name='Auto-ARIMA')
+                exog, future_exog = causal_exogenous(features, horizon, self.name)
+                series = series.iloc[1:].reset_index(drop=True)
+            series = np.log(series)
+            need_search = self._cached_order is None
+            if self.refit_order_every and self._calls_since_search >= self.refit_order_every:
+                need_search = True
+            if need_search:
+                d = self._find_d(series)
+                self._cached_order = self._stepwise_search(series, d, exog=exog)
+                self._calls_since_search = 0
+            self._calls_since_search += 1
+            candidates = [self._cached_order, (0, 1, 0), (1, 1, 0), (0, 1, 1), (1, 1, 1)]
+            last_error = None
+            for order in dict.fromkeys(candidates):
+                try:
+                    with quiet_fit():
+                        fit = fit_arima(arima(series, exog=exog, order=order))
+                        raw = fit.forecast(steps=horizon, exog=future_exog) if exog is not None else fit.forecast(steps=horizon)
+                    forecast = np.exp(np.asarray(raw, dtype=float))
+                    if forecast.shape == (horizon,) and np.isfinite(forecast).all() and (forecast > 0).all():
+                        selected_order = self._cached_order
+                        self._cached_order = order
+                        self.last_diagnostics = {'converged': True, 'selected_order': selected_order, 'executed_order': order, 'order_fallback': order != selected_order, 'feature_policy': 'lagged one candle; unknown future covariates persist'}
+                        return [float(x) for x in forecast]
+                except Exception as error:
+                    last_error = error
+            raise ValueError(f'Auto-ARIMA could not fit a stable positive-price model: {last_error}') from None

models/base_model.py

@@ -0,0 +1,71 @@
+import numpy as np
+import pandas as pd
+from abc import ABC, abstractmethod
+
+def validate_history_and_horizon(history, horizon: int, model_name: str) -> None:
+    if isinstance(horizon, bool) or not isinstance(horizon, (int, np.integer)) or horizon < 1:
+        raise ValueError(f'{model_name}: horizon must be >= 1, got {horizon}.')
+    values = np.asarray(pd.Series(history).astype(float), dtype=float)
+    if values.ndim != 1 or values.size == 0:
+        raise ValueError(f'{model_name}: price history must contain at least one candle.')
+    if not np.all(np.isfinite(values)):
+        raise ValueError(f'{model_name}: price history contains NaN/inf. This usually means a data glitch left a missing or malformed candle -- re-fetch the history and try again.')
+    if (values <= 0).any():
+        raise ValueError(f'{model_name}: price history must be strictly positive.')
+    if isinstance(history, pd.Series) and isinstance(history.index, pd.DatetimeIndex):
+        if history.index.isna().any() or history.index.duplicated().any() or not history.index.is_monotonic_increasing:
+            raise ValueError(f'{model_name}: history timestamps must be ordered and unique.')
+
+
+def standardize_exogenous(features: pd.DataFrame, model_name: str) -> pd.DataFrame:
+    """Scale causal regressors using statistics from the supplied history only.
+
+    OHLC prices and oscillator values have very different magnitudes. Passing
+    them unscaled to statsmodels frequently creates singular matrices on short
+    windows, especially for FX. Scaling here is causal (the future is never
+    included) and keeps the same columns so the feature contract remains
+    auditable.
+    """
+    if not isinstance(features, pd.DataFrame) or features.empty:
+        raise ValueError(f'{model_name}: features must be a non-empty DataFrame.')
+    numeric = features.apply(pd.to_numeric, errors='coerce').astype(float)
+    if not np.isfinite(numeric.to_numpy()).all():
+        raise ValueError(f'{model_name}: features contain non-finite values.')
+    mean = numeric.mean(axis=0)
+    scale = numeric.std(axis=0, ddof=0).replace(0.0, 1.0).fillna(1.0)
+    scaled = (numeric - mean) / scale
+    # A single outlier should not make the optimizer overflow. This is a
+    # bounded transform, not an imputation or a fabricated observation.
+    return scaled.clip(-12.0, 12.0)
+
+
+def causal_exogenous(features, horizon, model_name):
+    """Pair target at t with covariates at t-1, using training-only scaling.
+
+    OHLC(t) and indicators(t) contain information about Close(t). They cannot
+    be used as exogenous predictors of that same target. The last observed
+    covariate is instead the legitimate first future regressor; later unknown
+    covariates explicitly use persistence, not realized future candles.
+    """
+    numeric = features.apply(pd.to_numeric, errors='coerce').astype(float).reset_index(drop=True)
+    if len(numeric) < 2 or not np.isfinite(numeric.to_numpy()).all():
+        raise ValueError(f'{model_name}: causal features need at least two finite rows.')
+    train = numeric.iloc[:-1].reset_index(drop=True)
+    mean = train.mean()
+    scale = train.std(ddof=0).replace(0, 1).fillna(1)
+    # Constant columns cannot identify a regression coefficient and may conflict
+    # with the ARIMA trend. Their future value cannot affect a learned fit.
+    columns = train.columns[train.nunique() > 1]
+    if len(columns) == 0:
+        return None, None
+    scaled = ((train[columns] - mean[columns]) / scale[columns]).clip(-12, 12)
+    future_row = ((numeric.iloc[[-1]][columns] - mean[columns]) / scale[columns]).clip(-12, 12)
+    future = pd.concat([future_row] * horizon, ignore_index=True)
+    return scaled, future
+
+class BaseForecastModel(ABC):
+    name = 'base'
+
+    @abstractmethod
+    def predict(self, history: pd.Series, horizon: int=1, features: pd.DataFrame=None) -> list:
+        raise NotImplementedError

models/dependencies.py

@@ -0,0 +1,95 @@
+import contextlib
+import inspect
+import warnings
+
+def _missing(package: str, model: str, error: Exception) -> ImportError:
+    return ImportError(f"{model} needs the '{package}' package. Install with:\n  pip install {package}\n(original error: {error})")
+
+def _patch_statsmodels_pandas_compat() -> None:
+    """Bridge the legacy decorator call used by statsmodels 0.14.x."""
+    try:
+        import pandas.util._decorators as pandas_decorators
+        pandas_original = pandas_decorators.deprecate_kwarg
+        pandas_parameters = list(inspect.signature(pandas_original).parameters.values())
+        if (
+            pandas_parameters
+            and pandas_parameters[0].name == 'klass'
+            and not getattr(pandas_original, "_rinking_compat", False)
+        ):
+            def pandas_compatible_deprecate_kwarg(*args, **kwargs):
+                if len(args) >= 2 and all(isinstance(value, str) for value in args[:2]):
+                    return pandas_original(FutureWarning, args[0], args[1], *args[2:], **kwargs)
+                return pandas_original(*args, **kwargs)
+
+            pandas_compatible_deprecate_kwarg._rinking_compat = True
+            pandas_decorators.deprecate_kwarg = pandas_compatible_deprecate_kwarg
+    except ImportError:
+        pass
+    try:
+        import statsmodels.compat.pandas as compat
+    except ImportError:
+        return
+    signature = inspect.signature(compat.deprecate_kwarg)
+    if len(signature.parameters) != 5:
+        return
+    original = compat.deprecate_kwarg
+    if getattr(original, "_rinking_compat", False):
+        return
+
+    def compatible_deprecate_kwarg(*args, **kwargs):
+        if len(args) >= 2 and all(isinstance(value, str) for value in args[:2]):
+            return original(FutureWarning, args[0], args[1], *args[2:], **kwargs)
+        return original(*args, **kwargs)
+
+    compatible_deprecate_kwarg._rinking_compat = True
+    compat.deprecate_kwarg = compatible_deprecate_kwarg
+
+def import_arima(model_name: str):
+    try:
+        _patch_statsmodels_pandas_compat()
+        from statsmodels.tsa.arima.model import ARIMA
+    except ImportError as e:
+        raise _missing('statsmodels', model_name, e) from e
+    return ARIMA
+
+def import_kpss(model_name: str):
+    try:
+        _patch_statsmodels_pandas_compat()
+        from statsmodels.tsa.stattools import kpss
+    except ImportError as e:
+        raise _missing('statsmodels', model_name, e) from e
+    return kpss
+
+def import_arch_model(model_name: str):
+    try:
+        _patch_statsmodels_pandas_compat()
+        from arch import arch_model
+    except ImportError as e:
+        raise _missing('arch', model_name, e) from e
+    return arch_model
+
+def fit_arima(model):
+    """Fit statsmodels ARIMA with a bounded optimizer budget.
+
+    Walk-forward evaluation can fit hundreds of models. An unbounded BFGS
+    call makes a free Space look hung while adding little value to a short
+    intraday window; the selected order is still re-estimated from each
+    training window.
+    """
+    fit = model.fit(method_kwargs={'maxiter': 120, 'disp': False})
+    if not getattr(fit, 'mle_retvals', {}).get('converged', True):
+        raise ValueError('ARIMA optimizer did not converge within the bounded iteration budget.')
+    if not __import__('numpy').isfinite(fit.params).all():
+        raise ValueError('ARIMA fit produced non-finite parameters.')
+    return fit
+
+@contextlib.contextmanager
+def quiet_fit():
+    with warnings.catch_warnings():
+        # Expected candidate-selection warnings only. Convergence is checked by
+        # fit_arima; unrelated warnings are not blanket-suppressed.
+        warnings.filterwarnings('ignore', message='Non-stationary starting autoregressive parameters.*')
+        warnings.filterwarnings('ignore', message='Non-invertible starting MA parameters.*')
+        warnings.filterwarnings('ignore', message='Maximum Likelihood optimization failed to converge.*')
+        warnings.filterwarnings('ignore', message='The test statistic is outside of the range.*')
+        yield

models/moirai_model.py

@@ -0,0 +1,129 @@
+import numpy as np
+import pandas as pd
+import threading
+from models.base_model import BaseForecastModel, validate_history_and_horizon
+from config import MOIRAI_CHECKPOINT, RANDOM_SEED
+from features.feature_pipeline import validate_features
+_RNG_LOCK = threading.Lock()
+_MODULE_CACHE = {}
+_MODULE_LOCK = threading.Lock()
+
+
+def _step_from_history(history: pd.Series) -> pd.Timedelta:
+    if isinstance(history.index, pd.DatetimeIndex) and len(history.index) >= 2:
+        diffs = history.index.to_series().diff().dropna()
+        if not diffs.empty:
+            step = diffs.median()
+            if step > pd.Timedelta(0):
+                return step
+    return pd.Timedelta(minutes=1)
+
+
+class MoiraiModel(BaseForecastModel):
+    name = "Moirai"
+
+    def __init__(
+        self,
+        checkpoint: str = MOIRAI_CHECKPOINT,
+        context_length: int = 256,
+        num_samples: int = 20,
+    ):
+        self.checkpoint = checkpoint
+        self.context_length = context_length
+        self.num_samples = num_samples
+        self._module = None
+        self.last_diagnostics = {}
+
+    def _get_module(self):
+        from uni2ts.model.moirai import MoiraiModule
+
+        if self._module is None:
+            with _MODULE_LOCK:
+                self._module = _MODULE_CACHE.get(self.checkpoint)
+                if self._module is None:
+                    self._module = MoiraiModule.from_pretrained(self.checkpoint)
+                    self._module.eval()
+                    _MODULE_CACHE[self.checkpoint] = self._module
+        return self._module
+
+    def predict(
+        self, history: pd.Series, horizon: int = 1, features: pd.DataFrame = None
+    ) -> list:
+        validate_history_and_horizon(history, horizon, self.name)
+        try:
+            import torch
+        except ImportError as error:
+            raise ImportError('Moirai requires the optional foundation-model packages. Install requirements-foundation.txt.') from error
+        with _RNG_LOCK, torch.random.fork_rng(devices=[]), torch.inference_mode():
+            torch.manual_seed(RANDOM_SEED)
+            return self._predict(history, horizon, features)
+
+    def _predict(self, history, horizon, features):
+        values = np.asarray(history, dtype=np.float32)
+        if len(values) < 10:
+            raise ValueError("Moirai needs at least 10 candles of history.")
+        validate_history_and_horizon(values, horizon, self.name)
+        try:
+            from uni2ts.model.moirai import MoiraiForecast
+            from gluonts.dataset.pandas import PandasDataset
+        except ImportError as e:
+            raise ImportError(
+                f"Moirai needs the 'uni2ts' and 'gluonts' packages. Install with:\n  pip install uni2ts torch\n(original error: {e})"
+            )
+        if features is not None:
+            validate_features(
+                features,
+                expected_len=len(values),
+                model_name="Moirai",
+                check_nan=False,
+            )
+        # Bound the context for the small checkpoint so a large dashboard
+        # window cannot cause an avoidable CPU-memory spike.
+        context_length = min(max(int(self.context_length), 32), 512)
+        context = values[-context_length:] if len(values) > context_length else values
+        step = _step_from_history(history)
+        end_ts = pd.Timestamp(history.index[-1]) if isinstance(history.index, pd.DatetimeIndex) else pd.Timestamp('2000-01-01')
+        if end_ts.tzinfo is not None:
+            end_ts = end_ts.tz_convert('UTC').tz_localize(None)
+        idx = pd.date_range(end=end_ts, periods=len(context), freq=step)
+        data = {"target": context}
+        feat_cols = []
+        if features is not None:
+            feat_context = features.iloc[-len(context) :].reset_index(drop=True)
+            validate_features(
+                feat_context, expected_len=len(context), model_name="Moirai"
+            )
+            for col in feat_context.columns:
+                data[col] = feat_context[col].to_numpy(dtype=np.float32)
+            feat_cols = list(feat_context.columns)
+        series_df = pd.DataFrame(data, index=idx)
+        dataset = PandasDataset(
+            series_df,
+            target="target",
+            past_feat_dynamic_real=feat_cols if feat_cols else None,
+        )
+        module = self._get_module()
+        forecaster = MoiraiForecast(
+            module=module,
+            prediction_length=horizon,
+            context_length=len(context),
+            patch_size="auto",
+            num_samples=self.num_samples,
+            target_dim=1,
+            feat_dynamic_real_dim=0,
+            past_feat_dynamic_real_dim=len(feat_cols),
+        )
+        predictor = forecaster.create_predictor(batch_size=1)
+        forecast = next(iter(predictor.predict(dataset)))
+        samples = np.asarray(forecast.samples, dtype=float)
+        if samples.ndim == 3 and samples.shape[-1] == 1:
+            samples = samples[..., 0]
+        if samples.ndim != 2 or samples.shape != (self.num_samples, horizon):
+            raise ValueError('Moirai returned an unexpected univariate sample/horizon shape.')
+        out = np.median(samples, axis=0).tolist()
+        if not all(np.isfinite(x) and x > 0 for x in out):
+            raise ValueError(
+                "Moirai produced a non-finite forecast (NaN/inf) for this window."
+            )
+        self.last_diagnostics = {'checkpoint': self.checkpoint, 'context_candles': len(context), 'point_statistic': 'sample median', 'seed': RANDOM_SEED, 'fallback': False}
+        return out

models/registry.py

@@ -0,0 +1,44 @@
+import threading
+from collections import OrderedDict
+from models.base_model import BaseForecastModel
+_ORDER_SEARCH_MODELS = {'ARIMA', 'Auto-ARIMA', 'ARIMA-GARCH'}
+_CACHE = OrderedDict()
+_MAX_CACHE = 32
+_CACHE_LOCK = threading.Lock()
+
+def _build_model(name: str) -> BaseForecastModel:
+    if name == 'ARIMA':
+        from models.arima_model import ArimaModel
+        return ArimaModel()
+    elif name == 'Auto-ARIMA':
+        from models.auto_arima_model import AutoArimaModel
+        return AutoArimaModel()
+    elif name == 'ARIMA-GARCH':
+        from models.arima_garch_model import ArimaGarchModel
+        return ArimaGarchModel()
+    elif name == 'Moirai':
+        from models.moirai_model import MoiraiModel
+        return MoiraiModel()
+    elif name == 'TimesFM':
+        from models.timesfm_model import TimesFMModel
+        return TimesFMModel()
+    raise ValueError(f"Unknown model '{name}'. Expected one of: ARIMA, Auto-ARIMA, ARIMA-GARCH, Moirai, TimesFM.")
+
+def get_model(name: str, symbol: str=None, timeframe: str=None, use_features: bool=False) -> BaseForecastModel:
+    if name in _ORDER_SEARCH_MODELS:
+        variant = 'featured' if use_features else 'base'
+        key = (name, symbol, timeframe, variant)
+    else:
+        key = name
+    with _CACHE_LOCK:
+        if key in _CACHE:
+            _CACHE.move_to_end(key)
+            return _CACHE[key]
+        model = _build_model(name)
+        if len(_CACHE) >= _MAX_CACHE:
+            _CACHE.popitem(last=False)
+        _CACHE[key] = model
+        return model
+
+def fresh_model(name: str) -> BaseForecastModel:
+    return _build_model(name)

models/timesfm_model.py

@@ -0,0 +1,107 @@
+import numpy as np
+import pandas as pd
+import threading
+from models.base_model import BaseForecastModel, validate_history_and_horizon
+from config import TIMESFM_CHECKPOINT
+from features.feature_pipeline import carry_forward_future, validate_features
+_MAX_HORIZON = 64
+_MODEL_CACHE = {}
+_MODEL_CACHE_LOCK = threading.RLock()
+_COMPILED_CONTEXT = {}
+
+class TimesFMModel(BaseForecastModel):
+    name = 'TimesFM'
+
+    def __init__(self, checkpoint: str=TIMESFM_CHECKPOINT, max_context: int=512):
+        self.checkpoint = checkpoint
+        self.max_context = ((min(max(int(max_context), 32), 512) + 31) // 32) * 32
+        self._model = None
+        self._compiled_context = None
+        self._lock = threading.Lock()
+        self.last_diagnostics = {}
+
+    @staticmethod
+    def _forecast_config(max_context: int):
+        import timesfm
+        return timesfm.ForecastConfig(max_context=max_context, max_horizon=_MAX_HORIZON, normalize_inputs=True, use_continuous_quantile_head=True, force_flip_invariance=True, infer_is_positive=True, fix_quantile_crossing=True, return_backcast=True)
+
+    def _load(self):
+        import torch
+        import timesfm
+        if self._model is None:
+            with _MODEL_CACHE_LOCK:
+                self._model = _MODEL_CACHE.get(self.checkpoint)
+                if self._model is None:
+                    torch.set_float32_matmul_precision('high')
+                    model_class = getattr(timesfm, 'TimesFM_2p5_200M_torch', None)
+                    if model_class is None:
+                        raise ImportError('Installed timesfm package does not expose the TimesFM 2.5 PyTorch model.')
+                    model = model_class.from_pretrained(self.checkpoint, torch_compile=False)
+                    model.compile(self._forecast_config(self.max_context))
+                    self._model = model
+                    _MODEL_CACHE[self.checkpoint] = model
+                    _COMPILED_CONTEXT[self.checkpoint] = self.max_context
+                self._compiled_context = _COMPILED_CONTEXT[self.checkpoint]
+        return self._model
+
+    def _ensure_compiled_for(self, context_len: int):
+        model = self._load()
+        patch = getattr(model.model, 'p', 32)
+        limit = min(512, max(32, int(self.max_context)))
+        limit = min(512, ((limit + patch - 1) // patch) * patch)
+        self.max_context = limit
+        desired = min(limit, max(patch, (context_len + patch - 1) // patch * patch))
+        if desired != _COMPILED_CONTEXT[self.checkpoint]:
+            model.compile(self._forecast_config(desired))
+            _COMPILED_CONTEXT[self.checkpoint] = desired
+        self._compiled_context = desired
+        return model
+
+    def predict(self, history: pd.Series, horizon: int=1, features: pd.DataFrame=None) -> list:
+        # Cached model weights AND compilation state are shared by wrappers.
+        # Per-wrapper locks cannot protect that shared mutable runtime.
+        with self._lock, _MODEL_CACHE_LOCK:
+            return self._predict(history, horizon, features)
+
+    def _predict(self, history, horizon, features):
+        validate_history_and_horizon(history, horizon, self.name)
+        values = np.asarray(history, dtype=np.float32)
+        validate_history_and_horizon(values, horizon, self.name)
+        if len(values) < 32:
+            raise ValueError('TimesFM needs at least 32 candles of history.')
+        try:
+            __import__('timesfm')
+        except ImportError as e:
+            raise ImportError(f"TimesFM needs the 'timesfm' package. Install with:\n  pip install timesfm[torch]\n(original error: {e})")
+        if horizon > _MAX_HORIZON:
+            raise ValueError(f'This TimesFM wrapper supports at most {_MAX_HORIZON} candles ahead.')
+        if features is not None:
+            validate_features(features, expected_len=len(values), model_name='TimesFM', check_nan=False)
+        context = values[-self.max_context:] if len(values) > self.max_context else values
+        model = self._ensure_compiled_for(len(context))
+        if features is None:
+            point_forecast, _quantiles = model.forecast(horizon=horizon, inputs=[context])
+            point_values = np.asarray(point_forecast[0], dtype=float)
+        else:
+            feat_context = features.iloc[-len(context):].reset_index(drop=True)
+            validate_features(feat_context, expected_len=len(context), model_name='TimesFM')
+            feat_future = carry_forward_future(feat_context, horizon)
+            dynamic_numerical_covariates = {}
+            for col in feat_context.columns:
+                # xreg is a fitted regression, so contemporaneous OHLC/indicator
+                # covariates leak the training target. Align features(t-1) to
+                # prices(t), followed by the last observed feature at the origin.
+                full_array = np.concatenate([feat_context[col].iloc[:-1].to_numpy(dtype=np.float32), feat_future[col].to_numpy(dtype=np.float32)])
+                dynamic_numerical_covariates[col] = [full_array]
+            try:
+                point_forecast, _quantiles = model.forecast_with_covariates(inputs=[context[1:]], dynamic_numerical_covariates=dynamic_numerical_covariates, xreg_mode='xreg + timesfm')
+                point_values = np.asarray(point_forecast[0], dtype=float)
+            except (ImportError, AttributeError) as e:
+                raise ImportError(f"TimesFM's covariate forecasting needs the 'xreg' extra (scikit-learn + JAX). Install with:\n  pip install timesfm[torch,xreg]\n(original error: {e})")
+        if point_values.shape != (horizon,):
+            raise ValueError('TimesFM returned an incorrect forecast horizon/shape.')
+        out = point_values.tolist()
+        if not all(np.isfinite(x) and x > 0 for x in out):
+            raise ValueError('TimesFM returned a non-finite forecast (NaN/inf) for this window. This is usually a sign the input window is degenerate (e.g. a flat or near-constant price run) — try a different window size or symbol.')
+        self.last_diagnostics = {'checkpoint': self.checkpoint, 'context_candles': len(context), 'feature_policy': 'lagged xreg; unknown future covariates persist', 'fallback': False}
+        return out

pipeline.py

@@ -0,0 +1,360 @@
+"""Shared, explicit pipeline used by both dashboard tabs and the live logger."""
+from __future__ import annotations
+
+from concurrent.futures import ThreadPoolExecutor
+from dataclasses import asdict, dataclass
+import hashlib
+import importlib.metadata
+import importlib.util
+import json
+from pathlib import Path
+import platform
+import uuid
+import zipfile
+
+import numpy as np
+import pandas as pd
+
+from backtest.backtester import backtest_model, summarize_backtest
+from config import MODEL_NAMES, OUTPUT_DIR, PROVIDERS, RANDOM_SEED, TIMEFRAMES
+from data.providers import fetch_market_history, normalize_symbol
+from features.feature_pipeline import compute_feature_frame_for_period
+from maythos_integration import backtest_maythos, latest_maythos_output
+from models.registry import fresh_model
+from risk_management import CostModel, build_trade_levels
+from utils.helpers import MINUTES_PER_CANDLE, feature_warmup_days, price_direction
+
+
+@dataclass(frozen=True)
+class RunConfig:
+    provider: str = 'Yahoo Finance'
+    market: str = 'Crypto'
+    symbol: str = 'BTC-USD'
+    timeframe: str = '5m'
+    mode: str = 'Single model'
+    models: tuple = ('ARIMA-GARCH',)
+    lookback_days: int = 30
+    window: int = 200
+    horizon: int = 1
+    step: int = 30
+    max_evaluations: int = 30
+    use_features: bool = False
+    spread_bps: float = 1.0
+    fee_bps: float = 1.0
+    slippage_bps: float = 1.0
+
+    def __post_init__(self):
+        if self.provider not in PROVIDERS or self.timeframe not in TIMEFRAMES:
+            raise ValueError('Choose a supported provider and timeframe.')
+        if self.mode not in ('Single model', 'Parallel models'):
+            raise ValueError('Choose Single model or Parallel models.')
+        if isinstance(self.models, str):
+            raise ValueError('models must be a sequence of model names, not a string.')
+        models = tuple(dict.fromkeys(self.models or ()))
+        if not models or any(name not in MODEL_NAMES for name in models):
+            raise ValueError('Select at least one supported model.')
+        if self.mode == 'Single model' and len(models) != 1:
+            raise ValueError('Single model mode requires exactly one model. Select Parallel models to run several.')
+        for name, minimum, maximum in (
+            ('lookback_days', 1, 730), ('window', 32, 2048), ('horizon', 1, 64),
+            ('step', 1, 10000), ('max_evaluations', 1, 1000),
+        ):
+            value = getattr(self, name)
+            try:
+                integral = int(value)
+                valid_number = np.isfinite(float(value))
+            except (TypeError, ValueError, OverflowError):
+                integral, valid_number = 0, False
+            if isinstance(value, (bool, np.bool_)) or not valid_number or integral != value or not minimum <= value <= maximum:
+                raise ValueError(f'{name} must be an integer between {minimum} and {maximum}.')
+            object.__setattr__(self, name, int(value))
+        for name in ('spread_bps', 'fee_bps', 'slippage_bps'):
+            value = getattr(self, name)
+            try:
+                finite = np.isfinite(value)
+            except TypeError:
+                finite = False
+            if isinstance(value, bool) or not finite or value < 0 or value > 1000:
+                raise ValueError(f'{name} must be a finite number between 0 and 1000 bps.')
+            object.__setattr__(self, name, float(value))
+        object.__setattr__(self, 'models', models)
+        object.__setattr__(self, 'symbol', normalize_symbol(self.symbol, self.provider, self.market))
+
+    @property
+    def candle_step(self):
+        return pd.Timedelta(minutes=MINUTES_PER_CANDLE[self.timeframe])
+
+    @property
+    def costs(self):
+        return CostModel(self.spread_bps, self.fee_bps, self.slippage_bps)
+
+
+def model_readiness():
+    required = {
+        'ARIMA': ['statsmodels'], 'Auto-ARIMA': ['statsmodels'],
+        'ARIMA-GARCH': ['statsmodels', 'arch'],
+        'Moirai': ['uni2ts', 'gluonts', 'torch'], 'TimesFM': ['timesfm', 'torch'],
+    }
+    rows = []
+    for name, packages in required.items():
+        missing = [package for package in packages if importlib.util.find_spec(package) is None]
+        rows.append({
+            'Model': name,
+            'Dependency status': 'Missing: ' + ', '.join(missing) if missing else 'Packages detected',
+            'Runtime requirement': 'Pretrained weights download on first use; inference must succeed' if name in ('Moirai', 'TimesFM') else 'Fits actual selected history',
+        })
+    return pd.DataFrame(rows)
+
+
+def prepare_history(config):
+    padding = feature_warmup_days(config.timeframe, minimum_days=0) if config.use_features else 0
+    frame = fetch_market_history(config.provider, config.market, config.symbol, config.timeframe, config.lookback_days + padding)
+    attrs = dict(frame.attrs)
+    features = None
+    if config.use_features:
+        frame, features = compute_feature_frame_for_period(frame, keep_days=config.lookback_days, timeframe=config.timeframe)
+        frame.attrs.update(attrs)
+    if len(frame) < config.window:
+        raise ValueError(f'Only {len(frame)} closed candles are available; the selected history window needs {config.window}. Increase lookback, reduce the window, or use a provider with deeper history.')
+    return frame, features
+
+
+def new_configured_model(name, config):
+    model = fresh_model(name)
+    if hasattr(model, 'refit_order_every'):
+        model.refit_order_every = 1
+    if hasattr(model, 'context_length'):
+        model.context_length = min(config.window, 512)
+    if hasattr(model, 'max_context'):
+        model.max_context = min(config.window, 512)
+    return model
+
+
+def data_metadata(frame, config):
+    if frame is None or frame.empty:
+        raise ValueError('Cannot create run metadata without closed candles.')
+    now = pd.Timestamp.now(tz='UTC')
+    last = pd.Timestamp(frame.index[-1])
+    if last.tzinfo is None:
+        last = last.tz_localize('UTC')
+    else:
+        last = last.tz_convert('UTC')
+    if last > now + config.candle_step:
+        raise ValueError('The latest candle is in the future relative to the runtime clock.')
+    first = pd.Timestamp(frame.index[0])
+    if first.tzinfo is None:
+        first = first.tz_localize('UTC')
+    else:
+        first = first.tz_convert('UTC')
+    age_seconds = max(0.0, (now - (last + config.candle_step)).total_seconds())
+    freshness_limit = config.candle_step.total_seconds() * 3
+    stale = age_seconds > freshness_limit
+    return {
+        'provider': config.provider, 'market': config.market, 'symbol': config.symbol,
+        'timeframe': config.timeframe, 'closed_candles': len(frame),
+        'first_candle_open_utc': first.isoformat(),
+        'last_candle_open_utc': last.isoformat(),
+        'last_candle_close_utc': (last + config.candle_step).isoformat(),
+        'fetched_at_utc': now.isoformat(), 'seconds_since_last_candle_close': round(age_seconds, 3),
+        'stale': stale,
+        'freshness_status': 'STALE' if stale else 'FRESH',
+        'freshness_policy': f'latest closed candle must be no older than {freshness_limit:g} seconds',
+        'noncontinuous_intervals': int((frame.index.to_series().diff().dropna() != config.candle_step).sum()),
+        'provider_details': dict(frame.attrs),
+    }
+
+
+def _map_models(config, worker):
+    if config.mode == 'Parallel models' and len(config.models) > 1:
+        # Loading both foundation checkpoints at once can exhaust a free CPU
+        # Space. They remain independent model executions, but are scheduled
+        # serially when either heavyweight model is selected.
+        foundation = {'Moirai', 'TimesFM'}
+        workers = 1 if foundation.intersection(config.models) else min(2, len(config.models))
+        with ThreadPoolExecutor(max_workers=workers) as pool:
+            return list(pool.map(worker, config.models))
+    return [worker(name) for name in config.models]
+
+
+def live_result(config, frame=None, features=None):
+    if frame is None:
+        frame, features = prepare_history(config)
+    elif config.use_features and features is None:
+        raise ValueError('This configuration requires an aligned feature frame; pass features or fetch history through prepare_history.')
+    elif not config.use_features:
+        features = None
+    history = frame['Close'].iloc[-config.window:]
+    feature_window = features.iloc[-config.window:].reset_index(drop=True) if features is not None else None
+    current = float(history.iloc[-1])
+    metadata = data_metadata(frame, config)
+
+    def forecast_one(name):
+        try:
+            model = new_configured_model(name, config)
+            values = model.predict(history, horizon=config.horizon, features=feature_window)
+            if len(values) != config.horizon or not np.isfinite(values).all() or any(value <= 0 for value in values):
+                raise ValueError('Model did not return a positive, finite forecast for the full horizon.')
+            predicted = float(values[-1])
+            return {
+                'Model': name, 'Status': 'STALE — historical forecast only' if metadata['stale'] else 'OK',
+                'Current close': current, 'Predicted close': predicted,
+                'Change (%)': (predicted / current - 1) * 100,
+                'Direction': price_direction(predicted, current).upper(),
+                'Order': str(getattr(model, '_cached_order', 'pretrained')),
+                'Error': '', 'forecast_path': [float(value) for value in values],
+            }
+        except Exception as error:
+            return {'Model': name, 'Status': 'FAILED', 'Current close': current, 'Predicted close': None,
+                    'Change (%)': None, 'Direction': None, 'Order': '', 'Error': str(error), 'forecast_path': []}
+
+    forecasts = _map_models(config, forecast_one)
+    successful_forecasts = [item for item in forecasts if item.get('forecast_path')]
+    if successful_forecasts:
+        directions = [item['Direction'] for item in successful_forecasts]
+        consensus = directions[0] if all(direction == directions[0] for direction in directions) else 'MIXED'
+    else:
+        consensus = 'UNAVAILABLE'
+    try:
+        signal = latest_maythos_output(frame)
+        signal = {
+            **signal,
+            'forecast_models_completed': len(successful_forecasts),
+            'forecast_models_requested': len(config.models),
+            'forecast_consensus': consensus,
+            'forecast_agrees_with_signal': (
+                consensus == {'BUY': 'UP', 'SELL': 'DOWN'}.get(signal.get('direction'))
+            ),
+            'forecast_coverage_pct': round(100.0 * len(successful_forecasts) / len(config.models), 2),
+            'live_actionable': (
+                not metadata['stale'] and len(successful_forecasts) == len(config.models) and
+                consensus in ('UP', 'DOWN') and
+                consensus == {'BUY': 'UP', 'SELL': 'DOWN'}.get(signal.get('direction')) and
+                not signal['blocked_flag'] and
+                signal['execution_suitability'] == 'strong' and
+                float(signal.get('confidence', 0.0)) > 0.69
+            ),
+        }
+        if signal['live_actionable']:
+            targets = [float(item['Predicted close']) for item in successful_forecasts if item.get('Predicted close') is not None]
+            forecast_target = float(np.median(targets)) if targets else None
+            levels = build_trade_levels(
+                frame, signal['direction'], config.market, config.symbol,
+                forecast_target=forecast_target,
+                forecast_only=True,
+                minimum_edge_atr=0.20,
+                minimum_risk_reward=1.20,
+                use_structure_stop=False,
+                minimum_edge_bps=(
+                    config.spread_bps + config.fee_bps + 2.0 * config.slippage_bps
+                ) * 1.25,
+            )
+            if levels is None:
+                signal['live_actionable'] = False
+                signal['trade_levels_error'] = 'No causal ATR/swing structure offered a valid positive-risk target.'
+            else:
+                signal['trade_levels'] = levels.as_dict()
+    except Exception as error:
+        signal = {'error': str(error), 'live_actionable': False}
+    return {'config': config, 'frame': frame, 'features': features, 'metadata': metadata, 'forecasts': forecasts, 'signal': signal}
+
+
+def backtest_result(config, frame=None, features=None):
+    if frame is None:
+        frame, features = prepare_history(config)
+    elif config.use_features and features is None:
+        raise ValueError('This configuration requires an aligned feature frame; pass features or fetch history through prepare_history.')
+    elif not config.use_features:
+        features = None
+    if len(frame) < config.window + config.horizon:
+        raise ValueError(f'Backtesting needs at least {config.window + config.horizon} closed candles; provider returned {len(frame)}.')
+
+    def evaluate_one(name):
+        try:
+            model = new_configured_model(name, config)
+            details = backtest_model(
+                frame, model, window=config.window, horizon=config.horizon,
+                step=config.step, features_df=features,
+                max_evaluations=config.max_evaluations, timeframe=config.timeframe,
+                market=config.market, symbol=config.symbol, costs=config.costs,
+            )
+            summary = summarize_backtest(details, name)
+        except Exception as error:
+            details = pd.DataFrame([{
+                'datetime': None, 'current_close': np.nan,
+                'predicted_close': np.nan, 'actual_close': np.nan,
+                'predicted_direction': None, 'actual_direction': None, 'correct': None,
+                'continuous_target': False, 'trade_taken': False, 'entry': np.nan,
+                'stop_loss': np.nan, 'take_profit': np.nan, 'risk_reward': np.nan,
+                'atr': np.nan, 'outcome': 'MODEL_FAILED', 'exit_price': np.nan,
+                'net_r': np.nan, 'bars_held': 0, 'error': str(error),
+            }])
+            summary = summarize_backtest(details, name)
+            summary['error'] = str(error)
+        summary['Model'] = summary.pop('model')
+        valid = details.dropna(subset=['correct'])
+        summary['evaluations_attempted'] = len(details)
+        summary['coverage_pct'] = round(100 * len(valid) / len(details), 2) if len(details) else None
+        summary['persistence_mae'] = float(np.abs(valid['current_close'] - valid['actual_close']).mean()) if len(valid) else None
+        summary['cost_spread_bps'] = config.spread_bps
+        summary['cost_fee_bps'] = config.fee_bps
+        summary['cost_slippage_bps'] = config.slippage_bps
+        summary['status'] = 'OK' if summary['failed_predictions'] == 0 else ('PARTIAL' if len(valid) else 'FAILED')
+        details.insert(0, 'Model', name)
+        return summary, details
+
+    evaluated = _map_models(config, evaluate_one)
+    try:
+        signal_details, signal_summary = backtest_maythos(
+            frame, window=config.window, horizon=config.horizon, step=config.step,
+            max_evaluations=config.max_evaluations, timeframe=config.timeframe,
+            market=config.market, symbol=config.symbol, costs=config.costs,
+        )
+    except Exception as error:
+        signal_details = pd.DataFrame()
+        signal_summary = {'engine': 'MAYTHOS', 'error': str(error)}
+    return {
+        'config': config, 'frame': frame, 'features': features,
+        'metadata': data_metadata(frame, config),
+        'forecast_summary': pd.DataFrame([summary for summary, _ in evaluated]),
+        'forecast_details': pd.concat([details for _, details in evaluated], ignore_index=True),
+        'signal_summary': pd.DataFrame([signal_summary]), 'signal_details': signal_details,
+    }
+
+
+def export_run(result, kind):
+    """Export the exact data, parameters, results and environment for auditing."""
+    run_id = pd.Timestamp.now(tz='UTC').strftime('%Y%m%dT%H%M%SZ') + '_' + uuid.uuid4().hex[:8]
+    directory = Path(OUTPUT_DIR) / f'{kind}_{run_id}'
+    Path(OUTPUT_DIR).mkdir(parents=True, exist_ok=True)
+    directory.mkdir(parents=True, exist_ok=False)
+    snapshot = result['frame'].to_csv(index_label='candle_open_utc', float_format='%.17g')
+    (directory / 'candles.csv').write_text(snapshot, encoding='utf-8')
+    if result['features'] is not None:
+        result['features'].to_csv(directory / 'features.csv', index=False, float_format='%.17g')
+    packages = {}
+    for name in ('numpy', 'pandas', 'scipy', 'statsmodels', 'arch', 'yfinance', 'gradio', 'torch', 'uni2ts', 'timesfm'):
+        try:
+            packages[name] = importlib.metadata.version(name)
+        except importlib.metadata.PackageNotFoundError:
+            packages[name] = None
+    manifest = {
+        'kind': kind, 'config': asdict(result['config']), 'data': result['metadata'],
+        'candles_sha256': hashlib.sha256(snapshot.encode('utf-8')).hexdigest(),
+        'random_seed': RANDOM_SEED, 'python': platform.python_version(), 'packages': packages,
+        'order_policy': 'ARIMA-family reselects order at each origin from training-window data only',
+        'feature_future_policy': 'Last observed covariates carried forward, never future actual indicators',
+        'evaluation': 'Walk-forward direction/price error plus causal candle-path SL/TP trade simulation; costs are explicit assumptions',
+        'trade_costs': result['config'].costs.as_dict(),
+    }
+    (directory / 'manifest.json').write_text(json.dumps(manifest, indent=2, default=str, allow_nan=False), encoding='utf-8')
+    if kind == 'live':
+        pd.DataFrame(result['forecasts']).to_csv(directory / 'forecasts.csv', index=False)
+        (directory / 'maythos.json').write_text(json.dumps(result['signal'], indent=2, default=str), encoding='utf-8')
+    else:
+        for key in ('forecast_summary', 'forecast_details', 'signal_summary', 'signal_details'):
+            result[key].to_csv(directory / f'{key}.csv', index=False)
+    path = directory.with_suffix('.zip')
+    with zipfile.ZipFile(path, 'w', zipfile.ZIP_DEFLATED) as archive:
+        for file in sorted(directory.iterdir()):
+            archive.write(file, file.name)
+    return str(path)

requirements-core.txt

@@ -0,0 +1,9 @@
+numpy==1.26.4
+pandas==2.1.4
+scipy==1.11.4
+statsmodels==0.14.6
+arch==8.0.0
+yfinance==1.7.0
+gradio==5.50.0
+plotly==6.9.0
+requests==2.34.2

requirements-dev.txt

@@ -0,0 +1,2 @@
+-r requirements.txt
+pytest==9.1.1

requirements-foundation.txt

@@ -0,0 +1,14 @@
+# Optional real Moirai + TimesFM inference (CPU). Install after requirements.txt.
+--extra-index-url https://download.pytorch.org/whl/cpu
+torch==2.4.1+cpu
+uni2ts==2.0.0
+timesfm[torch]==3.0.2
+gluonts==0.14.4
+lightning==2.4.0
+pytorch-lightning==2.4.0
+torchmetrics==1.4.3
+huggingface_hub==0.36.0
+pyarrow==16.1.0
+jax[cpu]==0.4.38
+scikit-learn==1.5.2
+safetensors>=0.5.3

requirements.txt

@@ -0,0 +1,4 @@
+# Hugging Face Spaces/core runtime. Foundation-model dependencies are
+# deliberately opt-in via requirements-foundation.txt; a missing optional
+# model is reported as failed instead of being replaced by fake output.
+-r requirements-core.txt

risk_management.py

@@ -0,0 +1,374 @@
+"""Causal trade-level construction and candle-path execution accounting.
+
+The functions in this module are deliberately shared by Live Signal and
+Backtesting.  A level is only built from candles available at the signal
+origin.  No future candle is used to choose ATR, structure, precision, SL or
+TP.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, asdict
+from decimal import Decimal
+import math
+
+import numpy as np
+import pandas as pd
+
+
+ATR_PERIOD = 14
+SWING_LOOKBACK = 20
+ATR_STOP_MULTIPLIER = 1.25
+ATR_STRUCTURE_BUFFER = 0.25
+MIN_RISK_REWARD = 1.0
+
+
+@dataclass(frozen=True)
+class CostModel:
+    """Explicit round-trip execution assumptions, in basis points.
+
+    OHLC feeds do not contain bid/ask or account fee schedules.  These values
+    are therefore exposed as assumptions and written into every manifest,
+    rather than being presented as provider-measured spreads.
+    """
+
+    spread_bps: float = 1.0
+    fee_bps: float = 1.0
+    slippage_bps: float = 1.0
+
+    def __post_init__(self):
+        for name in ('spread_bps', 'fee_bps', 'slippage_bps'):
+            value = getattr(self, name)
+            try:
+                value = float(value)
+            except (TypeError, ValueError):
+                value = float('nan')
+            if isinstance(getattr(self, name), (bool, np.bool_)) or not np.isfinite(value) or value < 0 or value > 1000:
+                raise ValueError(f'{name} must be a finite number between 0 and 1000 bps.')
+            object.__setattr__(self, name, value)
+
+    def as_dict(self):
+        return asdict(self)
+
+
+@dataclass(frozen=True)
+class TradeLevels:
+    direction: str
+    entry: float
+    stop_loss: float
+    take_profit: float
+    risk_per_unit: float
+    reward_per_unit: float
+    risk_reward: float
+    atr: float
+    swing_support: float
+    swing_resistance: float
+    tick_size: float
+    precision_digits: int
+    level_basis: str
+
+    def as_dict(self):
+        return asdict(self)
+
+
+def _finite_positive(value) -> bool:
+    return np.isfinite(value) and float(value) > 0
+
+
+def infer_tick_size(frame: pd.DataFrame, market: str, symbol: str) -> tuple[float, int]:
+    """Infer the quote increment from provider metadata or real observations.
+
+    Binance adapters can provide an exact exchange filter through
+    ``frame.attrs['price_tick_size']``.  Forex uses the conventional quoted
+    pip precision (five decimals, three for JPY quotes).  For other feeds the
+    most precise decimal scale present in recent real closes is retained.
+    """
+    if market not in ('Forex', 'Crypto'):
+        raise ValueError('market must be Forex or Crypto.')
+    if frame is None or 'Close' not in frame or frame.empty:
+        raise ValueError('Cannot infer tick size without close observations.')
+    exact = frame.attrs.get('price_tick_size')
+    if exact is not None and _finite_positive(float(exact)):
+        tick = float(exact)
+        digits = max(0, min(12, int(math.ceil(-math.log10(tick)))))
+        return tick, digits
+    if market == 'Forex':
+        clean = str(symbol).upper().replace('=X', '').replace('/', '')
+        digits = 3 if clean.endswith('JPY') else 5
+        return 10.0 ** (-digits), digits
+    decimals = 0
+    for value in pd.to_numeric(frame['Close'].tail(100), errors='coerce').dropna():
+        exponent = Decimal(str(float(value))).as_tuple().exponent
+        decimals = max(decimals, min(8, max(0, -exponent)))
+    decimals = min(decimals, 8)
+    tick = 10.0 ** (-decimals)
+    return tick, decimals
+
+
+def _round_to_tick(value: float, tick: float, direction: str) -> float:
+    if direction not in ('down', 'up') or not _finite_positive(tick) or not np.isfinite(value):
+        raise ValueError('Tick rounding needs a finite value, positive tick, and up/down direction.')
+    scaled = float(value) / tick
+    if direction == 'down':
+        rounded = math.floor(scaled + 1e-12) * tick
+    else:
+        rounded = math.ceil(scaled - 1e-12) * tick
+    return float(round(rounded, max(0, int(round(-math.log10(tick)))) + 2))
+
+
+def _true_range(frame: pd.DataFrame) -> pd.Series:
+    previous_close = frame['Close'].shift(1)
+    return pd.concat(
+        [frame['High'] - frame['Low'],
+         (frame['High'] - previous_close).abs(),
+         (frame['Low'] - previous_close).abs()], axis=1,
+    ).max(axis=1)
+
+
+def average_true_range(frame: pd.DataFrame, period: int = ATR_PERIOD) -> float:
+    if len(frame) < period:
+        return float('nan')
+    tr = _true_range(frame)
+    # Wilder's ATR seeds with the arithmetic mean of the first `period` true
+    # ranges, then applies the recursive moving-average update. pandas' EWM
+    # ``adjust=False`` starts from the first observation, which is a different
+    # estimator and materially changes short causal trade levels.
+    values = tr.to_numpy(dtype=float)
+    if not np.isfinite(values[:period]).all():
+        return float('nan')
+    atr = float(values[:period].mean())
+    for value in values[period:]:
+        if not np.isfinite(value):
+            return float('nan')
+        atr = (atr * (period - 1) + float(value)) / period
+    value = atr
+    return value if _finite_positive(value) else float('nan')
+
+
+def build_trade_levels(
+    frame: pd.DataFrame,
+    direction: str,
+    market: str,
+    symbol: str,
+    forecast_target: float | None = None,
+    atr_period: int = ATR_PERIOD,
+    swing_lookback: int = SWING_LOOKBACK,
+    forecast_only: bool = False,
+    minimum_edge_atr: float = 0.0,
+    minimum_risk_reward: float = MIN_RISK_REWARD,
+    use_structure_stop: bool = True,
+    minimum_edge_bps: float = 0.0,
+) -> TradeLevels | None:
+    """Build causal, volatility-aware levels for one closed-candle signal.
+
+    The stop is the farther of an ATR noise stop and a structure stop.  The
+    target must be a real opposite-side structure level or a compatible real
+    model forecast, and signals without at least 1R structural reward are
+    rejected instead of receiving an arbitrary target.
+    """
+    direction = str(direction or '').upper()
+    try:
+        forecast_value = None if forecast_target is None else float(forecast_target)
+    except (TypeError, ValueError):
+        forecast_value = None
+    if (not np.isfinite(minimum_edge_atr) or minimum_edge_atr < 0 or
+            not np.isfinite(minimum_risk_reward) or minimum_risk_reward <= 0 or
+            not np.isfinite(minimum_edge_bps) or minimum_edge_bps < 0):
+        raise ValueError('minimum_edge_atr/minimum_edge_bps must be non-negative and minimum_risk_reward must be positive.')
+    if (isinstance(atr_period, bool) or not isinstance(atr_period, int) or atr_period < 1 or
+            isinstance(swing_lookback, bool) or not isinstance(swing_lookback, int) or swing_lookback < 1):
+        raise ValueError('atr_period and swing_lookback must be positive integers.')
+    if direction not in ('BUY', 'SELL') or frame is None or len(frame) < max(atr_period, swing_lookback) + 1:
+        return None
+    required = {'High', 'Low', 'Close'}
+    if not required.issubset(frame.columns):
+        return None
+    numeric = frame[list(required)].apply(pd.to_numeric, errors='coerce')
+    if not np.isfinite(numeric.to_numpy(dtype=float)).all() or (numeric <= 0).any().any():
+        return None
+    entry = float(frame['Close'].iloc[-1])
+    atr = average_true_range(frame, atr_period)
+    if not _finite_positive(atr) or not _finite_positive(entry):
+        return None
+    tick, precision_digits = infer_tick_size(frame, market, symbol)
+    forecast_used = False
+    # Use only candles before the signal candle for structural levels. This
+    # makes the swing level stable at the exact point the order is created.
+    structure = frame.iloc[:-1].tail(swing_lookback)
+    support = float(structure['Low'].min())
+    resistance = float(structure['High'].max())
+    buffer = max(ATR_STRUCTURE_BUFFER * atr, 2.0 * tick)
+    if direction == 'BUY':
+        atr_stop = entry - ATR_STOP_MULTIPLIER * atr
+        stop_raw = min(atr_stop, support - buffer) if use_structure_stop else atr_stop
+        forecast_valid = (forecast_value is not None and
+                          np.isfinite(forecast_value) and forecast_value > 0 and forecast_value > entry)
+        if forecast_only and not forecast_valid:
+            return None
+        # A model target is an explicit horizon hypothesis. Do not silently
+        # extend it to a farther swing level, which turns a one-bar forecast
+        # into a timeout-prone structural trade.
+        target_raw = forecast_value if forecast_valid else resistance + buffer
+        forecast_used = forecast_valid
+        stop = _round_to_tick(stop_raw, tick, 'down')
+        target = _round_to_tick(target_raw, tick, 'up')
+        risk = entry - stop
+        reward = target - entry
+    else:
+        atr_stop = entry + ATR_STOP_MULTIPLIER * atr
+        stop_raw = max(atr_stop, resistance + buffer) if use_structure_stop else atr_stop
+        forecast_valid = (forecast_value is not None and
+                          np.isfinite(forecast_value) and forecast_value > 0 and forecast_value < entry)
+        if forecast_only and not forecast_valid:
+            return None
+        target_raw = forecast_value if forecast_valid else support - buffer
+        forecast_used = forecast_valid
+        stop = _round_to_tick(stop_raw, tick, 'up')
+        target = _round_to_tick(target_raw, tick, 'down')
+        risk = stop - entry
+        reward = entry - target
+    if not (_finite_positive(stop) and _finite_positive(target) and
+            _finite_positive(risk) and _finite_positive(reward)):
+        return None
+    rr = reward / risk
+    edge_atr = reward / atr
+    edge_bps = reward / entry * 10000.0
+    if (not np.isfinite(rr) or rr < minimum_risk_reward or
+            not np.isfinite(edge_atr) or edge_atr < minimum_edge_atr or
+            not np.isfinite(edge_bps) or edge_bps < minimum_edge_bps):
+        return None
+    return TradeLevels(
+        direction=direction, entry=entry, stop_loss=stop, take_profit=target,
+        risk_per_unit=float(risk), reward_per_unit=float(reward),
+        risk_reward=float(rr), atr=float(atr), swing_support=support,
+        swing_resistance=resistance, tick_size=tick,
+        precision_digits=precision_digits,
+        level_basis=(
+            'Wilder ATR + prior closed-candle structure + forecast-compatible target'
+            if forecast_used else
+            'Wilder ATR + prior closed-candle swing structure'
+        ),
+    )
+
+
+def _execution_price(raw_price: float, direction: str, is_entry: bool, costs: CostModel) -> float:
+    half_spread = costs.spread_bps / 2.0
+    adjustment = (half_spread + costs.slippage_bps) / 10000.0
+    sign = 1.0 if direction == 'BUY' else -1.0
+    if is_entry:
+        return raw_price * (1.0 + sign * adjustment)
+    return raw_price * (1.0 - sign * adjustment)
+
+
+def _net_r(entry: float, exit_price: float, direction: str, risk: float, costs: CostModel) -> float:
+    entry_fill = _execution_price(entry, direction, True, costs)
+    exit_fill = _execution_price(exit_price, direction, False, costs)
+    sign = 1.0 if direction == 'BUY' else -1.0
+    gross = sign * (exit_fill - entry_fill)
+    # CostModel.fee_bps is explicitly a round-trip assumption. Apply it once
+    # to the average filled notional rather than charging the same round-trip
+    # rate independently to both entry and exit.
+    fees = (costs.fee_bps / 10000.0) * ((entry_fill + exit_fill) / 2.0)
+    return float((gross - fees) / risk)
+
+
+def simulate_trade(levels: TradeLevels, path: pd.DataFrame, costs: CostModel) -> dict:
+    """Resolve SL/TP from actual future candle high/low data.
+
+    If both levels are touched within one OHLC candle, SL is deliberately
+    chosen first because intrabar ordering is unavailable and that is the
+    conservative policy requested by the dashboard.
+    """
+    if not isinstance(levels, TradeLevels):
+        raise TypeError('simulate_trade needs TradeLevels from build_trade_levels.')
+    if not isinstance(costs, CostModel):
+        raise TypeError('simulate_trade needs a CostModel.')
+    if path is None or path.empty:
+        return {'outcome': 'NO_PATH', 'exit_price': None, 'net_r': None, 'bars_held': 0}
+    required = {'High', 'Low', 'Close'}
+    if not required.issubset(path.columns):
+        raise ValueError(f'Trade path is missing columns: {sorted(required - set(path.columns))}.')
+    path_required = set(required)
+    if 'Open' in path.columns:
+        path_required.add('Open')
+    numeric = path[sorted(path_required)].apply(pd.to_numeric, errors='coerce')
+    if not np.isfinite(numeric.to_numpy(dtype=float)).all():
+        raise ValueError('Trade path contains non-finite OHLC values.')
+    if (numeric['High'] < numeric[['Low', 'Close']].max(axis=1)).any() or (numeric['Low'] > numeric[['High', 'Close']].min(axis=1)).any():
+        raise ValueError('Trade path contains invalid OHLC geometry.')
+    if 'Open' in numeric and ((numeric['Open'] > numeric['High']) | (numeric['Open'] < numeric['Low'])).any():
+        raise ValueError('Trade path contains an open outside its candle range.')
+    direction = levels.direction
+    for offset, (timestamp, candle) in enumerate(path.iterrows(), start=1):
+        high = float(candle['High'])
+        low = float(candle['Low'])
+        opening = float(candle['Open']) if 'Open' in candle else None
+        if direction == 'BUY':
+            if opening is not None and opening <= levels.stop_loss:
+                exit_price, outcome = opening, 'SL'
+            elif opening is not None and opening >= levels.take_profit:
+                exit_price, outcome = opening, 'TP'
+            elif low <= levels.stop_loss:
+                exit_price, outcome = levels.stop_loss, 'SL'
+            elif high >= levels.take_profit:
+                exit_price, outcome = levels.take_profit, 'TP'
+            else:
+                continue
+        else:
+            if opening is not None and opening >= levels.stop_loss:
+                exit_price, outcome = opening, 'SL'
+            elif opening is not None and opening <= levels.take_profit:
+                exit_price, outcome = opening, 'TP'
+            elif high >= levels.stop_loss:
+                exit_price, outcome = levels.stop_loss, 'SL'
+            elif low <= levels.take_profit:
+                exit_price, outcome = levels.take_profit, 'TP'
+            else:
+                continue
+        return {
+            'outcome': outcome, 'exit_price': float(exit_price),
+            'exit_timestamp': timestamp.isoformat() if hasattr(timestamp, 'isoformat') else str(timestamp),
+            'net_r': _net_r(levels.entry, exit_price, direction, levels.risk_per_unit, costs),
+            'bars_held': offset,
+        }
+    timestamp = path.index[-1]
+    exit_price = float(path['Close'].iloc[-1])
+    return {
+        'outcome': 'TIMEOUT', 'exit_price': exit_price,
+        'exit_timestamp': timestamp.isoformat() if hasattr(timestamp, 'isoformat') else str(timestamp),
+        'net_r': _net_r(levels.entry, exit_price, direction, levels.risk_per_unit, costs),
+        'bars_held': len(path),
+    }
+
+
+def summarize_trades(details: pd.DataFrame) -> dict:
+    if details is None or details.empty or 'trade_taken' not in details:
+        return {
+            'trades': 0, 'wins': 0, 'losses': 0, 'timeouts': 0,
+            'win_rate_pct': None, 'profit_factor': None,
+            'expectancy_r': None, 'max_drawdown_r': None, 'average_r': None,
+        }
+    trades = details[details['trade_taken'] & details['net_r'].notna()].copy()
+    if trades.empty:
+        return {
+            'trades': 0, 'wins': 0, 'losses': 0, 'timeouts': 0,
+            'win_rate_pct': None, 'profit_factor': None,
+            'expectancy_r': None, 'max_drawdown_r': None, 'average_r': None,
+        }
+    returns = trades['net_r'].astype(float).to_numpy()
+    wins = int((trades['outcome'] == 'TP').sum())
+    losses = int((trades['outcome'] == 'SL').sum())
+    timeouts = int((trades['outcome'] == 'TIMEOUT').sum())
+    gross_profit = float(returns[returns > 0].sum())
+    gross_loss = float(-returns[returns < 0].sum())
+    profit_factor = gross_profit / gross_loss if gross_loss > 0 else (float('inf') if gross_profit > 0 else None)
+    equity = np.cumsum(returns)
+    peaks = np.maximum.accumulate(np.r_[0.0, equity])[1:]
+    drawdown = peaks - equity
+    return {
+        'trades': int(len(trades)), 'wins': wins, 'losses': losses, 'timeouts': timeouts,
+        'win_rate_pct': round(100.0 * wins / len(trades), 2),
+        'profit_factor': round(profit_factor, 6) if profit_factor is not None and np.isfinite(profit_factor) else profit_factor,
+        'expectancy_r': round(float(returns.mean()), 6),
+        'max_drawdown_r': round(float(drawdown.max()) if len(drawdown) else 0.0, 6),
+        'average_r': round(float(returns.mean()), 6),
+    }

setup_windows.bat

@@ -0,0 +1,26 @@
+@echo off
+setlocal
+cd /d "%~dp0"
+py -3.11 --version >nul 2>&1
+if errorlevel 1 (
+  echo Install Python 3.11 from python.org, including the Python launcher, then rerun this file.
+  pause
+  exit /b 1
+)
+if not exist ".venv\Scripts\python.exe" py -3.11 -m venv .venv
+if errorlevel 1 exit /b 1
+".venv\Scripts\python.exe" -m pip install --upgrade pip
+if errorlevel 1 exit /b 1
+".venv\Scripts\python.exe" -m pip install -r requirements-core.txt
+if errorlevel 1 (
+  pause
+  exit /b 1
+)
+".venv\Scripts\python.exe" -m pip check
+if errorlevel 1 (
+  echo Dependency consistency check failed.
+  pause
+  exit /b 1
+)
+echo Core setup complete. Use start_windows.bat. For Moirai and TimesFM, also run install_foundation_windows.bat.
+pause

start_windows.bat

@@ -0,0 +1,10 @@
+@echo off
+setlocal
+cd /d "%~dp0"
+if not exist ".venv\Scripts\python.exe" (
+  echo Run setup_windows.bat first.
+  pause
+  exit /b 1
+)
+".venv\Scripts\python.exe" app.py
+pause

tests/__init__.py

@@ -0,0 +1 @@
+pass

tests/test_audit_regressions.py

@@ -0,0 +1,170 @@
+"""Analytical contract fixtures, not production/provider verification.
+
+Deliberately controlled prices/forecasts make execution arithmetic independently
+checkable. No provider or real-model runtime is replaced by these fixtures.
+"""
+import numpy as np
+import pandas as pd
+import pytest
+from unittest.mock import patch
+
+from backtest.backtester import backtest_model
+from maythos import AdaptiveScoringEngine, BollingerEngine, Candle, MAYTHOS, RSIEngine, SupportResistanceEngine
+from data.providers import _validate_frame
+from data.validation import DataFetchError
+from models.base_model import causal_exogenous
+from pipeline import RunConfig, backtest_result
+from risk_management import (
+    CostModel, TradeLevels, average_true_range, simulate_trade, summarize_trades,
+)
+from tests.test_risk_management import _history
+
+
+@pytest.mark.parametrize('sign', [1, -1])
+def test_forecast_direction_reaches_trade_execution(sign):
+    class AnalyticalForecast:
+        def predict(self, history, horizon=1, features=None):
+            return [float(history.iloc[-1]) + sign * 10] * horizon
+
+    rows = backtest_model(_history(80), AnalyticalForecast(), window=40,
+                          horizon=2, step=10, timeframe='5m')
+    assert rows['trade_taken'].all()
+    assert rows['net_r'].notna().all()
+
+
+def levels(direction='BUY'):
+    return TradeLevels(direction, 100, 90 if direction == 'BUY' else 110,
+                       120 if direction == 'BUY' else 80, 10, 20, 2,
+                       2, 90, 120, 0.01, 2, 'analytical fixture')
+
+
+def test_round_trip_fee_is_not_charged_twice():
+    path = pd.DataFrame({'Open': [100], 'High': [101], 'Low': [99], 'Close': [100]},
+                        index=pd.date_range('2026-01-01', periods=1, tz='UTC'))
+    result = simulate_trade(levels(), path, CostModel(0, 10, 0))
+    # A 10 bps ROUND TRIP fee on a 100-unit notional is 0.1, or 0.01R.
+    assert result['net_r'] == pytest.approx(-0.01)
+
+
+def test_atr_uses_arithmetic_wilder_seed():
+    frame = pd.DataFrame({'High': [101, 103, 105, 109],
+                          'Low': [99, 97, 95, 91], 'Close': [100] * 4})
+    # TR = 2,6,10,18. Seed ATR(3)=6, then (2*6+18)/3=10.
+    assert average_true_range(frame, 3) == pytest.approx(10)
+
+
+@pytest.mark.parametrize('direction,opening', [('BUY', 85), ('SELL', 115)])
+def test_stop_gap_exits_at_available_open(direction, opening):
+    path = pd.DataFrame({'Open': [opening], 'High': [opening+1],
+                          'Low': [opening-1], 'Close': [opening]},
+                        index=pd.date_range('2026-01-01', periods=1, tz='UTC'))
+    result = simulate_trade(levels(direction), path, CostModel(0, 0, 0))
+    assert result['outcome'] == 'SL'
+    assert result['exit_price'] == opening
+    assert result['net_r'] == pytest.approx(-1.5)
+
+
+def test_win_loss_counts_reconcile_with_net_profit():
+    frame = pd.DataFrame({'trade_taken': [True]*3, 'net_r': [-0.1, 0.3, -1.0],
+                          'outcome': ['TP', 'TIMEOUT', 'SL']})
+    summary = summarize_trades(frame)
+    assert summary['wins'] == 1
+    assert summary['losses'] == 1
+    assert summary['win_rate_pct'] == pytest.approx(33.33)
+
+
+def test_invalid_intermediate_forecast_is_not_scored():
+    class InvalidPath:
+        def predict(self, history, horizon=1, features=None):
+            return [np.nan, float(history.iloc[-1]) + 10]
+
+    rows = backtest_model(_history(80), InvalidPath(), window=40, horizon=2, step=10)
+    assert rows['correct'].isna().all()
+    assert rows['error'].notna().all()
+
+
+def test_flat_maythos_rsi_is_neutral():
+    engine = RSIEngine()
+    for i in range(30):
+        result = engine.update(Candle(1_700_000_000+i*60, 100, 100, 100, 100), {})
+    assert result['rsi'] == 50
+    assert not result['rsi_ob_flag']
+
+
+def test_bollinger_upper_touch_is_bearish_confirmation():
+    engine = BollingerEngine()
+    for i in range(20):
+        close = 100 if i < 19 else 110
+        result = engine.update(Candle(1_700_000_000+i*60, close, close+1, close-1, close))
+    assert result['upper_touch']
+    assert result['bb_confirmation'] < 0
+
+
+def test_resistance_confirmation_preserves_bearish_sign():
+    engine = SupportResistanceEngine()
+    engine._levels = {110.0: {'strength': 0.8, 'type': 'resistance', 'age': 0}}
+    result = engine.update(Candle(1_700_000_000, 109.5, 110, 109, 109.5), {})
+    assert result['at_resistance']
+    assert result['sr_confirmation'] < 0
+
+
+def test_causal_exogenous_lags_target_features_and_persists_origin_row():
+    features = pd.DataFrame({'indicator': [10.0, 20.0, 30.0, 40.0]})
+    train, future = causal_exogenous(features, horizon=2, model_name='fixture')
+    assert len(train) == 3
+    assert len(future) == 2
+    assert train.iloc[0, 0] != train.iloc[-1, 0]
+    assert future.iloc[0, 0] == future.iloc[1, 0]
+    # The first future regressor is derived from the last observed row, not a
+    # realized future feature.
+    assert future.iloc[0, 0] > train.iloc[-1, 0]
+
+
+def test_provider_rejects_invalid_ohlcv_without_repairing_it():
+    frame = pd.DataFrame(
+        {'Open': [100.0], 'High': [99.0], 'Low': [98.0], 'Close': [98.5], 'Volume': [1.0]},
+        index=pd.date_range('2026-01-01', periods=1, tz='UTC'),
+    )
+    with pytest.raises(DataFetchError):
+        _validate_frame(frame)
+
+
+def test_open_and_duplicate_candles_do_not_advance_maythos_warmup():
+    engine = MAYTHOS()
+    first = Candle(1_700_000_000, 100, 101, 99, 100)
+    assert engine.validate(engine.tick(first)) == []
+    count = engine.tick_count
+    forming = Candle(1_700_000_060, 100, 101, 99, 100, is_closed=False)
+    assert engine.tick(forming)['blocked_flag']
+    assert engine.tick_count == count
+    assert engine.tick(first)['blocked_flag']
+    assert engine.tick_count == count
+
+
+def test_candle_with_non_positive_quote_is_invalid_and_blocked():
+    candle = Candle(1_700_000_000, 0, 1, 0, 1)
+    assert not candle.valid
+    output = MAYTHOS().tick(candle)
+    assert output['blocked_flag']
+
+
+def test_neutral_maythos_direction_is_blocked_not_treated_as_buy():
+    score = AdaptiveScoringEngine().score(
+        {}, {}, {}, {}, {}, {}, {}, {}, tick_count=100,
+    )
+    assert score['raw_direction'] == 0.0
+    assert score['execution_suitability'] == 'blocked'
+
+
+def test_pipeline_reports_model_construction_failure_without_corrupting_details():
+    index = pd.date_range('2026-01-01', periods=45, freq='5min', tz='UTC')
+    close = np.linspace(100.0, 101.0, len(index))
+    frame = pd.DataFrame(
+        {'Open': close, 'High': close + .1, 'Low': close - .1, 'Close': close, 'Volume': 100.0},
+        index=index,
+    )
+    config = RunConfig(models=('ARIMA',), window=32, horizon=1, max_evaluations=1, lookback_days=1)
+    with patch('pipeline.new_configured_model', side_effect=RuntimeError('fixture construction failure')):
+        result = backtest_result(config, frame=frame)
+    assert result['forecast_summary'].iloc[0]['status'] == 'FAILED'
+    assert list(result['forecast_details'].columns).count('Model') == 1

tests/test_backtester_alignment.py

@@ -0,0 +1,175 @@
+import unittest
+import json
+import numpy as np
+import pandas as pd
+from backtest.backtester import backtest_model, run_all_models_backtest, run_comparison_backtest
+from features.feature_pipeline import FEATURE_COLUMNS, compute_feature_frame
+from models.base_model import BaseForecastModel
+
+def _synthetic_ohlcv(n: int, seed: int=7) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    steps = rng.normal(loc=0.0, scale=0.4, size=n)
+    close = 100 + np.cumsum(steps)
+    close = np.abs(close) + 5.0
+    open_ = close + rng.normal(0, 0.1, n)
+    high = np.maximum(open_, close) + np.abs(rng.normal(0, 0.2, n))
+    low = np.minimum(open_, close) - np.abs(rng.normal(0, 0.2, n))
+    volume = rng.integers(100, 10000, n).astype(float)
+    idx = pd.date_range('2024-01-01', periods=n, freq='min')
+    return pd.DataFrame({'Open': open_, 'High': high, 'Low': low, 'Close': close, 'Volume': volume}, index=idx)
+
+class _AlignmentCheckingModel(BaseForecastModel):
+    name = 'AlignmentChecker'
+    RAW_COLUMNS = ['Open', 'High', 'Low', 'Volume']
+
+    def __init__(self, df: pd.DataFrame):
+        self.full_close = df['Close'].reset_index(drop=True)
+        self.raw = df[self.RAW_COLUMNS].reset_index(drop=True)
+
+    def predict(self, history, horizon=1, features=None):
+        history = pd.Series(history).reset_index(drop=True)
+        if features is not None:
+            assert len(features) == len(history), f'length mismatch: history={len(history)} features={len(features)}'
+            assert list(features.columns) == FEATURE_COLUMNS
+        n = len(self.full_close)
+        w = len(history)
+        match_start = None
+        for start in range(0, n - w + 1):
+            if np.allclose(self.full_close.iloc[start:start + w].to_numpy(), history.to_numpy()):
+                match_start = start
+                break
+        assert match_start is not None, 'history window not found in full series -- corrupted slice'
+        if features is not None:
+            expected = self.raw.iloc[match_start:match_start + w].to_numpy()
+            got = features[self.RAW_COLUMNS].to_numpy()
+            assert np.allclose(got, expected), f'features window is shifted relative to history at position {match_start}'
+        return [float(history.iloc[-1])] * horizon
+
+class TestBacktesterAlignment(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _synthetic_ohlcv(200, seed=11)
+        self.features_df = compute_feature_frame(self.df)
+
+    def test_no_failed_rows_with_features(self):
+        model = _AlignmentCheckingModel(self.df)
+        results = backtest_model(self.df, model, window=40, horizon=1, step=1, features_df=self.features_df)
+        failed = results['error'].notna().sum()
+        self.assertEqual(failed, 0, f"{failed} rows failed alignment checks:\n{results[results['error'].notna()]}")
+        self.assertGreater(len(results), 0)
+
+    def test_features_window_matches_close_window_exactly(self):
+        close = self.df['Close'].reset_index(drop=True)
+        window, horizon, step = (35, 2, 3)
+        last_i = len(close) - horizon
+        for i in range(window, last_i + 1, step):
+            expected_close_window = close.iloc[i - window:i].reset_index(drop=True)
+            expected_feat_window = self.features_df.iloc[i - window:i].reset_index(drop=True)
+            source_volume = self.df['Volume'].reset_index(drop=True).iloc[i - window:i].reset_index(drop=True)
+            pd.testing.assert_series_equal(expected_feat_window['Volume'], source_volume, check_names=False)
+            self.assertEqual(len(expected_close_window), len(expected_feat_window))
+
+    def test_baseline_mode_never_touches_features_module(self):
+
+        class RecordsFeatureArg(BaseForecastModel):
+            name = 'Recorder'
+            saw_features = []
+
+            def predict(self, history, horizon=1, features=None):
+                RecordsFeatureArg.saw_features.append(features)
+                return [float(pd.Series(history).iloc[-1])] * horizon
+        RecordsFeatureArg.saw_features.clear()
+        _ = backtest_model(self.df, RecordsFeatureArg(), window=40, horizon=1, step=5)
+        self.assertTrue(len(RecordsFeatureArg.saw_features) > 0)
+        self.assertTrue(all((f is None for f in RecordsFeatureArg.saw_features)))
+
+class TestRunAllModelsBacktestFeaturesDf(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _synthetic_ohlcv(150, seed=13)
+        self.features_df = compute_feature_frame(self.df)
+
+    def test_features_df_none_passes_none_through(self):
+
+        class Recorder(BaseForecastModel):
+            name = 'Recorder'
+
+            def __init__(self):
+                self.seen = []
+
+            def predict(self, history, horizon=1, features=None):
+                self.seen.append(features is None)
+                return [float(pd.Series(history).iloc[-1])] * horizon
+        m = Recorder()
+        run_all_models_backtest(self.df, {'Recorder': m}, window=40, horizon=1, step=10, features_df=None)
+        self.assertTrue(all(m.seen))
+
+    def test_features_df_given_passes_dataframe_through(self):
+
+        class Recorder(BaseForecastModel):
+            name = 'Recorder'
+
+            def __init__(self):
+                self.seen = []
+
+            def predict(self, history, horizon=1, features=None):
+                self.seen.append(features is not None and len(features) == len(history))
+                return [float(pd.Series(history).iloc[-1])] * horizon
+        m = Recorder()
+        run_all_models_backtest(self.df, {'Recorder': m}, window=40, horizon=1, step=10, features_df=self.features_df)
+        self.assertTrue(len(m.seen) > 0)
+        self.assertTrue(all(m.seen))
+
+class TestRunComparisonBacktest(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _synthetic_ohlcv(150, seed=17)
+        self.features_df = compute_feature_frame(self.df)
+
+    def test_both_modes_run_and_are_labeled_separately(self):
+
+        class Recorder(BaseForecastModel):
+            name = 'Recorder'
+
+            def __init__(self):
+                self.seen_features = []
+
+            def predict(self, history, horizon=1, features=None):
+                self.seen_features.append(features is not None)
+                return [float(pd.Series(history).iloc[-1])] * horizon
+        baseline_model, featured_model = (Recorder(), Recorder())
+        results, summaries = run_comparison_backtest(self.df, {'Recorder': (baseline_model, featured_model)}, self.features_df, window=40, horizon=1, step=10)
+        self.assertTrue(all((f is False for f in baseline_model.seen_features)))
+        self.assertTrue(all((f is True for f in featured_model.seen_features)))
+        modes_present = set(results['Recorder']['mode'].unique())
+        self.assertEqual(modes_present, {'Close-only', '+14 Features'})
+        self.assertEqual(len(summaries['Recorder']), 2)
+        labels = {s['mode'] for s in summaries['Recorder']}
+        self.assertEqual(labels, {'Close-only', '+14 Features'})
+
+class TestUiSerializableTimestamps(unittest.TestCase):
+
+    def test_backtest_results_json_serializable(self):
+        model = _ConstantModel()
+        results = backtest_model(self._ohlcv(), model, window=20, horizon=1, step=5)
+        self.assertEqual(results['datetime'].map(type).eq(str).all(), True)
+        json.dumps(results.to_dict(orient='records'))
+
+    def test_live_log_new_row_json_serializable(self):
+        row = {'predicted_at': pd.Timestamp('2026-01-01 00:00:00+00:00').isoformat(), 'mode': 'Close-only', 'target_time': pd.Timestamp('2026-01-01 00:01:00+00:00').isoformat(), 'current_close': 1.0, 'predicted_close': 1.0, 'predicted_direction': 'up', 'actual_close': np.nan, 'actual_direction': None, 'correct': None}
+        json.dumps(row)
+
+    @staticmethod
+    def _ohlcv(n: int=60, seed: int=3) -> pd.DataFrame:
+        rng = np.random.default_rng(seed)
+        close = 100 + np.cumsum(rng.normal(0, 0.5, n))
+        idx = pd.date_range('2026-01-01', periods=n, freq='15min', tz='UTC')
+        return pd.DataFrame({'Open': close, 'High': close + 1, 'Low': close - 1, 'Close': close, 'Volume': 1000.0}, index=idx)
+
+class _ConstantModel(BaseForecastModel):
+
+    def predict(self, history, horizon=1, features=None):
+        return np.array([float(history.iloc[-1])] * horizon)
+    name = 'ConstantModel'
+if __name__ == '__main__':
+    unittest.main()

tests/test_data_layer.py

@@ -0,0 +1,191 @@
+import math
+import unittest
+from datetime import timedelta
+from unittest import mock
+import numpy as np
+import pandas as pd
+from config import YF_NATIVE_INTERVALS
+from data.historical import OHLCV_COLUMNS, DataFetchError, get_historical, get_historical_with_features, resample_ohlc
+from features.feature_pipeline import FEATURE_COLUMNS
+
+def _window_candles(start, end, freq: str) -> pd.DataFrame:
+    idx = pd.date_range(start=pd.Timestamp(start).ceil(freq), end=end, freq=freq, inclusive='left')
+    if len(idx) == 0:
+        return pd.DataFrame()
+    n = len(idx)
+    base = 100.0 + np.arange(n, dtype=float) * 0.1
+    return pd.DataFrame({'Open': base, 'High': base + 0.5, 'Low': base - 0.5, 'Close': base + 0.2, 'Volume': np.arange(n, dtype=float) + 1.0, 'Dividends': np.zeros(n)}, index=idx)
+
+class _RecordingFetcher:
+
+    def __init__(self, freq: str='1h', empty_calls=()):
+        self.freq = freq
+        self.empty_calls = set(empty_calls)
+        self.calls = []
+        self.frames = []
+
+    def __call__(self, symbol, interval, start, end):
+        self.calls.append({'symbol': symbol, 'interval': interval, 'start': start, 'end': end})
+        if len(self.calls) in self.empty_calls:
+            return pd.DataFrame()
+        frame = _window_candles(start, end, self.freq)
+        self.frames.append(frame)
+        return frame
+
+    @property
+    def spans(self):
+        return [c['end'] - c['start'] for c in self.calls]
+
+    @property
+    def total_span(self):
+        return max((c['end'] for c in self.calls)) - min((c['start'] for c in self.calls))
+
+    def raw(self) -> pd.DataFrame:
+        return pd.concat(self.frames).sort_index()
+
+class TestLookbackIsHonored(unittest.TestCase):
+
+    def test_daily_lookback_requests_exactly_that_window(self):
+        fetcher = _RecordingFetcher(freq='1D')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df = get_historical('BTC-USD', '1d', lookback_days=30)
+        self.assertEqual(len(fetcher.calls), 1)
+        self.assertEqual(fetcher.calls[0]['interval'], '1d')
+        self.assertEqual(fetcher.spans[0], timedelta(days=30))
+        self.assertLessEqual(len(df), 31)
+        self.assertGreater(len(df), 0)
+
+    def test_returned_frame_is_ordered_and_ohlcv_only(self):
+        fetcher = _RecordingFetcher(freq='1D')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df = get_historical('BTC-USD', '1d', lookback_days=10)
+        self.assertEqual(list(df.columns), OHLCV_COLUMNS)
+        self.assertNotIn('Dividends', df.columns)
+        self.assertTrue(df.index.is_monotonic_increasing)
+        self.assertFalse(df.index.duplicated().any())
+
+    def test_no_lookback_falls_back_to_the_interval_limit(self):
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            get_historical('BTC-USD', '60m')
+        self.assertEqual(fetcher.total_span, timedelta(days=YF_NATIVE_INTERVALS['60m']['max_days']))
+
+class TestChunkedFetching(unittest.TestCase):
+
+    def setUp(self):
+        self.limits = YF_NATIVE_INTERVALS['1m']
+
+    def test_window_wider_than_the_request_limit_is_split(self):
+        requested = 30
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df = get_historical('BTC-USD', '1m', lookback_days=requested)
+        request_days = self.limits['max_request_days']
+        self.assertEqual(len(fetcher.calls), math.ceil(requested / request_days))
+        for span in fetcher.spans:
+            self.assertLessEqual(span, timedelta(days=request_days))
+        self.assertEqual(sum(fetcher.spans, timedelta()), timedelta(days=requested))
+        self.assertEqual(fetcher.total_span, timedelta(days=requested))
+        self.assertGreater(len(df), 0)
+
+    def test_chunks_are_contiguous_with_no_gap_or_overlap(self):
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            get_historical('BTC-USD', '1m', lookback_days=20)
+        for newer, older in zip(fetcher.calls, fetcher.calls[1:]):
+            self.assertEqual(newer['start'], older['end'])
+
+    def test_lookback_beyond_the_source_limit_is_clamped(self):
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            get_historical('BTC-USD', '1m', lookback_days=90)
+        max_days = self.limits['max_days']
+        self.assertEqual(fetcher.total_span, timedelta(days=max_days))
+        self.assertEqual(len(fetcher.calls), math.ceil(max_days / self.limits['max_request_days']))
+
+    def test_one_empty_chunk_does_not_abort_the_fetch(self):
+        fetcher = _RecordingFetcher(freq='1h', empty_calls=(2,))
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df = get_historical('BTC-USD', '1m', lookback_days=30)
+        self.assertEqual(len(fetcher.calls), 5)
+        self.assertEqual(len(df), len(fetcher.raw()))
+        self.assertGreater(len(df), 0)
+
+    def test_all_chunks_empty_reports_the_source_limit(self):
+        fetcher = _RecordingFetcher(freq='1h', empty_calls=range(1, 6))
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            with self.assertRaises(DataFetchError) as ctx:
+                get_historical('NOPE-USD', '1m', lookback_days=30)
+        message = str(ctx.exception)
+        self.assertIn('NOPE-USD', message)
+        self.assertIn('1m', message)
+        self.assertIn(str(self.limits['max_days']), message)
+
+class TestTimeframeValidation(unittest.TestCase):
+
+    def test_unknown_timeframe_raises_before_any_request(self):
+        fetcher = _RecordingFetcher()
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            with self.assertRaises(DataFetchError):
+                get_historical('BTC-USD', '3y', lookback_days=5)
+        self.assertEqual(fetcher.calls, [])
+
+    def test_blank_symbol_raises_before_any_request(self):
+        fetcher = _RecordingFetcher()
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            with self.assertRaises(DataFetchError):
+                get_historical('   ', '1d', lookback_days=5)
+        self.assertEqual(fetcher.calls, [])
+
+class TestResampling(unittest.TestCase):
+
+    def test_custom_timeframe_resamples_from_its_source_interval(self):
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df = get_historical('BTC-USD', '4h', lookback_days=5)
+        self.assertEqual([c['interval'] for c in fetcher.calls], ['60m'])
+        spacing = df.index.to_series().diff().dropna().unique()
+        self.assertEqual(list(spacing), [pd.Timedelta(hours=4)])
+        raw = fetcher.raw()
+        bar_time = df.index[len(df) // 2]
+        bucket = raw[(raw.index >= bar_time) & (raw.index < bar_time + pd.Timedelta(hours=4))]
+        self.assertEqual(len(bucket), 4)
+        self.assertAlmostEqual(df.loc[bar_time, 'Open'], float(bucket['Open'].iloc[0]))
+        self.assertAlmostEqual(df.loc[bar_time, 'High'], float(bucket['High'].max()))
+        self.assertAlmostEqual(df.loc[bar_time, 'Low'], float(bucket['Low'].min()))
+        self.assertAlmostEqual(df.loc[bar_time, 'Close'], float(bucket['Close'].iloc[-1]))
+        self.assertAlmostEqual(df.loc[bar_time, 'Volume'], float(bucket['Volume'].sum()))
+
+    def test_resample_ohlc_aggregates_each_field(self):
+        idx = pd.date_range('2024-01-01', periods=4, freq='1h')
+        df = pd.DataFrame({'Open': [10.0, 11.0, 12.0, 13.0], 'High': [15.0, 11.5, 20.0, 13.5], 'Low': [9.0, 8.0, 11.0, 12.0], 'Close': [11.0, 12.0, 13.0, 14.0], 'Volume': [1.0, 2.0, 3.0, 4.0]}, index=idx)
+        out = resample_ohlc(df, '4h')
+        self.assertEqual(len(out), 1)
+        row = out.iloc[0]
+        self.assertEqual(row['Open'], 10.0)
+        self.assertEqual(row['High'], 20.0)
+        self.assertEqual(row['Low'], 8.0)
+        self.assertEqual(row['Close'], 14.0)
+        self.assertEqual(row['Volume'], 10.0)
+
+class TestGetHistoricalWithFeatures(unittest.TestCase):
+
+    def test_features_off_returns_none_and_requests_no_padding(self):
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df, features = get_historical_with_features('BTC-USD', '1h', lookback_days=3, use_features=False)
+        self.assertIsNone(features)
+        self.assertEqual(fetcher.total_span, timedelta(days=3))
+        self.assertGreater(len(df), 0)
+
+    def test_features_on_pads_history_and_returns_aligned_clean_frames(self):
+        fetcher = _RecordingFetcher(freq='1h')
+        with mock.patch('data.historical._fetch_yf', fetcher):
+            df, features = get_historical_with_features('BTC-USD', '1h', lookback_days=3, use_features=True)
+        self.assertGreater(fetcher.total_span, timedelta(days=3))
+        self.assertEqual(list(features.columns), FEATURE_COLUMNS)
+        self.assertEqual(len(features), len(df))
+        self.assertFalse(features.isna().any().any())
+        self.assertLess(len(df), len(fetcher.raw()))
+if __name__ == '__main__':
+    unittest.main()

tests/test_feature_pipeline.py

@@ -0,0 +1,167 @@
+import unittest
+import numpy as np
+import pandas as pd
+from features.feature_pipeline import FEATURE_COLUMNS, carry_forward_future, compute_feature_frame, compute_feature_frame_for_period, slice_features, validate_features
+
+def _synthetic_ohlcv(n: int, seed: int=0) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    steps = rng.normal(loc=0.0, scale=0.5, size=n)
+    close = 100 + np.cumsum(steps)
+    close = np.abs(close) + 1.0
+    open_ = close + rng.normal(0, 0.1, n)
+    high = np.maximum(open_, close) + np.abs(rng.normal(0, 0.2, n))
+    low = np.minimum(open_, close) - np.abs(rng.normal(0, 0.2, n))
+    volume = rng.integers(100, 10000, n).astype(float)
+    idx = pd.date_range('2024-01-01', periods=n, freq='min')
+    return pd.DataFrame({'Open': open_, 'High': high, 'Low': low, 'Close': close, 'Volume': volume}, index=idx)
+
+class TestNoLookahead(unittest.TestCase):
+
+    def test_truncation_invariance(self):
+        df = _synthetic_ohlcv(300, seed=1)
+        full = compute_feature_frame(df)
+        for cutoff in (50, 100, 150, 250, 299):
+            truncated = compute_feature_frame(df.iloc[:cutoff + 1])
+            overlap = min(len(full), len(truncated))
+            pd.testing.assert_frame_equal(full.iloc[:overlap].reset_index(drop=True), truncated.iloc[:overlap].reset_index(drop=True), check_exact=False, atol=1e-10, obj=f'feature frame truncated at row {cutoff}')
+
+    def test_single_row_extension_does_not_rewrite_history(self):
+        df = _synthetic_ohlcv(120, seed=2)
+        shorter = compute_feature_frame(df.iloc[:100])
+        longer = compute_feature_frame(df.iloc[:101])
+        pd.testing.assert_frame_equal(shorter.reset_index(drop=True), longer.iloc[:100].reset_index(drop=True), check_exact=False, atol=1e-10)
+
+class TestShapeAndColumns(unittest.TestCase):
+
+    def test_columns_are_exactly_the_expected_14(self):
+        df = _synthetic_ohlcv(200, seed=3)
+        feats = compute_feature_frame(df)
+        self.assertEqual(list(feats.columns), FEATURE_COLUMNS)
+        self.assertEqual(len(FEATURE_COLUMNS), 14)
+
+    def test_close_is_not_a_feature_column(self):
+        self.assertNotIn('Close', FEATURE_COLUMNS)
+
+    def test_bb_mid_not_duplicated_as_sma20(self):
+        self.assertIn('SMA_20', FEATURE_COLUMNS)
+        self.assertNotIn('BB_Mid', FEATURE_COLUMNS)
+
+class TestNoNaN(unittest.TestCase):
+
+    def test_no_nan_once_past_warmup(self):
+        df = _synthetic_ohlcv(200, seed=4)
+        feats = compute_feature_frame(df)
+        self.assertFalse(feats.iloc[60:].isna().any().any())
+
+    def test_first_19_rows_ARE_nan_without_padding(self):
+        df = _synthetic_ohlcv(500, seed=99)
+        feats = compute_feature_frame(df)
+        self.assertTrue(feats.iloc[:19].isna().any().any())
+        self.assertFalse(feats.iloc[19:].isna().any().any())
+
+    def test_missing_ohlcv_column_raises(self):
+        bad = pd.DataFrame({'Open': [1, 2], 'High': [1, 2], 'Low': [1, 2]})
+        with self.assertRaises(ValueError):
+            compute_feature_frame(bad)
+
+class TestComputeFeatureFrameForPeriod(unittest.TestCase):
+
+    def test_no_nan_anywhere_in_the_trimmed_result(self):
+        df_padded = _synthetic_ohlcv(60 * 24 * 3, seed=21)
+        df_trimmed, feats_trimmed = compute_feature_frame_for_period(df_padded, keep_days=2)
+        self.assertFalse(feats_trimmed.isna().any().any(), 'compute_feature_frame_for_period must never leave leading NaN in the period actually requested')
+
+    def test_trimmed_period_is_the_requested_length_not_the_padded_one(self):
+        df_padded = _synthetic_ohlcv(60 * 24 * 3, seed=22)
+        keep_days = 1
+        df_trimmed, feats_trimmed = compute_feature_frame_for_period(df_padded, keep_days=keep_days)
+        expected_span = df_padded.index[-1] - df_trimmed.index[0]
+        self.assertLessEqual(expected_span, pd.Timedelta(days=keep_days) + pd.Timedelta(minutes=1))
+        self.assertEqual(len(df_trimmed), len(feats_trimmed))
+
+    def test_trimmed_values_match_computing_on_the_full_padded_series(self):
+        df_padded = _synthetic_ohlcv(60 * 24 * 2, seed=23)
+        full_features = compute_feature_frame(df_padded)
+        df_trimmed, feats_trimmed = compute_feature_frame_for_period(df_padded, keep_days=1)
+        keep_mask = df_padded.index >= df_padded.index[-1] - pd.Timedelta(days=1)
+        expected = full_features.iloc[keep_mask].reset_index(drop=True)
+        pd.testing.assert_frame_equal(feats_trimmed, expected, atol=1e-10)
+
+    def test_reproduces_and_fixes_the_reported_btc_1m_window60_case(self):
+        from utils.helpers import feature_warmup_days
+        keep_days = 1
+        padding_days = feature_warmup_days('1m', minimum_days=0)
+        df_padded = _synthetic_ohlcv(60 * 24 * (keep_days + padding_days) + 100, seed=24)
+        df_trimmed, feats_trimmed = compute_feature_frame_for_period(df_padded, keep_days=keep_days)
+        window = 60
+        first_window = slice_features(feats_trimmed, 0, window)
+        self.assertFalse(first_window.isna().any().any(), 'the FIRST walk-forward window must be NaN-free after padding')
+
+class TestSliceFeatures(unittest.TestCase):
+
+    def test_slice_matches_close_slice_positionally(self):
+        df = _synthetic_ohlcv(200, seed=5)
+        feats = compute_feature_frame(df)
+        close = df['Close'].reset_index(drop=True)
+        window, i = (60, 130)
+        close_window = close.iloc[i - window:i]
+        feat_window = slice_features(feats, i - window, i)
+        self.assertEqual(len(close_window), len(feat_window))
+        self.assertAlmostEqual(df['Close'].iloc[i - 1], close_window.iloc[-1], places=8)
+
+class TestValidateFeatures(unittest.TestCase):
+
+    def test_length_mismatch_raises(self):
+        feats = pd.DataFrame({c: [1.0, 2.0] for c in FEATURE_COLUMNS})
+        with self.assertRaises(ValueError):
+            validate_features(feats, expected_len=3, model_name='Test')
+
+    def test_nan_raises(self):
+        feats = pd.DataFrame({c: [1.0, np.nan] for c in FEATURE_COLUMNS})
+        with self.assertRaises(ValueError):
+            validate_features(feats, expected_len=2, model_name='Test')
+
+    def test_check_nan_false_skips_nan_but_still_checks_length(self):
+        feats_with_nan = pd.DataFrame({c: [np.nan, 1.0, 2.0] for c in FEATURE_COLUMNS})
+        validate_features(feats_with_nan, expected_len=3, model_name='Test', check_nan=False)
+        with self.assertRaises(ValueError):
+            validate_features(feats_with_nan, expected_len=2, model_name='Test', check_nan=False)
+
+    def test_check_nan_false_then_true_on_truncated_tail_mirrors_model_usage(self):
+        feats = pd.DataFrame({c: [np.nan, 1.0, 2.0] for c in FEATURE_COLUMNS})
+        validate_features(feats, expected_len=3, model_name='Test', check_nan=False)
+        clean_tail = feats.iloc[-2:].reset_index(drop=True)
+        validate_features(clean_tail, expected_len=2, model_name='Test')
+        dirty_tail = feats.iloc[-3:].reset_index(drop=True)
+        with self.assertRaises(ValueError):
+            validate_features(dirty_tail, expected_len=3, model_name='Test')
+
+    def test_clean_window_passes(self):
+        feats = pd.DataFrame({c: [1.0, 2.0, 3.0] for c in FEATURE_COLUMNS})
+        validate_features(feats, expected_len=3, model_name='Test')
+
+class TestCarryForwardFuture(unittest.TestCase):
+
+    def test_repeats_last_row(self):
+        feats = pd.DataFrame({'a': [1.0, 2.0, 3.0], 'b': [10.0, 20.0, 30.0]})
+        future = carry_forward_future(feats, horizon=4)
+        self.assertEqual(len(future), 4)
+        self.assertTrue((future['a'] == 3.0).all())
+        self.assertTrue((future['b'] == 30.0).all())
+
+    def test_uses_zero_future_information(self):
+        df = _synthetic_ohlcv(150, seed=6)
+        feats_short = compute_feature_frame(df.iloc[:80])
+        feats_long = compute_feature_frame(df)
+        future_from_short = carry_forward_future(feats_short.iloc[:80], horizon=5)
+        future_from_long = carry_forward_future(feats_long.iloc[:80], horizon=5)
+        pd.testing.assert_frame_equal(future_from_short, future_from_long, atol=1e-10)
+
+    def test_rejects_bad_input(self):
+        feats = pd.DataFrame({'a': [1.0]})
+        with self.assertRaises(ValueError):
+            carry_forward_future(feats, horizon=0)
+        with self.assertRaises(ValueError):
+            carry_forward_future(pd.DataFrame({'a': []}), horizon=1)
+if __name__ == '__main__':
+    unittest.main()

tests/test_helpers.py

@@ -0,0 +1,113 @@
+import unittest
+import pandas as pd
+from config import CUSTOM_TIMEFRAMES, YF_NATIVE_INTERVALS
+from utils.helpers import INDICATOR_WARMUP_CANDLES, MINUTES_PER_CANDLE, PREDICTION_HISTORY_CANDLES, drop_forming_candle, feature_warmup_days, future_index, history_days_for_candles, infer_step
+
+def _frame(index) -> pd.DataFrame:
+    return pd.DataFrame({'Close': range(len(index))}, index=index)
+
+class TestHistoryDaysForCandles(unittest.TestCase):
+
+    def test_window_covers_the_requested_candle_count(self):
+        for timeframe, minutes in MINUTES_PER_CANDLE.items():
+            days = history_days_for_candles(timeframe, 200)
+            self.assertGreaterEqual(days * 24 * 60, 200 * minutes, timeframe)
+
+    def test_known_sizes(self):
+        self.assertEqual(history_days_for_candles('1d', 200), 200)
+        self.assertEqual(history_days_for_candles('4h', 200), 34)
+        self.assertEqual(history_days_for_candles('5m', 200), 1)
+
+    def test_minimum_days_acts_as_a_floor_only(self):
+        self.assertEqual(history_days_for_candles('1m', 10, minimum_days=2), 2)
+        self.assertEqual(history_days_for_candles('1d', 10, minimum_days=2), 10)
+
+    def test_non_positive_candle_counts_still_request_a_real_window(self):
+        self.assertEqual(history_days_for_candles('1h', 0), 1)
+        self.assertEqual(history_days_for_candles('1h', -5), 1)
+
+    def test_unknown_timeframe_uses_the_hourly_fallback(self):
+        self.assertEqual(history_days_for_candles('7m', 200), history_days_for_candles('1h', 200))
+
+class TestFeatureWarmupDays(unittest.TestCase):
+
+    def test_padding_covers_the_indicator_warmup(self):
+        for timeframe, minutes in MINUTES_PER_CANDLE.items():
+            days = feature_warmup_days(timeframe, minimum_days=0)
+            self.assertGreaterEqual(days * 24 * 60, INDICATOR_WARMUP_CANDLES * minutes, timeframe)
+
+    def test_default_floor_is_two_days(self):
+        self.assertEqual(feature_warmup_days('1m'), 2)
+
+    def test_floor_can_be_dropped_for_padding_math(self):
+        self.assertEqual(feature_warmup_days('1m', minimum_days=0), 1)
+        self.assertEqual(feature_warmup_days('1d', minimum_days=0), 30)
+
+class TestLiveSizingFitsSourceLimits(unittest.TestCase):
+
+    def test_prediction_history_is_servable_for_every_timeframe(self):
+        for timeframe in MINUTES_PER_CANDLE:
+            source = CUSTOM_TIMEFRAMES.get(timeframe, {}).get('source', timeframe)
+            limit = YF_NATIVE_INTERVALS.get(source, {}).get('max_days')
+            if limit is None:
+                continue
+            wanted = history_days_for_candles(timeframe, PREDICTION_HISTORY_CANDLES)
+            self.assertLessEqual(wanted, limit, f"{timeframe} wants {wanted}d but '{source}' serves {limit}d")
+
+class TestInferStep(unittest.TestCase):
+
+    def test_uses_the_median_spacing(self):
+        idx = pd.date_range('2024-01-01', periods=5, freq='15min')
+        self.assertEqual(infer_step(idx), pd.Timedelta(minutes=15))
+
+    def test_market_closure_gap_does_not_dominate(self):
+        idx = pd.DatetimeIndex(['2024-01-05 20:00', '2024-01-05 21:00', '2024-01-05 22:00', '2024-01-08 20:00', '2024-01-08 21:00'])
+        self.assertEqual(infer_step(idx), pd.Timedelta(hours=1))
+
+    def test_single_candle_falls_back_to_one_minute(self):
+        idx = pd.date_range('2024-01-01', periods=1, freq='5min')
+        self.assertEqual(infer_step(idx), pd.Timedelta(minutes=1))
+
+class TestFutureIndex(unittest.TestCase):
+
+    def test_projects_timestamps_after_the_last_candle(self):
+        last = pd.Timestamp('2024-01-01 00:00')
+        self.assertEqual(future_index(last, pd.Timedelta(minutes=5), 3), [pd.Timestamp('2024-01-01 00:05'), pd.Timestamp('2024-01-01 00:10'), pd.Timestamp('2024-01-01 00:15')])
+
+    def test_zero_horizon_projects_nothing(self):
+        self.assertEqual(future_index(pd.Timestamp('2024-01-01'), pd.Timedelta(minutes=1), 0), [])
+
+class TestDropFormingCandle(unittest.TestCase):
+
+    def test_drops_the_still_forming_candle_and_its_feature_row(self):
+        now = pd.Timestamp.now(tz='UTC').floor('min')
+        idx = pd.DatetimeIndex([now - pd.Timedelta(minutes=10), now - pd.Timedelta(minutes=5), now])
+        features = pd.DataFrame({'f': [1.0, 2.0, 3.0]})
+        df, trimmed_features = drop_forming_candle(_frame(idx), features)
+        self.assertEqual(len(df), 2)
+        self.assertEqual(df.index[-1], idx[-2])
+        self.assertEqual(len(trimmed_features), 2)
+        self.assertEqual(list(trimmed_features.index), [0, 1])
+
+    def test_keeps_a_closed_final_candle(self):
+        now = pd.Timestamp.now(tz='UTC').floor('min')
+        idx = pd.DatetimeIndex([now - pd.Timedelta(minutes=20), now - pd.Timedelta(minutes=15), now - pd.Timedelta(minutes=10)])
+        features = pd.DataFrame({'f': [1.0, 2.0, 3.0]})
+        df, kept_features = drop_forming_candle(_frame(idx), features)
+        self.assertEqual(len(df), 3)
+        self.assertEqual(len(kept_features), 3)
+
+    def test_handles_a_tz_naive_index(self):
+        now = pd.Timestamp.now(tz='UTC').tz_localize(None).floor('min')
+        idx = pd.DatetimeIndex([now - pd.Timedelta(minutes=10), now - pd.Timedelta(minutes=5), now])
+        df, features = drop_forming_candle(_frame(idx), None)
+        self.assertEqual(len(df), 2)
+        self.assertIsNone(features)
+
+    def test_empty_frame_passes_through(self):
+        empty = pd.DataFrame({'Close': []}, index=pd.DatetimeIndex([]))
+        df, features = drop_forming_candle(empty, None)
+        self.assertEqual(len(df), 0)
+        self.assertIsNone(features)
+if __name__ == '__main__':
+    unittest.main()

tests/test_indicators.py

@@ -0,0 +1,144 @@
+import unittest
+import numpy as np
+import pandas as pd
+from indicators.indicators import _wilder_rma, add_all_indicators, bollinger_bands, ema, latest_signals, macd, rsi, sma, stochastic_oscillator
+
+def _ohlcv(n: int, seed: int=5) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    close = 100 + np.cumsum(rng.normal(0, 0.5, n))
+    open_ = close + rng.normal(0, 0.1, n)
+    high = np.maximum(open_, close) + np.abs(rng.normal(0, 0.3, n))
+    low = np.minimum(open_, close) - np.abs(rng.normal(0, 0.3, n))
+    volume = rng.integers(100, 5000, n).astype(float)
+    idx = pd.date_range('2024-01-01', periods=n, freq='5min')
+    return pd.DataFrame({'Open': open_, 'High': high, 'Low': low, 'Close': close, 'Volume': volume}, index=idx)
+
+class TestWilderRma(unittest.TestCase):
+
+    def test_seeds_from_a_window_length_average_then_recurses(self):
+        values = pd.Series([1.0, 2.0, 3.0, 4.0, 5.0])
+        out = _wilder_rma(values, 3)
+        self.assertTrue(out.iloc[:2].isna().all())
+        self.assertAlmostEqual(out.iloc[2], 2.0)
+        self.assertAlmostEqual(out.iloc[3], (2.0 * 2 + 4.0) / 3)
+        self.assertAlmostEqual(out.iloc[4], (out.iloc[3] * 2 + 5.0) / 3)
+
+    def test_skips_a_leading_nan_before_seeding(self):
+        values = pd.Series([np.nan, 2.0, 4.0, 6.0])
+        out = _wilder_rma(values, 3)
+        self.assertTrue(out.iloc[:3].isna().all())
+        self.assertAlmostEqual(out.iloc[3], 4.0)
+
+    def test_series_shorter_than_the_window_is_all_nan(self):
+        out = _wilder_rma(pd.Series([1.0, 2.0]), 14)
+        self.assertTrue(out.isna().all())
+
+class TestRsi(unittest.TestCase):
+
+    def test_warmup_rows_are_nan_not_fifty(self):
+        out = rsi(_ohlcv(60)['Close'], 14)
+        self.assertTrue(out.iloc[:14].isna().all())
+        self.assertTrue(out.iloc[14:].notna().all())
+        self.assertEqual(int((out.iloc[:14] == 50.0).sum()), 0)
+
+    def test_only_gains_reads_one_hundred(self):
+        out = rsi(pd.Series(np.arange(40, dtype=float) + 100), 14)
+        self.assertTrue(out.iloc[:14].isna().all())
+        self.assertTrue((out.iloc[14:] == 100.0).all())
+
+    def test_flat_market_is_neutral_only_after_warmup(self):
+        out = rsi(pd.Series([100.0] * 40), 14)
+        self.assertTrue(out.iloc[:14].isna().all())
+        self.assertTrue((out.iloc[14:] == 50.0).all())
+
+    def test_stays_within_bounds(self):
+        out = rsi(_ohlcv(200, seed=9)['Close'], 14).dropna()
+        self.assertGreaterEqual(out.min(), 0.0)
+        self.assertLessEqual(out.max(), 100.0)
+
+class TestMacdAndBollinger(unittest.TestCase):
+
+    def setUp(self):
+        self.close = _ohlcv(80, seed=3)['Close']
+
+    def test_macd_is_the_fast_minus_slow_ema_with_its_own_signal_ema(self):
+        line, signal, hist = macd(self.close)
+        expected_line = ema(self.close, 12) - ema(self.close, 26)
+        pd.testing.assert_series_equal(line, expected_line, check_names=False)
+        pd.testing.assert_series_equal(signal, ema(expected_line, 9), check_names=False)
+        pd.testing.assert_series_equal(hist, expected_line - signal, check_names=False)
+
+    def test_bollinger_uses_population_std_around_the_sma(self):
+        upper, mid, lower = bollinger_bands(self.close, window=20, num_std=2.0)
+        pd.testing.assert_series_equal(mid, sma(self.close, 20), check_names=False)
+        self.assertTrue(upper.iloc[:19].isna().all())
+        window = self.close.iloc[-20:].to_numpy()
+        expected_std = float(np.std(window))
+        self.assertAlmostEqual(float(upper.iloc[-1]), float(window.mean()) + 2 * expected_std, places=9)
+        self.assertAlmostEqual(float(lower.iloc[-1]), float(window.mean()) - 2 * expected_std, places=9)
+
+class TestStochastic(unittest.TestCase):
+
+    def test_warmup_rows_are_nan_not_fifty(self):
+        df = _ohlcv(60, seed=4)
+        k, d = stochastic_oscillator(df['High'], df['Low'], df['Close'])
+        self.assertTrue(k.iloc[:13].isna().all())
+        self.assertTrue(k.iloc[13:].notna().all())
+        self.assertTrue(d.iloc[:15].isna().all())
+        self.assertEqual(int((k.iloc[:13] == 50.0).sum()), 0)
+
+    def test_percent_k_matches_the_definition(self):
+        df = _ohlcv(40, seed=6)
+        k, _ = stochastic_oscillator(df['High'], df['Low'], df['Close'])
+        lowest = float(df['Low'].iloc[-14:].min())
+        highest = float(df['High'].iloc[-14:].max())
+        expected = 100 * (float(df['Close'].iloc[-1]) - lowest) / (highest - lowest)
+        self.assertAlmostEqual(float(k.iloc[-1]), expected, places=9)
+
+    def test_zero_range_is_neutral_only_after_warmup(self):
+        flat = pd.Series([100.0] * 30)
+        k, d = stochastic_oscillator(flat, flat, flat)
+        self.assertTrue(k.iloc[:13].isna().all())
+        self.assertTrue((k.iloc[13:] == 50.0).all())
+        self.assertTrue((d.iloc[15:] == 50.0).all())
+
+class TestAddAllIndicators(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _ohlcv(60, seed=8)
+        self.out = add_all_indicators(self.df)
+
+    def test_adds_every_indicator_column_without_mutating_the_input(self):
+        expected = ['SMA_20', 'EMA_20', 'RSI_14', 'MACD', 'MACD_Signal', 'MACD_Hist', 'BB_Upper', 'BB_Mid', 'BB_Lower', 'Stoch_%K', 'Stoch_%D']
+        for column in expected:
+            self.assertIn(column, self.out.columns)
+            self.assertNotIn(column, self.df.columns)
+
+    def test_warmup_nan_counts_are_the_real_indicator_periods(self):
+        self.assertEqual(int(self.out['RSI_14'].isna().sum()), 14)
+        self.assertEqual(int(self.out['Stoch_%K'].isna().sum()), 13)
+        self.assertEqual(int(self.out['Stoch_%D'].isna().sum()), 15)
+        self.assertEqual(int(self.out['SMA_20'].isna().sum()), 19)
+        self.assertEqual(int(self.out['BB_Upper'].isna().sum()), 19)
+
+class TestLatestSignals(unittest.TestCase):
+
+    def test_reports_insufficient_data_instead_of_a_fabricated_reading(self):
+        out = add_all_indicators(_ohlcv(3, seed=2))
+        signals = latest_signals(out)
+        self.assertEqual(signals['RSI (14)']['value'], 'n/a')
+        self.assertEqual(signals['RSI (14)']['signal'], 'Insufficient data')
+        self.assertEqual(signals['Stochastic (14,3)']['value'], 'n/a')
+        self.assertEqual(signals['Stochastic (14,3)']['signal'], 'Insufficient data')
+        self.assertEqual(signals['Bollinger Bands']['signal'], 'Insufficient data')
+        self.assertEqual(signals['Moving Averages']['signal'], 'Insufficient data for SMA')
+        self.assertIn('SMA20=n/a', signals['Moving Averages']['value'])
+
+    def test_reports_real_readings_once_warmed_up(self):
+        signals = latest_signals(add_all_indicators(_ohlcv(80, seed=1)))
+        for name, entry in signals.items():
+            self.assertNotIn('Insufficient', entry['signal'], name)
+        self.assertIsInstance(signals['RSI (14)']['value'], float)
+        self.assertIsInstance(signals['Stochastic (14,3)']['value'], float)
+if __name__ == '__main__':
+    unittest.main()

tests/test_maythos_integration.py

@@ -0,0 +1,70 @@
+import ast
+from pathlib import Path
+import unittest
+
+import numpy as np
+import pandas as pd
+
+from maythos_integration import backtest_maythos, run_maythos_history
+
+
+def _synthetic_ohlcv(count=120):
+    index = pd.date_range('2026-01-01', periods=count, freq='5min', tz='UTC')
+    close = 100.0 + np.arange(count, dtype=float) * 0.05
+    opening = np.concatenate(([close[0]], close[:-1]))
+    return pd.DataFrame(
+        {
+            'Open': opening,
+            'High': close + 0.1,
+            'Low': opening - 0.1,
+            'Close': close,
+            'Volume': np.full(count, 100.0),
+        },
+        index=index,
+    )
+
+
+class TestMaythosIntegration(unittest.TestCase):
+    def test_every_closed_candle_produces_a_valid_maythos_output(self):
+        history = _synthetic_ohlcv()
+        rows = run_maythos_history(history)
+
+        self.assertEqual(len(rows), len(history))
+        self.assertEqual(rows.iloc[0]['blocked'], True)
+        self.assertIn(rows.iloc[-1]['direction'], ('BUY', 'SELL'))
+        self.assertTrue(rows.iloc[-1]['confidence'] >= 0.0)
+        self.assertIn(rows.iloc[-1]['execution_suitability'], ('blocked', 'weak', 'moderate', 'strong'))
+
+    def test_backtest_uses_signal_before_the_target_candle(self):
+        history = _synthetic_ohlcv()
+        signal_rows = run_maythos_history(history)
+        results, summary = backtest_maythos(history, window=60, horizon=1, step=5)
+
+        self.assertEqual(results.iloc[0]['current_close'], history['Close'].iloc[59])
+        self.assertEqual(results.iloc[0]['actual_close'], history['Close'].iloc[60])
+        self.assertEqual(results.iloc[0]['confidence'], signal_rows.iloc[59]['confidence'])
+        self.assertEqual(summary['engine'], 'MAYTHOS')
+        self.assertEqual(summary['evaluations'], len(results))
+
+
+class TestDashboardLayout(unittest.TestCase):
+    def test_dashboard_has_two_required_tabs_and_market_model_selectors(self):
+        source = Path(__file__).parents[1].joinpath('app.py').read_text(encoding='utf-8')
+        tree = ast.parse(source)
+        tab_calls = [
+            node for node in ast.walk(tree)
+            if isinstance(node, ast.Call)
+            and isinstance(node.func, ast.Attribute)
+            and node.func.attr == 'Tab'
+        ]
+        self.assertEqual(len(tab_calls), 2)
+        constructor_names = {
+            node.func.attr
+            for node in ast.walk(tree)
+            if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute)
+        }
+        self.assertTrue({'Dropdown', 'Checkbox', 'CheckboxGroup', 'Radio'}.issubset(constructor_names))
+
+
+if __name__ == '__main__':
+    unittest.main()

tests/test_models_with_features.py

@@ -0,0 +1,176 @@
+import importlib.util
+import unittest
+import numpy as np
+import pandas as pd
+from features.feature_pipeline import compute_feature_frame
+
+def _have(module_name: str) -> bool:
+    return importlib.util.find_spec(module_name) is not None
+_HAVE_STATSMODELS = _have('statsmodels')
+_HAVE_ARCH = _have('arch')
+_HAVE_MOIRAI = _have('uni2ts') and _have('gluonts')
+_HAVE_TIMESFM = _have('timesfm') and _have('torch')
+
+def _synthetic_ohlcv(n: int, seed: int=42) -> pd.DataFrame:
+    rng = np.random.default_rng(seed)
+    steps = rng.normal(loc=0.0, scale=0.4, size=n)
+    close = 100 + np.cumsum(steps)
+    close = np.abs(close) + 5.0
+    open_ = close + rng.normal(0, 0.1, n)
+    high = np.maximum(open_, close) + np.abs(rng.normal(0, 0.2, n))
+    low = np.minimum(open_, close) - np.abs(rng.normal(0, 0.2, n))
+    volume = rng.integers(100, 10000, n).astype(float)
+    idx = pd.date_range('2024-01-01', periods=n, freq='min')
+    return pd.DataFrame({'Open': open_, 'High': high, 'Low': low, 'Close': close, 'Volume': volume}, index=idx)
+
+def _warm_close_and_features(df: pd.DataFrame, feats: pd.DataFrame):
+    complete = ~feats.isna().any(axis=1).to_numpy()
+    start = int(np.argmax(complete)) if complete.any() else len(df)
+    return (df['Close'].reset_index(drop=True).iloc[start:], feats.iloc[start:].reset_index(drop=True))
+
+class TestArimaFamilyAlignment(unittest.TestCase):
+
+    def test_returns_alignment_has_no_off_by_one(self):
+        n = 10
+        prices = pd.Series(100 + np.arange(n, dtype=float))
+        features = pd.Series([float(i) for i in range(n)])
+        log_returns = np.log(prices / prices.shift(1)).dropna().reset_index(drop=True)
+        exog = features.iloc[1:].reset_index(drop=True)
+        self.assertEqual(len(exog), len(log_returns))
+        for k in range(len(log_returns)):
+            self.assertEqual(exog[k], float(k + 1))
+
+@unittest.skipUnless(_HAVE_STATSMODELS, 'statsmodels not installed in this environment')
+class TestArimaWithFeatures(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _synthetic_ohlcv(120)
+        self.feats = compute_feature_frame(self.df)
+        self.close, self.warm_feats = _warm_close_and_features(self.df, self.feats)
+
+    def test_baseline_unchanged_shape(self):
+        from models.arima_model import ArimaModel
+        m = ArimaModel(max_p=2, max_d=1, max_q=2)
+        out = m.predict(self.close, horizon=3)
+        self.assertEqual(len(out), 3)
+        self.assertTrue(all(np.isfinite(out)))
+
+    def test_with_features_runs_and_is_finite(self):
+        from models.arima_model import ArimaModel
+        m = ArimaModel(max_p=2, max_d=1, max_q=2)
+        out = m.predict(self.close, horizon=3, features=self.warm_feats)
+        self.assertEqual(len(out), 3)
+        self.assertTrue(all(np.isfinite(out)))
+
+    def test_mismatched_features_length_raises(self):
+        from models.arima_model import ArimaModel
+        m = ArimaModel(max_p=2, max_d=1, max_q=2)
+        short_window = self.warm_feats.iloc[:-5].reset_index(drop=True)
+        with self.assertRaises(ValueError):
+            m.predict(self.close, horizon=3, features=short_window)
+
+@unittest.skipUnless(_HAVE_STATSMODELS, 'statsmodels not installed in this environment')
+class TestAutoArimaWithFeatures(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _synthetic_ohlcv(120)
+        self.feats = compute_feature_frame(self.df)
+        self.close, self.warm_feats = _warm_close_and_features(self.df, self.feats)
+
+    def test_with_features_runs_and_is_finite(self):
+        from models.auto_arima_model import AutoArimaModel
+        m = AutoArimaModel(max_p=3, max_q=3, max_d=1)
+        out = m.predict(self.close, horizon=2, features=self.warm_feats)
+        self.assertEqual(len(out), 2)
+        self.assertTrue(all(np.isfinite(out)))
+
+@unittest.skipUnless(_HAVE_STATSMODELS and _HAVE_ARCH, 'statsmodels+arch not installed in this environment')
+class TestArimaGarchWithFeatures(unittest.TestCase):
+
+    def setUp(self):
+        self.df = _synthetic_ohlcv(150)
+        self.feats = compute_feature_frame(self.df)
+        self.close, self.warm_feats = _warm_close_and_features(self.df, self.feats)
+
+    def test_with_features_runs_and_is_finite(self):
+        from models.arima_garch_model import ArimaGarchModel
+        m = ArimaGarchModel(max_p=2, max_q=2)
+        out = m.predict(self.close, horizon=2, features=self.warm_feats)
+        self.assertEqual(len(out), 2)
+        self.assertTrue(all(np.isfinite(out)))
+        self.assertIsNotNone(m.last_volatility_forecast)
+
+@unittest.skipUnless(_HAVE_MOIRAI, 'uni2ts/gluonts not installed in this environment')
+class TestMoiraiWithFeatures(unittest.TestCase):
+
+    def test_with_features_runs_and_is_finite(self):
+        from models.moirai_model import MoiraiModel
+        df = _synthetic_ohlcv(60)
+        feats = compute_feature_frame(df)
+        close, warm_feats = _warm_close_and_features(df, feats)
+        m = MoiraiModel(context_length=60, num_samples=10)
+        out = m.predict(close, horizon=2, features=warm_feats)
+        self.assertEqual(len(out), 2)
+        self.assertTrue(all(np.isfinite(out)))
+
+@unittest.skipUnless(_HAVE_TIMESFM, 'timesfm/torch not installed in this environment')
+class TestTimesFMWithFeatures(unittest.TestCase):
+
+    def test_with_features_runs_and_is_finite(self):
+        from models.timesfm_model import TimesFMModel
+        df = _synthetic_ohlcv(64)
+        feats = compute_feature_frame(df)
+        close, warm_feats = _warm_close_and_features(df, feats)
+        m = TimesFMModel(max_context=64)
+        out = m.predict(close, horizon=2, features=warm_feats)
+        self.assertEqual(len(out), 2)
+        self.assertTrue(all(np.isfinite(out)))
+
+    def test_short_input_below_compiled_context_is_finite(self):
+        from models.timesfm_model import TimesFMModel
+        df = _synthetic_ohlcv(60, seed=99)
+        close = 40000 + df['Close'].reset_index(drop=True)
+        m = TimesFMModel(max_context=512)
+        out = m.predict(close, horizon=2)
+        self.assertEqual(len(out), 2)
+        self.assertTrue(all(np.isfinite(out)), f'TimesFM NaN on short window: {out}')
+
+    def test_repeated_window_lengths_stay_finite(self):
+        from models.timesfm_model import TimesFMModel
+        df = _synthetic_ohlcv(120, seed=7)
+        close = 40000 + df['Close'].reset_index(drop=True)
+        m = TimesFMModel(max_context=512)
+        for w in (40, 100, 40):
+            out = m.predict(close.iloc[-w:], horizon=1)
+            self.assertEqual(len(out), 1)
+            self.assertTrue(np.isfinite(out[0]), f'TimesFM NaN at w={w}: {out[0]}')
+
+class TestInputContractGuards(unittest.TestCase):
+
+    def test_nan_history_raises_for_all_models(self):
+        from models.registry import fresh_model
+        rng = np.random.default_rng(3)
+        good = pd.Series(100 + np.cumsum(rng.normal(0, 0.5, 200)))
+        for name in ('ARIMA', 'Auto-ARIMA', 'ARIMA-GARCH', 'Moirai'):
+            m = fresh_model(name)
+            h = good.copy()
+            h.iloc[50] = np.nan
+            with self.assertRaises(ValueError, msg=name):
+                m.predict(h, horizon=1)
+        m = fresh_model('TimesFM')
+        h = good.copy()
+        h.iloc[50] = np.nan
+        with self.assertRaises(ValueError):
+            m.predict(h, horizon=1)
+
+    def test_horizon_below_one_raises_for_all_models(self):
+        from models.registry import fresh_model
+        rng = np.random.default_rng(3)
+        good = pd.Series(100 + np.cumsum(rng.normal(0, 0.5, 200)))
+        for name in ('ARIMA', 'Auto-ARIMA', 'ARIMA-GARCH', 'Moirai', 'TimesFM'):
+            m = fresh_model(name)
+            for bad in (0, -1):
+                with self.assertRaises(ValueError, msg=f'{name} horizon={bad}'):
+                    m.predict(good, horizon=bad)
+if __name__ == '__main__':
+    unittest.main()

tests/test_risk_management.py

@@ -0,0 +1,110 @@
+import unittest
+
+import numpy as np
+import pandas as pd
+
+from backtest.backtester import backtest_model, summarize_backtest
+from risk_management import CostModel, build_trade_levels, infer_tick_size, simulate_trade
+from models.base_model import BaseForecastModel
+
+
+def _history(count=60):
+    index = pd.date_range('2026-01-01', periods=count, freq='5min', tz='UTC')
+    close = np.linspace(98.0, 100.0, count)
+    close[-1] = 100.0
+    opening = close - 0.05
+    return pd.DataFrame({
+        'Open': opening,
+        'High': close + 0.8,
+        'Low': close - 0.8,
+        'Close': close,
+        'Volume': 1000.0,
+    }, index=index)
+
+
+class _UpModel(BaseForecastModel):
+    name = 'UpModel'
+
+    def predict(self, history, horizon=1, features=None):
+        current = float(pd.Series(history).iloc[-1])
+        return [current + 3.0] * horizon
+
+
+class TestTradeLevels(unittest.TestCase):
+    def test_levels_use_atr_and_prior_structure_with_quote_precision(self):
+        frame = _history()
+        levels = build_trade_levels(frame, 'BUY', 'Crypto', 'BTC-USD', forecast_target=103.0)
+        self.assertIsNotNone(levels)
+        self.assertGreater(levels.atr, 0)
+        self.assertLess(levels.stop_loss, levels.entry)
+        self.assertGreater(levels.take_profit, levels.entry)
+        self.assertGreaterEqual(levels.risk_reward, 1.0)
+        self.assertEqual(round(levels.stop_loss, 2), levels.stop_loss)
+        self.assertEqual(round(levels.take_profit, 2), levels.take_profit)
+
+    def test_future_rows_do_not_change_origin_levels(self):
+        frame = _history()
+        origin = frame.iloc[:50]
+        changed_future = frame.copy()
+        changed_future.iloc[50:, changed_future.columns.get_loc('High')] = 10000.0
+        changed_future.iloc[50:, changed_future.columns.get_loc('Low')] = 1.0
+        first = build_trade_levels(origin, 'BUY', 'Crypto', 'BTC-USD', forecast_target=103.0)
+        second = build_trade_levels(changed_future.iloc[:50], 'BUY', 'Crypto', 'BTC-USD', forecast_target=103.0)
+        self.assertEqual(first, second)
+
+    def test_same_candle_sl_is_conservatively_first(self):
+        frame = _history()
+        levels = build_trade_levels(frame, 'BUY', 'Crypto', 'BTC-USD', forecast_target=103.0)
+        path = pd.DataFrame({'High': [levels.take_profit + 1], 'Low': [levels.stop_loss - 1], 'Close': [levels.take_profit]}, index=frame.index[-1:])
+        trade = simulate_trade(levels, path, CostModel(spread_bps=0, fee_bps=0, slippage_bps=0))
+        self.assertEqual(trade['outcome'], 'SL')
+
+    def test_costs_reduce_realized_r(self):
+        frame = _history()
+        levels = build_trade_levels(frame, 'BUY', 'Crypto', 'BTC-USD', forecast_target=103.0)
+        path = pd.DataFrame({'High': [levels.take_profit], 'Low': [levels.entry], 'Close': [levels.take_profit]}, index=frame.index[-1:])
+        free = simulate_trade(levels, path, CostModel(spread_bps=0, fee_bps=0, slippage_bps=0))
+        costly = simulate_trade(levels, path, CostModel(spread_bps=10, fee_bps=10, slippage_bps=10))
+        self.assertGreater(free['net_r'], costly['net_r'])
+
+    def test_forex_quote_precision_handles_pips_and_jpy(self):
+        frame = _history()
+        eur_tick, eur_digits = infer_tick_size(frame, 'Forex', 'EURUSD=X')
+        jpy_tick, jpy_digits = infer_tick_size(frame, 'Forex', 'USDJPY=X')
+        self.assertEqual((eur_tick, eur_digits), (0.00001, 5))
+        self.assertEqual((jpy_tick, jpy_digits), (0.001, 3))
+
+    def test_forecast_target_is_not_extended_to_distant_swing(self):
+        frame = _history()
+        levels = build_trade_levels(
+            frame, 'BUY', 'Crypto', 'BTC-USD', forecast_target=100.5,
+            forecast_only=True, use_structure_stop=False, minimum_risk_reward=0.1,
+        )
+        self.assertIsNotNone(levels)
+        self.assertEqual(levels.take_profit, 100.5)
+
+    def test_forecast_execution_rejects_target_below_cost_edge(self):
+        frame = _history()
+        levels = build_trade_levels(
+            frame, 'BUY', 'Crypto', 'BTC-USD', forecast_target=100.01,
+            forecast_only=True, use_structure_stop=False,
+            minimum_edge_bps=10.0,
+        )
+        self.assertIsNone(levels)
+
+
+class TestBacktestTradeMetrics(unittest.TestCase):
+    def test_backtest_contains_candle_path_trade_fields_and_summary(self):
+        frame = _history(80)
+        results = backtest_model(frame, _UpModel(), window=40, horizon=2, step=10, timeframe='5m', market='Crypto', symbol='BTC-USD', costs=CostModel())
+        self.assertIn('stop_loss', results.columns)
+        self.assertIn('take_profit', results.columns)
+        self.assertIn('outcome', results.columns)
+        summary = summarize_backtest(results, 'UpModel')
+        self.assertIn('trades', summary)
+        self.assertIn('profit_factor', summary)
+        self.assertIn('max_drawdown_r', summary)
+
+
+if __name__ == '__main__':
+    unittest.main()

utils/__init__.py

@@ -0,0 +1 @@
+pass

utils/helpers.py

@@ -0,0 +1,65 @@
+import math
+import numbers
+import numpy as np
+import pandas as pd
+MINUTES_PER_CANDLE = {'1m': 1, '2m': 2, '5m': 5, '10m': 10, '15m': 15, '30m': 30, '1h': 60, '2h': 120, '4h': 240, '1d': 1440}
+INDICATOR_WARMUP_CANDLES = 30
+PREDICTION_HISTORY_CANDLES = 200
+
+def infer_step(index: pd.DatetimeIndex):
+    if not isinstance(index, pd.DatetimeIndex) or index.isna().any() or index.duplicated().any() or not index.is_monotonic_increasing:
+        raise ValueError('infer_step needs ordered unique datetime timestamps.')
+    if len(index) < 2:
+        return pd.Timedelta(minutes=1)
+    diffs = index.to_series().diff().dropna()
+    if diffs.empty:
+        return pd.Timedelta(minutes=1)
+    return diffs.median()
+
+def future_index(last_timestamp, step, n):
+    if isinstance(n, (bool, np.bool_)) or not isinstance(n, numbers.Integral) or n < 0 or pd.Timedelta(step) <= pd.Timedelta(0):
+        raise ValueError('Future index requires a non-negative integer horizon and positive step.')
+    return [last_timestamp + step * (i + 1) for i in range(n)]
+
+def history_days_for_candles(timeframe: str, candles: int, minimum_days: int=1) -> int:
+    minutes = MINUTES_PER_CANDLE.get(timeframe, MINUTES_PER_CANDLE['1h']) * max(int(candles), 0)
+    return max(minimum_days, math.ceil(minutes / (24 * 60)))
+
+def feature_warmup_days(timeframe: str, minimum_days: int=2) -> int:
+    return history_days_for_candles(timeframe, INDICATOR_WARMUP_CANDLES, minimum_days=minimum_days)
+
+def drop_forming_candle(df: pd.DataFrame, features_df: pd.DataFrame=None, timeframe: str=None, now=None):
+    if df is None or len(df) == 0:
+        return (df, features_df)
+    if not isinstance(df.index, pd.DatetimeIndex) or df.index.isna().any():
+        raise ValueError('Closed-candle filtering requires valid datetime timestamps.')
+    if features_df is not None and len(features_df) != len(df):
+        raise ValueError('Feature rows must align with candles before closed-candle filtering.')
+    now = pd.Timestamp(now) if now is not None else pd.Timestamp.now(tz='UTC')
+    if df.index.tz is None:
+        if now.tzinfo is not None:
+            now = now.tz_convert('UTC').tz_localize(None)
+    elif now.tzinfo is None:
+        now = now.tz_localize('UTC')
+    step = pd.Timedelta(minutes=MINUTES_PER_CANDLE[timeframe]) if timeframe else infer_step(df.index)
+    mask = df.index + step <= now
+    df = df.loc[mask]
+    if features_df is not None:
+        features_df = features_df.loc[mask].reset_index(drop=True)
+    return (df, features_df)
+
+def price_direction(predicted: float, current: float) -> str:
+    # Treat optimizer-scale numerical noise as flat. Without this guard a
+    # persistence forecast can become DOWN solely because exp/log round-trip
+    # error is a few ulps, which makes live consensus and accuracy tables look
+    # more certain than the underlying prices justify.
+    if not math.isfinite(float(predicted)) or not math.isfinite(float(current)):
+        raise ValueError('price_direction needs finite numeric prices.')
+    predicted, current = float(predicted), float(current)
+    if math.isclose(float(predicted), float(current), rel_tol=1e-9, abs_tol=1e-12):
+        return 'flat'
+    if predicted > current:
+        return 'up'
+    if predicted < current:
+        return 'down'
+    return 'flat'