drift-quant / docs /frontend /FRONTEND_REDESIGN.md
deveshc23
feat: Add Free Gradio Space support, Git LFS tracking for data artifacts, and Quant Allocator
3e33e2e
|
Raw History Blame Contribute Delete
19.6 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade

Frontend Redesign & Quantitative Research Workstation Architecture

Project: Drift Quant β€” NSE Quantitative Research Lab
Document Version: 1.0.0
Status: Approved for Implementation
Target Environment: React 19 + TypeScript + Vite + Tailwind/Modern CSS + FastAPI Backend


1. Executive Summary & Audit Findings

The initial version of Drift Quant proved the core data-integrity foundation: streaming inspection of a 260 MB ZIP archive (17,808,743 raw 15-minute observations across 488 symbols from 2019 to 2025), a verifiable audit pipeline, next-bar execution simulation, and four persisted holdout experiments.

However, the user interface was a single-view, read-only dashboard displaying only the fixed 2024–2025 holdout period across 10 hardcoded symbols. It lacked interactivity, exploration capabilities, dynamic backtesting, and multi-faceted quantitative analytics.

Why the Previous Dashboard Was Restricted

  1. Static Experiment Index: scripts/run_nse_holdout_backtests.py ran once with fixed parameters (START = 2024-01-01, END = 2025-07-15, 10 symbols) and saved to artifacts/experiments/index.json.
  2. Missing Market Data Endpoints: The API only had 5 endpoints reading static files in artifacts/experiments/ and dataset_audit.json. There was no endpoint to retrieve historical OHLCV bars, search symbols, or stream date ranges.
  3. No Execution API: Backtests could only be launched via CLI scripts; there was no API to parameterize or trigger simulations.
  4. Single Monolithic View: All components were crammed into App.tsx with a single Recharts line chart and no page routing.

Data Audit Verification: 2019–2025 vs. 2024–2025 Holdout

The full dataset and holdout periods are strictly distinguished across the application:

Dimension Full Archive Dataset Fixed Holdout Experiment
Start Date 2019-01-01 09:15:00 2024-01-01 00:00:00
End Date 2025-07-14 11:00:00 2025-07-15 00:00:00
Total Rows 17,808,743 observations 93,008 valid observations
Symbol Coverage 488 NSE symbols 10 selected large-cap symbols
Invalid OHLC 72 rows (excluded) 2 rows (excluded)
Zero-Volume Bars 34,867 bars Counted in symbol accounting
Intraday Gaps 4,894 unexpected gaps Monitored in quality metrics
Purpose Full historical research universe Out-of-sample holdout verification

2. Visual Design System

Drift Quant is redesigned as a dark-first, high-density institutional quantitative research terminal, inspired by Bloomberg Terminal, FactSet, and Koyfin.

Design Principles

  1. Information Density with Breathing Room: Compact tables, dense metric grids, and maximized chart canvas area.
  2. Semantic Color Palette:
    • Backgrounds: Slate dark #0a0d14 (root), #111622 (surfaces/cards), #171e2e (hover/active), #222b3d (borders).
    • Text: High-contrast white #f1f5f9 (headers, values), muted slate #94a3b8 (labels, subtitles), faint #64748b (metadata).
    • Returns / P&L: Emerald #10b981 (positive returns, profit), Rose/Crimson #ef4444 (negative returns, drawdown), Cyan #06b6d4 (benchmark/neutral), Amber #f59e0b (warnings, cost friction).
  3. Typography: Monospace numbers (JetBrains Mono, Fira Code, or ui-monospace) for tabular financial figures with fixed-width glyph alignment; clean sans-serif (Inter, system-ui) for UI labels.
  4. Currency & Accounting Precision:
    • All monetary figures explicitly display Indian Rupee (β‚Ή) symbols with Indian numbering formatting (β‚Ή12.45L, β‚Ή1.50Cr, or β‚Ή1,24,500.00).
    • Percentage returns (+17.11%, -39.86%) are strictly segregated from absolute currency amounts.
    • Basis points (bps) are explicitly labelled on costs and slippage.

3. Global Architecture & Navigation

The application is structured into a persistent collapsible left navigation rail, a compact global top bar, and ten dedicated analytical sections:

flowchart TD
    App[Drift Quant Workstation] --> TopBar[Global Header: System Status, Active Universe, Quick Stats]
    App --> NavRail[Collapsible Navigation Rail]
    App --> Content[Active Page Viewport]
    
    NavRail --> P1[1. Overview Cockpit]
    NavRail --> P2[2. Market Explorer]
    NavRail --> P3[3. Strategy Lab]
    NavRail --> P4[4. Backtest Results]
    NavRail --> P5[5. Experiment Comparison]
    NavRail --> P6[6. Portfolio Analytics]
    NavRail --> P7[7. Trade Explorer]
    NavRail --> P8[8. Data Quality & Coverage]
    NavRail --> P9[9. Research Reports]
    NavRail --> P10[10. System Health]

Page Breakdown

1. Overview Cockpit

  • Firm-Level KPI Bar: Total symbols (488), historical rows (17.8M), dataset span (2019–2025), completed experiments, best strategy by Sharpe, portfolio return, annualized volatility, max drawdown, total trades, total transaction costs.
  • Interactive Multi-Strategy Equity Chart: Compare net equity curves of all saved experiments on normalized or absolute scales.
  • Drawdown Underwater Plot: Synchronized drawdown curves highlighting maximum peak-to-trough drops and recovery durations.
  • Performance by Period: Monthly returns matrix and calendar heatmap.
  • Strategy Leaderboard: Sortable, filterable table of experiments with metric toggles.
  • Data Coverage & Quality Timeline: Visual observation distribution across years and symbols.
  • Recent Experiments Feed: Quick-access links with execution parameters and results.

2. Market Explorer

  • Instrument Selector: Full searchable catalog of 488 symbols with instant filtering by sector, row count, and data health status.
  • Interactive Candlestick & Volume Chart: High-performance OHLC candlestick chart with zoom/pan, crosshairs, time-resolution aggregations (15m, 1h, 1D), and indicator overlays:
    • Exponential Moving Average (EMA 9, 21, 50, 200).
    • Intraday VWAP Proxy (cumulative session typical price * volume).
    • Average True Range (ATR 14).
  • Date Range Selector: Presets (1M, 3M, 6M, 1Y, 3Y, 5Y, Full 2019–2025) and custom date picker.
  • Historical Observations Table: Paginated raw bar records with timestamp, open, high, low, close, volume, and CSV export.
  • Data Quality Drawer: Specific symbol statistics (zero-volume bars, invalid OHLC, intraday gaps).

3. Strategy Lab (Interactive Backtest Configurator)

  • Strategy Catalog: Detailed documentation for each registered strategy:
    • buy_and_hold_baseline
    • intraday_ema_momentum
    • vwap_proxy_reversion
    • atr_trend
    • sector_relative_pairs (New)
    • sector_cross_momentum (New)
  • Configuration Controls:
    • Date ranges (with explicit validation against dataset availability).
    • Universe selector (Holdout 10, Sector Baskets, or Custom symbol selection).
    • Initial capital (default β‚Ή10,00,000), position sizing rule, max gross exposure.
    • Transaction cost assumptions (default 10.0 bps) and slippage assumptions (default 5.0 bps).
    • Strategy-specific parameters (fast/slow periods, deviation thresholds, ATR multiples).
  • Execution Engine: Local async job queue with real-time progress, status transitions (Queued β†’ Running β†’ Completed / Failed), and instant link to generated Backtest Results.

4. Backtest Results

  • Summary Header: Strategy version, dataset hash, simulation dates, capital, total net return, Sharpe, Sortino, max drawdown, win rate, profit factor, average win/loss, average holding time.
  • Net & Gross Equity Curves: Interactive chart showing portfolio equity before and after transaction costs.
  • Drawdown Analysis: Depth, duration, and time spent underwater.
  • Monthly & Yearly Heatmap: Return breakdown by year and month.
  • Trade Analysis & Exit Reason Breakdown: Donut/bar charts of win vs. loss, exits by rule vs. trailing stop vs. end-of-data liquidation.
  • Symbol Attribution: Contribution by symbol (best and worst performing assets in the basket).
  • Configuration & Artifact Provenance: Complete auditable JSON snapshot, command line invocation, and output file paths.

5. Experiment Comparison

  • Multi-Experiment Selector: Select 2 to 6 experiments for side-by-side evaluation.
  • Normalized Cumulative Returns: Base 100 or percentage return plot over common timeline.
  • Comparative Metrics Table: Side-by-side matrix of return, Sharpe, Sortino, volatility, drawdown, trade count, turnover, and fees.
  • Configuration Difference Matrix: Visual diff highlighting discrepancies in universe, dates, costs, or execution rules, with prominent incomparability warnings.

6. Portfolio & Risk Analytics

  • Asset Allocation & Weights: Stacked area or time-series weight chart showing cash vs. equity allocation.
  • Gross & Net Exposure: Leverage and cash buffer utilization over time.
  • Sector Concentration: Exposure broken down by industry (Banking, IT, Energy, FMCG, Auto, etc.).
  • Rolling Volatility: 30-day and 60-day annualized rolling volatility curves.
  • Correlation Matrix: Inter-asset return correlation heatmap for active universe.

7. Trade Explorer

  • Comprehensive Ledger: Full trade history across selected experiment.
  • Filters: Symbol search, date range, direction (long), outcome (profitable vs. unprofitable), exit reason.
  • Summary Metrics: Total trades, total gross P&L, total transaction costs, net P&L, profit factor.
  • Trade Inspection Modal: Detailed bar-level breakdown of the trade entry and exit with price context.
  • CSV Export: Direct download of the filtered trade ledger.

8. Data Quality & Coverage

  • Firm-Level Dataset Audit: Total rows (17.8M), unique symbols (488), start/end dates.
  • Data Integrity Audit:
    • 72 Invalid OHLC records (detailed explanation of failure criteria).
    • 34,867 Zero-volume bars.
    • 4,894 Intraday unexpected gaps.
    • 0 Duplicate timestamps / regressions.
  • Yearly Observation Bar Chart: Bar volume by calendar year (2019 to 2025).
  • Per-Symbol Audit Table: Filterable by symbol with individual gap counts, zero-volume counts, and bar counts.
  • Material Disclosures: Transparent disclosure regarding absence of confirmed corporate actions, split/dividend adjustments, and historical NSE 500 survivorship bias.

9. Research Reports

  • Methodology Documentation: Explicit explanation of next-bar execution rules, fee assumptions, and annualization factors.
  • Validation Archive: Saved reports from initial and subsequent holdout runs.
  • Bias Analysis: Overfitting warnings, multiple hypothesis testing disclosures.

10. System Health & Diagnostics

  • Backend Service Health: Uptime, memory usage, API latency, artifact repository path.
  • Market Data Storage Health: Status of SQLite cache, JSON caches, and ZIP member accessibility.
  • Active Job Monitor: Current backtest worker state and execution log viewer.

4. Better Strategy Design: Sector-Relative Quantitative Strategies

The existing holdout experiments revealed severe weaknesses in naive intraday momentum, mean-reversion, and breakout strategies on 15-minute bars:

  • intraday_ema_momentum: -39.86% return, 1,768 trades, β‚Ή3.32L costs (churned by intraday noise).
  • vwap_proxy_reversion: -70.58% return, 2,518 trades, β‚Ή4.73L costs (caught in trending intraday moves).
  • atr_trend: -23.30% return, 1,293 trades, β‚Ή2.43L costs (stop-outs and friction).

With 15 bps per side (30 bps roundtrip), trading 2,000 times destroys 60% of capital purely in transaction friction!

Solution 1: Sector-Relative Pairs Trading (sector_pairs_reversion)

  • Premise: Indian equities in the same sector share fundamental macroeconomic drivers, interest rate sensitivity, and sector rotation flows.
  • Pair Baskets:
    • Banking: HDFCBANK vs ICICIBANK, AXISBANK vs SBIN, KOTAKBANK vs ICICIBANK.
    • Technology: TCS vs INFY, WIPRO vs HCLTECH.
    • FMCG: ITC vs HINDUNILVR.
    • Energy: RELIANCE vs ONGC.
  • Mechanism: Computes rolling log price spread $S_t = \ln(P_A) - \ln(P_B)$ over a lookback window (e.g. 80 bars = ~2 trading days). Standardizes to Z-score: $$Z_t = \frac{S_t - \mu_S}{\sigma_S}$$
  • Execution Rules (Long-Only):
    • When $Z_t < -1.75$: Stock A is significantly undervalued relative to Stock B. Enter long Stock A.
    • When $Z_t > +1.75$: Stock B is significantly undervalued relative to Stock A. Enter long Stock B.
    • Exit when spread normalizes ($|Z_t| < 0.25$) or on adverse divergence stop ($|Z_t| > 3.5$).
  • Turnover Control: Minimum holding time filter (prevents rapid intraday switching), slashing transaction costs by 80%.

Solution 2: Sector Cross-Sectional Momentum with Regime Filter (sector_cross_momentum)

  • Premise: Sector leaders exhibit persistence over multi-day horizons.
  • Mechanism: For each sector group, ranks constituent stocks by their intermediate-term returns (e.g. 50 bars = 1.5 days). Long only the top-ranked stock if the sector index is above its 100-bar moving average.
  • Turnover Control: Rebalancing only occurs when rank shifts persist for at least 8 bars.

5. API Data Contracts & Endpoint Specifications

To power the workstation, the following backend endpoints are added to src/drift_quant/api/app.py:

GET  /health                                -> System status, cache state, experiment count
GET  /api/v1/dataset/summary                -> Firm-level dataset summary (17.8M rows, 488 symbols)
GET  /api/v1/dataset/symbols                -> List of all 488 symbols with date coverage, row counts, sectors, data quality
GET  /api/v1/dataset/coverage-timeline      -> Yearly and monthly observation counts
GET  /api/v1/market/bars                    -> Historical OHLCV bars (?symbol=RELIANCE&start=...&end=...&resolution=15m|1h|1d)
GET  /api/v1/market/indicators              -> Computed indicators (EMA, VWAP, ATR) for a symbol
GET  /api/v1/strategies                     -> Catalog of registered strategies with JSON schemas
POST /api/v1/backtest/run                   -> Submit backtest job (validated strategy & parameters)
GET  /api/v1/jobs/{job_id}                  -> Query job status (queued, running, completed, failed)
GET  /api/v1/experiments                    -> List persisted experiments with summary metrics
GET  /api/v1/experiments/{id}               -> Full experiment detail & configuration
GET  /api/v1/experiments/{id}/equity        -> Downsampled or full net/gross equity series
GET  /api/v1/experiments/{id}/drawdown      -> Drawdown series with peak, trough, duration
GET  /api/v1/experiments/{id}/trades        -> Filterable, paginated trade ledger
GET  /api/v1/experiments/{id}/attribution   -> Per-symbol P&L attribution and win rates
GET  /api/v1/experiments/{id}/monthly       -> Monthly and yearly return matrix
GET  /api/v1/experiments/{id}/risk          -> Advanced risk metrics (VaR, Sortino, exposure)
GET  /api/v1/experiments/compare            -> Comparative multi-experiment summary
GET  /api/v1/reports/{id}/export-csv        -> CSV download for trades or equity curve

6. Frontend Component Hierarchy

frontend/src/
β”œβ”€β”€ api/
β”‚   β”œβ”€β”€ client.ts             # Strongly-typed fetch client with error handling & cancellation
β”‚   └── types.ts              # Full TypeScript models matching backend schemas
β”œβ”€β”€ components/
β”‚   β”œβ”€β”€ common/
β”‚   β”‚   β”œβ”€β”€ Badge.tsx         # Tone badges (positive, negative, warning, neutral)
β”‚   β”‚   β”œβ”€β”€ Button.tsx        # Styled buttons (primary, secondary, ghost, danger)
β”‚   β”‚   β”œβ”€β”€ Card.tsx          # Analytical panel containers with titles & actions
β”‚   β”‚   β”œβ”€β”€ DatePicker.tsx    # Date range selector with presets
β”‚   β”‚   β”œβ”€β”€ EmptyState.tsx    # Informative empty states with guidance
β”‚   β”‚   β”œβ”€β”€ ErrorState.tsx    # Actionable error banners
β”‚   β”‚   β”œβ”€β”€ KpiCard.tsx       # Standardized financial KPI card with tone & format
β”‚   β”‚   β”œβ”€β”€ LoadingSkeleton.tsx # Skeleton loaders for charts, cards, and tables
β”‚   β”‚   β”œβ”€β”€ Modal.tsx         # Accessible overlay dialog
β”‚   β”‚   └── Tooltip.tsx       # Contextual financial definitions
β”‚   β”œβ”€β”€ charts/
β”‚   β”‚   β”œβ”€β”€ CandlestickChart.tsx # High-density OHLC candlestick + volume + overlays
β”‚   β”‚   β”œβ”€β”€ EquityChart.tsx   # Cumulative equity curve (net vs gross vs benchmark)
β”‚   β”‚   β”œβ”€β”€ DrawdownChart.tsx # Underwater drawdown curve
β”‚   β”‚   β”œβ”€β”€ MonthlyHeatmap.tsx # Calendar month/year return matrix
β”‚   β”‚   └── AllocationChart.tsx # Exposure & position weights over time
β”‚   β”œβ”€β”€ layout/
β”‚   β”‚   β”œβ”€β”€ TopBar.tsx        # Global header with system health & active universe
β”‚   β”‚   β”œβ”€β”€ Sidebar.tsx       # Collapsible navigation rail with active route
β”‚   β”‚   └── Shell.tsx         # Master application shell
β”œβ”€β”€ pages/
β”‚   β”œβ”€β”€ OverviewPage.tsx      # Firm research overview cockpit
β”‚   β”œβ”€β”€ MarketExplorerPage.tsx # Deep 2019-2025 market data explorer & candles
β”‚   β”œβ”€β”€ StrategyLabPage.tsx   # Interactive backtest configurator & job runner
β”‚   β”œβ”€β”€ BacktestResultsPage.tsx # In-depth single experiment analytics
β”‚   β”œβ”€β”€ ExperimentComparePage.tsx # Multi-experiment side-by-side comparison
β”‚   β”œβ”€β”€ PortfolioPage.tsx     # Portfolio allocation, concentration & risk
β”‚   β”œβ”€β”€ TradeExplorerPage.tsx # Filterable trade ledger & trade modal
β”‚   β”œβ”€β”€ DataQualityPage.tsx   # Dataset audit, gaps, zero-volume records
β”‚   β”œβ”€β”€ ResearchReportsPage.tsx # Methodology & validation documentation
β”‚   └── SystemHealthPage.tsx  # API health, SQLite cache status, job queue
β”œβ”€β”€ utils/
β”‚   β”œβ”€β”€ formatters.ts         # Currency (β‚Ή INR), percentages, numbers, dates
β”‚   └── math.ts               # Downsampling, rolling metrics, attribution
β”œβ”€β”€ App.tsx                   # Main router and global context provider
β”œβ”€β”€ main.tsx                  # React entry point
└── styles.css                # Polished design system tokens and styling

7. Implementation Roadmap

  1. Step 1: Build market data cache (SQLite + JSON) and verify full 2019–2025 coverage across liquid sectors.
  2. Step 2: Implement the new sector_pairs_reversion and sector_cross_momentum strategies in src/drift_quant/backtesting/strategies.py and register them.
  3. Step 3: Extend backend FastAPI endpoints (app.py, repository.py) to serve market bars, symbol catalog, backtest execution runner, attribution, and risk.
  4. Step 4: Write comprehensive backend tests for new endpoints, strategy algorithms, and caching logic.
  5. Step 5: Build the frontend API client layer (api/client.ts, api/types.ts) and formatting utilities with complete INR (β‚Ή) conventions.
  6. Step 6: Construct the workstation UI components (Sidebar, TopBar, KpiCards, Financial Charts, Data Tables).
  7. Step 7: Implement the 10 dedicated workstation pages and connect to live backend data.
  8. Step 8: Optimize build bundles with dynamic code-splitting and evaluate bundle size warnings.
  9. Step 9: Execute full frontend tests and verification, run backend test suite, and document results.
  10. Step 10: Update docs/frontend/FRONTEND_GUIDE.md, root README.md, and API reference.