Spaces:
Running on Zero
Running on Zero
deveshc23
feat: Add Free Gradio Space support, Git LFS tracking for data artifacts, and Quant Allocator
3e33e2e |
Download docs/frontend/FRONTEND_REDESIGN.md from deveshc23/drift-quant: direct link, hf CLI and curl.
- Browser
- Download file 19.6 kB
-
https://huggingface.co/spaces/deveshc23/drift-quant/resolve/main/docs/frontend/FRONTEND_REDESIGN.md
- Command line
-
hf download hf://spaces/deveshc23/drift-quant/docs/frontend/FRONTEND_REDESIGN.md
-
curl -L -o FRONTEND_REDESIGN.md https://huggingface.co/spaces/deveshc23/drift-quant/resolve/main/docs/frontend/FRONTEND_REDESIGN.md
19.6 kB
| # Frontend Redesign & Quantitative Research Workstation Architecture | |
| **Project:** Drift Quant β NSE Quantitative Research Lab | |
| **Document Version:** 1.0.0 | |
| **Status:** Approved for Implementation | |
| **Target Environment:** React 19 + TypeScript + Vite + Tailwind/Modern CSS + FastAPI Backend | |
| --- | |
| ## 1. Executive Summary & Audit Findings | |
| The initial version of Drift Quant proved the core data-integrity foundation: streaming inspection of a 260 MB ZIP archive (17,808,743 raw 15-minute observations across 488 symbols from 2019 to 2025), a verifiable audit pipeline, next-bar execution simulation, and four persisted holdout experiments. | |
| However, the user interface was a single-view, read-only dashboard displaying only the fixed 2024β2025 holdout period across 10 hardcoded symbols. It lacked interactivity, exploration capabilities, dynamic backtesting, and multi-faceted quantitative analytics. | |
| ### Why the Previous Dashboard Was Restricted | |
| 1. **Static Experiment Index:** `scripts/run_nse_holdout_backtests.py` ran once with fixed parameters (`START = 2024-01-01`, `END = 2025-07-15`, 10 symbols) and saved to `artifacts/experiments/index.json`. | |
| 2. **Missing Market Data Endpoints:** The API only had 5 endpoints reading static files in `artifacts/experiments/` and `dataset_audit.json`. There was no endpoint to retrieve historical OHLCV bars, search symbols, or stream date ranges. | |
| 3. **No Execution API:** Backtests could only be launched via CLI scripts; there was no API to parameterize or trigger simulations. | |
| 4. **Single Monolithic View:** All components were crammed into `App.tsx` with a single Recharts line chart and no page routing. | |
| ### Data Audit Verification: 2019β2025 vs. 2024β2025 Holdout | |
| The full dataset and holdout periods are strictly distinguished across the application: | |
| | Dimension | Full Archive Dataset | Fixed Holdout Experiment | | |
| | :--- | :--- | :--- | | |
| | **Start Date** | `2019-01-01 09:15:00` | `2024-01-01 00:00:00` | | |
| | **End Date** | `2025-07-14 11:00:00` | `2025-07-15 00:00:00` | | |
| | **Total Rows** | 17,808,743 observations | 93,008 valid observations | | |
| | **Symbol Coverage** | 488 NSE symbols | 10 selected large-cap symbols | | |
| | **Invalid OHLC** | 72 rows (excluded) | 2 rows (excluded) | | |
| | **Zero-Volume Bars** | 34,867 bars | Counted in symbol accounting | | |
| | **Intraday Gaps** | 4,894 unexpected gaps | Monitored in quality metrics | | |
| | **Purpose** | Full historical research universe | Out-of-sample holdout verification | | |
| --- | |
| ## 2. Visual Design System | |
| Drift Quant is redesigned as a **dark-first, high-density institutional quantitative research terminal**, inspired by Bloomberg Terminal, FactSet, and Koyfin. | |
| ### Design Principles | |
| 1. **Information Density with Breathing Room:** Compact tables, dense metric grids, and maximized chart canvas area. | |
| 2. **Semantic Color Palette:** | |
| - **Backgrounds:** Slate dark `#0a0d14` (root), `#111622` (surfaces/cards), `#171e2e` (hover/active), `#222b3d` (borders). | |
| - **Text:** High-contrast white `#f1f5f9` (headers, values), muted slate `#94a3b8` (labels, subtitles), faint `#64748b` (metadata). | |
| - **Returns / P&L:** Emerald `#10b981` (positive returns, profit), Rose/Crimson `#ef4444` (negative returns, drawdown), Cyan `#06b6d4` (benchmark/neutral), Amber `#f59e0b` (warnings, cost friction). | |
| 3. **Typography:** Monospace numbers (`JetBrains Mono`, `Fira Code`, or `ui-monospace`) for tabular financial figures with fixed-width glyph alignment; clean sans-serif (`Inter`, `system-ui`) for UI labels. | |
| 4. **Currency & Accounting Precision:** | |
| - All monetary figures explicitly display Indian Rupee (`βΉ`) symbols with Indian numbering formatting (`βΉ12.45L`, `βΉ1.50Cr`, or `βΉ1,24,500.00`). | |
| - Percentage returns (`+17.11%`, `-39.86%`) are strictly segregated from absolute currency amounts. | |
| - Basis points (`bps`) are explicitly labelled on costs and slippage. | |
| --- | |
| ## 3. Global Architecture & Navigation | |
| The application is structured into a persistent **collapsible left navigation rail**, a **compact global top bar**, and **ten dedicated analytical sections**: | |
| ```mermaid | |
| flowchart TD | |
| App[Drift Quant Workstation] --> TopBar[Global Header: System Status, Active Universe, Quick Stats] | |
| App --> NavRail[Collapsible Navigation Rail] | |
| App --> Content[Active Page Viewport] | |
| NavRail --> P1[1. Overview Cockpit] | |
| NavRail --> P2[2. Market Explorer] | |
| NavRail --> P3[3. Strategy Lab] | |
| NavRail --> P4[4. Backtest Results] | |
| NavRail --> P5[5. Experiment Comparison] | |
| NavRail --> P6[6. Portfolio Analytics] | |
| NavRail --> P7[7. Trade Explorer] | |
| NavRail --> P8[8. Data Quality & Coverage] | |
| NavRail --> P9[9. Research Reports] | |
| NavRail --> P10[10. System Health] | |
| ``` | |
| ### Page Breakdown | |
| ### 1. Overview Cockpit | |
| - **Firm-Level KPI Bar:** Total symbols (488), historical rows (17.8M), dataset span (2019β2025), completed experiments, best strategy by Sharpe, portfolio return, annualized volatility, max drawdown, total trades, total transaction costs. | |
| - **Interactive Multi-Strategy Equity Chart:** Compare net equity curves of all saved experiments on normalized or absolute scales. | |
| - **Drawdown Underwater Plot:** Synchronized drawdown curves highlighting maximum peak-to-trough drops and recovery durations. | |
| - **Performance by Period:** Monthly returns matrix and calendar heatmap. | |
| - **Strategy Leaderboard:** Sortable, filterable table of experiments with metric toggles. | |
| - **Data Coverage & Quality Timeline:** Visual observation distribution across years and symbols. | |
| - **Recent Experiments Feed:** Quick-access links with execution parameters and results. | |
| ### 2. Market Explorer | |
| - **Instrument Selector:** Full searchable catalog of 488 symbols with instant filtering by sector, row count, and data health status. | |
| - **Interactive Candlestick & Volume Chart:** High-performance OHLC candlestick chart with zoom/pan, crosshairs, time-resolution aggregations (15m, 1h, 1D), and indicator overlays: | |
| - Exponential Moving Average (EMA 9, 21, 50, 200). | |
| - Intraday VWAP Proxy (cumulative session typical price * volume). | |
| - Average True Range (ATR 14). | |
| - **Date Range Selector:** Presets (1M, 3M, 6M, 1Y, 3Y, 5Y, Full 2019β2025) and custom date picker. | |
| - **Historical Observations Table:** Paginated raw bar records with timestamp, open, high, low, close, volume, and CSV export. | |
| - **Data Quality Drawer:** Specific symbol statistics (zero-volume bars, invalid OHLC, intraday gaps). | |
| ### 3. Strategy Lab (Interactive Backtest Configurator) | |
| - **Strategy Catalog:** Detailed documentation for each registered strategy: | |
| - `buy_and_hold_baseline` | |
| - `intraday_ema_momentum` | |
| - `vwap_proxy_reversion` | |
| - `atr_trend` | |
| - `sector_relative_pairs` *(New)* | |
| - `sector_cross_momentum` *(New)* | |
| - **Configuration Controls:** | |
| - Date ranges (with explicit validation against dataset availability). | |
| - Universe selector (Holdout 10, Sector Baskets, or Custom symbol selection). | |
| - Initial capital (default βΉ10,00,000), position sizing rule, max gross exposure. | |
| - Transaction cost assumptions (default 10.0 bps) and slippage assumptions (default 5.0 bps). | |
| - Strategy-specific parameters (fast/slow periods, deviation thresholds, ATR multiples). | |
| - **Execution Engine:** Local async job queue with real-time progress, status transitions (Queued β Running β Completed / Failed), and instant link to generated Backtest Results. | |
| ### 4. Backtest Results | |
| - **Summary Header:** Strategy version, dataset hash, simulation dates, capital, total net return, Sharpe, Sortino, max drawdown, win rate, profit factor, average win/loss, average holding time. | |
| - **Net & Gross Equity Curves:** Interactive chart showing portfolio equity before and after transaction costs. | |
| - **Drawdown Analysis:** Depth, duration, and time spent underwater. | |
| - **Monthly & Yearly Heatmap:** Return breakdown by year and month. | |
| - **Trade Analysis & Exit Reason Breakdown:** Donut/bar charts of win vs. loss, exits by rule vs. trailing stop vs. end-of-data liquidation. | |
| - **Symbol Attribution:** Contribution by symbol (best and worst performing assets in the basket). | |
| - **Configuration & Artifact Provenance:** Complete auditable JSON snapshot, command line invocation, and output file paths. | |
| ### 5. Experiment Comparison | |
| - **Multi-Experiment Selector:** Select 2 to 6 experiments for side-by-side evaluation. | |
| - **Normalized Cumulative Returns:** Base 100 or percentage return plot over common timeline. | |
| - **Comparative Metrics Table:** Side-by-side matrix of return, Sharpe, Sortino, volatility, drawdown, trade count, turnover, and fees. | |
| - **Configuration Difference Matrix:** Visual diff highlighting discrepancies in universe, dates, costs, or execution rules, with prominent incomparability warnings. | |
| ### 6. Portfolio & Risk Analytics | |
| - **Asset Allocation & Weights:** Stacked area or time-series weight chart showing cash vs. equity allocation. | |
| - **Gross & Net Exposure:** Leverage and cash buffer utilization over time. | |
| - **Sector Concentration:** Exposure broken down by industry (Banking, IT, Energy, FMCG, Auto, etc.). | |
| - **Rolling Volatility:** 30-day and 60-day annualized rolling volatility curves. | |
| - **Correlation Matrix:** Inter-asset return correlation heatmap for active universe. | |
| ### 7. Trade Explorer | |
| - **Comprehensive Ledger:** Full trade history across selected experiment. | |
| - **Filters:** Symbol search, date range, direction (long), outcome (profitable vs. unprofitable), exit reason. | |
| - **Summary Metrics:** Total trades, total gross P&L, total transaction costs, net P&L, profit factor. | |
| - **Trade Inspection Modal:** Detailed bar-level breakdown of the trade entry and exit with price context. | |
| - **CSV Export:** Direct download of the filtered trade ledger. | |
| ### 8. Data Quality & Coverage | |
| - **Firm-Level Dataset Audit:** Total rows (17.8M), unique symbols (488), start/end dates. | |
| - **Data Integrity Audit:** | |
| - 72 Invalid OHLC records (detailed explanation of failure criteria). | |
| - 34,867 Zero-volume bars. | |
| - 4,894 Intraday unexpected gaps. | |
| - 0 Duplicate timestamps / regressions. | |
| - **Yearly Observation Bar Chart:** Bar volume by calendar year (2019 to 2025). | |
| - **Per-Symbol Audit Table:** Filterable by symbol with individual gap counts, zero-volume counts, and bar counts. | |
| - **Material Disclosures:** Transparent disclosure regarding absence of confirmed corporate actions, split/dividend adjustments, and historical NSE 500 survivorship bias. | |
| ### 9. Research Reports | |
| - **Methodology Documentation:** Explicit explanation of next-bar execution rules, fee assumptions, and annualization factors. | |
| - **Validation Archive:** Saved reports from initial and subsequent holdout runs. | |
| - **Bias Analysis:** Overfitting warnings, multiple hypothesis testing disclosures. | |
| ### 10. System Health & Diagnostics | |
| - **Backend Service Health:** Uptime, memory usage, API latency, artifact repository path. | |
| - **Market Data Storage Health:** Status of SQLite cache, JSON caches, and ZIP member accessibility. | |
| - **Active Job Monitor:** Current backtest worker state and execution log viewer. | |
| --- | |
| ## 4. Better Strategy Design: Sector-Relative Quantitative Strategies | |
| The existing holdout experiments revealed severe weaknesses in naive intraday momentum, mean-reversion, and breakout strategies on 15-minute bars: | |
| - `intraday_ema_momentum`: -39.86% return, 1,768 trades, βΉ3.32L costs (churned by intraday noise). | |
| - `vwap_proxy_reversion`: -70.58% return, 2,518 trades, βΉ4.73L costs (caught in trending intraday moves). | |
| - `atr_trend`: -23.30% return, 1,293 trades, βΉ2.43L costs (stop-outs and friction). | |
| With 15 bps per side (30 bps roundtrip), trading 2,000 times destroys 60% of capital purely in transaction friction! | |
| ### Solution 1: Sector-Relative Pairs Trading (`sector_pairs_reversion`) | |
| - **Premise:** Indian equities in the same sector share fundamental macroeconomic drivers, interest rate sensitivity, and sector rotation flows. | |
| - **Pair Baskets:** | |
| - Banking: `HDFCBANK` vs `ICICIBANK`, `AXISBANK` vs `SBIN`, `KOTAKBANK` vs `ICICIBANK`. | |
| - Technology: `TCS` vs `INFY`, `WIPRO` vs `HCLTECH`. | |
| - FMCG: `ITC` vs `HINDUNILVR`. | |
| - Energy: `RELIANCE` vs `ONGC`. | |
| - **Mechanism:** Computes rolling log price spread $S_t = \ln(P_A) - \ln(P_B)$ over a lookback window (e.g. 80 bars = ~2 trading days). Standardizes to Z-score: | |
| $$Z_t = \frac{S_t - \mu_S}{\sigma_S}$$ | |
| - **Execution Rules (Long-Only):** | |
| - When $Z_t < -1.75$: Stock A is significantly undervalued relative to Stock B. Enter long Stock A. | |
| - When $Z_t > +1.75$: Stock B is significantly undervalued relative to Stock A. Enter long Stock B. | |
| - Exit when spread normalizes ($|Z_t| < 0.25$) or on adverse divergence stop ($|Z_t| > 3.5$). | |
| - **Turnover Control:** Minimum holding time filter (prevents rapid intraday switching), slashing transaction costs by 80%. | |
| ### Solution 2: Sector Cross-Sectional Momentum with Regime Filter (`sector_cross_momentum`) | |
| - **Premise:** Sector leaders exhibit persistence over multi-day horizons. | |
| - **Mechanism:** For each sector group, ranks constituent stocks by their intermediate-term returns (e.g. 50 bars = 1.5 days). Long only the top-ranked stock if the sector index is above its 100-bar moving average. | |
| - **Turnover Control:** Rebalancing only occurs when rank shifts persist for at least 8 bars. | |
| --- | |
| ## 5. API Data Contracts & Endpoint Specifications | |
| To power the workstation, the following backend endpoints are added to `src/drift_quant/api/app.py`: | |
| ``` | |
| GET /health -> System status, cache state, experiment count | |
| GET /api/v1/dataset/summary -> Firm-level dataset summary (17.8M rows, 488 symbols) | |
| GET /api/v1/dataset/symbols -> List of all 488 symbols with date coverage, row counts, sectors, data quality | |
| GET /api/v1/dataset/coverage-timeline -> Yearly and monthly observation counts | |
| GET /api/v1/market/bars -> Historical OHLCV bars (?symbol=RELIANCE&start=...&end=...&resolution=15m|1h|1d) | |
| GET /api/v1/market/indicators -> Computed indicators (EMA, VWAP, ATR) for a symbol | |
| GET /api/v1/strategies -> Catalog of registered strategies with JSON schemas | |
| POST /api/v1/backtest/run -> Submit backtest job (validated strategy & parameters) | |
| GET /api/v1/jobs/{job_id} -> Query job status (queued, running, completed, failed) | |
| GET /api/v1/experiments -> List persisted experiments with summary metrics | |
| GET /api/v1/experiments/{id} -> Full experiment detail & configuration | |
| GET /api/v1/experiments/{id}/equity -> Downsampled or full net/gross equity series | |
| GET /api/v1/experiments/{id}/drawdown -> Drawdown series with peak, trough, duration | |
| GET /api/v1/experiments/{id}/trades -> Filterable, paginated trade ledger | |
| GET /api/v1/experiments/{id}/attribution -> Per-symbol P&L attribution and win rates | |
| GET /api/v1/experiments/{id}/monthly -> Monthly and yearly return matrix | |
| GET /api/v1/experiments/{id}/risk -> Advanced risk metrics (VaR, Sortino, exposure) | |
| GET /api/v1/experiments/compare -> Comparative multi-experiment summary | |
| GET /api/v1/reports/{id}/export-csv -> CSV download for trades or equity curve | |
| ``` | |
| --- | |
| ## 6. Frontend Component Hierarchy | |
| ```text | |
| frontend/src/ | |
| βββ api/ | |
| β βββ client.ts # Strongly-typed fetch client with error handling & cancellation | |
| β βββ types.ts # Full TypeScript models matching backend schemas | |
| βββ components/ | |
| β βββ common/ | |
| β β βββ Badge.tsx # Tone badges (positive, negative, warning, neutral) | |
| β β βββ Button.tsx # Styled buttons (primary, secondary, ghost, danger) | |
| β β βββ Card.tsx # Analytical panel containers with titles & actions | |
| β β βββ DatePicker.tsx # Date range selector with presets | |
| β β βββ EmptyState.tsx # Informative empty states with guidance | |
| β β βββ ErrorState.tsx # Actionable error banners | |
| β β βββ KpiCard.tsx # Standardized financial KPI card with tone & format | |
| β β βββ LoadingSkeleton.tsx # Skeleton loaders for charts, cards, and tables | |
| β β βββ Modal.tsx # Accessible overlay dialog | |
| β β βββ Tooltip.tsx # Contextual financial definitions | |
| β βββ charts/ | |
| β β βββ CandlestickChart.tsx # High-density OHLC candlestick + volume + overlays | |
| β β βββ EquityChart.tsx # Cumulative equity curve (net vs gross vs benchmark) | |
| β β βββ DrawdownChart.tsx # Underwater drawdown curve | |
| β β βββ MonthlyHeatmap.tsx # Calendar month/year return matrix | |
| β β βββ AllocationChart.tsx # Exposure & position weights over time | |
| β βββ layout/ | |
| β β βββ TopBar.tsx # Global header with system health & active universe | |
| β β βββ Sidebar.tsx # Collapsible navigation rail with active route | |
| β β βββ Shell.tsx # Master application shell | |
| βββ pages/ | |
| β βββ OverviewPage.tsx # Firm research overview cockpit | |
| β βββ MarketExplorerPage.tsx # Deep 2019-2025 market data explorer & candles | |
| β βββ StrategyLabPage.tsx # Interactive backtest configurator & job runner | |
| β βββ BacktestResultsPage.tsx # In-depth single experiment analytics | |
| β βββ ExperimentComparePage.tsx # Multi-experiment side-by-side comparison | |
| β βββ PortfolioPage.tsx # Portfolio allocation, concentration & risk | |
| β βββ TradeExplorerPage.tsx # Filterable trade ledger & trade modal | |
| β βββ DataQualityPage.tsx # Dataset audit, gaps, zero-volume records | |
| β βββ ResearchReportsPage.tsx # Methodology & validation documentation | |
| β βββ SystemHealthPage.tsx # API health, SQLite cache status, job queue | |
| βββ utils/ | |
| β βββ formatters.ts # Currency (βΉ INR), percentages, numbers, dates | |
| β βββ math.ts # Downsampling, rolling metrics, attribution | |
| βββ App.tsx # Main router and global context provider | |
| βββ main.tsx # React entry point | |
| βββ styles.css # Polished design system tokens and styling | |
| ``` | |
| --- | |
| ## 7. Implementation Roadmap | |
| 1. **Step 1:** Build market data cache (SQLite + JSON) and verify full 2019β2025 coverage across liquid sectors. | |
| 2. **Step 2:** Implement the new `sector_pairs_reversion` and `sector_cross_momentum` strategies in `src/drift_quant/backtesting/strategies.py` and register them. | |
| 3. **Step 3:** Extend backend FastAPI endpoints (`app.py`, `repository.py`) to serve market bars, symbol catalog, backtest execution runner, attribution, and risk. | |
| 4. **Step 4:** Write comprehensive backend tests for new endpoints, strategy algorithms, and caching logic. | |
| 5. **Step 5:** Build the frontend API client layer (`api/client.ts`, `api/types.ts`) and formatting utilities with complete INR (βΉ) conventions. | |
| 6. **Step 6:** Construct the workstation UI components (Sidebar, TopBar, KpiCards, Financial Charts, Data Tables). | |
| 7. **Step 7:** Implement the 10 dedicated workstation pages and connect to live backend data. | |
| 8. **Step 8:** Optimize build bundles with dynamic code-splitting and evaluate bundle size warnings. | |
| 9. **Step 9:** Execute full frontend tests and verification, run backend test suite, and document results. | |
| 10. **Step 10:** Update `docs/frontend/FRONTEND_GUIDE.md`, root `README.md`, and API reference. | |