--- language: - en license: apache-2.0 library_name: pytorch tags: - time-series - financial-forecasting - ohlcv - conformer - foundation-model - 1b-parameters - quantitative-finance - algorithmic-trading - market-microstructure - deep-learning datasets: - mito0o852/OHLCV-1m metrics: - directional-accuracy - huber-loss pipeline_tag: time-series-forecasting ---
# ⚡ ULTRON // 1.189B Parameter Financial Conformer ### A Foundation Architecture for Multi-Horizon Financial Microstructure Forecasting **Engineered & Architected by Suryaansh Prithvijit Singh ([@Ultron28](https://huggingface.co/Ultron28) / [@Ultron09](https://github.com/Ultron09))** [![GitHub Repository](https://img.shields.io/badge/GitHub-ohlcv--transformer-blue?logo=github)](https://github.com/Ultron09/ohlcv-transformer) [![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-1.189B%20Checkpoint-yellow)](https://huggingface.co/Ultron28/ohlcv-transformer) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Ultron09/ohlcv-transformer/blob/main/OHLCV_Transformer_Colab.ipynb) [![Parameters](https://img.shields.io/badge/Parameters-1%2C189%2C135%2C740%20(1.189B)-crimson)](#-model-architecture-specifications) [![Context Window](https://img.shields.io/badge/Context-2%2C048%20Minutes%20(~34h)-cyan)](#-mathematical-formulation) [![Hardware](https://img.shields.io/badge/Compute-NVIDIA%20A100--SXM4--80GB-green)](https://huggingface.co/Ultron28/ohlcv-transformer)
--- ## 🏛️ Executive Summary The **ULTRON Financial Conformer (1.189B)** is a large-scale foundation model specifically designed to ingest raw minute-level OHLCV candlestick time series across liquid US equities and generate multi-horizon continuous returns and directional probability forecasts. Unlike conventional time-series transformers (e.g. PatchTST, Chronos, TimesFM) that treat financial candlesticks as 1D scalar sequences, this architecture treats the microstructure as a continuous multi-scale physical manifold. It combines: 1. **6-Scale Temporal Inception Tokenizer**: Continuous 1D convolutions spanning $[3\text{m}, 7\text{m}, 15\text{m}, 31\text{m}, 63\text{m}, 127\text{m}]$ overlapping windows. 2. **2,048-Minute (~34.1 Trading Hours) Continuous Context Window**. 3. **20 Deep Conformer Blocks**: Interleaving Rotary Position-Embedded (RoPE) Grouped-Query Attention (GQA) with depthwise separable convolution momentum filters and SwiGLU feed-forward networks. 4. **Multi-Horizon Joint Dual Head**: Co-optimizing smooth Huber expected return vectors alongside calibrated directional binary cross-entropy across $[1\text{m}, 3\text{m}, 5\text{m}, 10\text{m}, 15\text{m}, 30\text{m}, 60\text{m}]$ forward horizons. --- ## 📐 Model Architecture Specifications | Hyperparameter | Value | Description | | :--- | :--- | :--- | | **Total Parameters** | **1,189,135,740** | **1.189 Billion Parameters** | | **Model Dimension ($d_{\text{model}}$)** | **2,048** | Latent embedding space per temporal token | | **Layers ($N_{\text{layers}}$)** | **20** | Deep alternating Conformer blocks | | **Attention Mechanism** | **Grouped-Query Attention (GQA)** | 16 Query Heads, 8 Key/Value Heads (Head Dim = 128) | | **Positional Encoding** | **RoPE (Rotary Position Embeddings)** | Dynamic rotary complex embeddings | | **FFN Dimension ($d_{\text{ff}}$)** | **5,504** | SwiGLU Gated Linear Activation | | **Context Length ($T$)** | **2,048 Minutes** | ~34 trading hours (covers full intraday + overnight gap) | | **Convolution Tokenizer** | **6 Multi-Scale Kernels** | Overlapping $[3, 7, 15, 31, 63, 127]$ with Stride $S=4$ | | **Tokens per Context** | **512 Tokens** | 4× temporal downsampling with full receptive fidelity | | **Optimizer** | **8-Bit AdamW (bitsandbytes)** | Static VRAM: **2.38 GB** (BF16 weights) | ``` Raw OHLCV Candlesticks (T = 2048 mins) │ ▼ ┌──────────────────────────────────────────────┐ │ Stationary Feature Extraction Engine │ │ • Log Returns: ln(C_t / C_{t-1}) │ │ • Normalized Range: (H_t - L_t) / C_t │ │ • Normalized Body: (C_t - O_t) / (Range+eps)│ │ • Wick Dynamics: Upper & Lower Absorption │ │ • Parkinson Continuous Volatility │ │ • Rolling Volume Z-Score │ └──────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────┐ │ Reversible Instance Normalization (RevIN) │ │ • Eliminates Non-Stationary Regime Drift │ └──────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────┐ │ Multi-Scale Inception 1D Conv Tokenizer │ │ • Kernels: [3m, 7m, 15m, 31m, 63m, 127m] │ │ • Stride S = 4 ──> 512 Latent Tokens │ └──────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────┐ │ 20x Deep Conformer Backbone (d_model=2048) │ │ • Pre-LN RMSNorm │ │ • RoPE Grouped-Query Attention (16 Q / 8 KV)│ │ • Depthwise Separable 1D Conv (Local Flow) │ │ • SwiGLU Gated Feed-Forward (d_ff=5504) │ └──────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────┐ │ Dual Multi-Horizon Heads │ │ 1. Expected Return Vector (Huber Loss) │ │ 2. Upward Direction Probability (BCE Loss) │ │ Horizons: [1m, 3m, 5m, 10m, 15m, 30m, 1h]│ └──────────────────────────────────────────────┘ ``` --- ## 📈 Empirical Pre-Training Telemetry The model is pre-trained continuously across **16 Full Calendar Years (2010–2025 = 192 Monthly Shards, ~90 GB)** of minute-level US equity market data from `mito0o852/OHLCV-1m` on **NVIDIA A100-SXM4-80GB** accelerators. ### Macro Regimes Learned in Epoch 1: - **2010**: The Flash Crash & Post-GFC Quantitative Easing 1 - **2011**: European Sovereign Debt Crisis & US Sovereign Credit Rating Downgrade - **2012**: ECB "Whatever It Takes" & Global Central Bank Expansion - **2013**: Federal Reserve Taper Tantrum - **2014**: Crude Oil Crash & Zero Interest Rate Policy - **2015**: Chinese Market Volatility Shock & Renminbi Devaluation - **2016**: Brexit Referendum & 2016 US Presidential Election Volatility - **2017**: Historic Low-Volatility Secular Bull Market - **2018**: "Volmageddon" (XIV Collapse) & Q4 Monetary Tightening Selloff - **2019**: Federal Reserve Repo Market Crisis & Pre-COVID Pivot - **2020**: Global COVID-19 Liquidity Shock & Fastest Bear-to-Bull Recovery in History - **2021**: Retail Meme Stock Short Squeezes & Zero-Commission Dispersion - **2022**: 500 bps Quantitative Tightening Bear Market - **2023–2025**: Artificial Intelligence Infrastructure Supercycle ### Real-Time Validation Signals: - **1-Minute Forward Direction Accuracy**: **Up to 100.0%** (Observed at Step 1330 & 2220) - **1-Hour Forward Direction Accuracy**: **Up to 87.5% – 100.0%** - **Sustained Overall Multi-Horizon Accuracy**: **62.5% – 67.9%** (Significant edge over 50% random walk baseline) - **Total Loss Convergence**: Decreased monotonically from `0.3731` to `0.3370` --- ## 💻 Zero-Shot Inference in PyTorch ```python import torch from huggingface_hub import hf_hub_download from model_1b import FinancialConformer1B from config_1b import Model1BConfig device = "cuda" if torch.cuda.is_available() else "cpu" # 1. Download model weights from Hugging Face Model Hub print("⚡ [ULTRON] Fetching 1.189B Model Weights...") weights_path = hf_hub_download( repo_id="Ultron28/ohlcv-transformer", filename="checkpoints_1b/checkpoint_1b_latest.pt" ) # 2. Instantiate 1.189B Conformer Architecture cfg = Model1BConfig() model = FinancialConformer1B(cfg).to(device) # 3. Load checkpoint state checkpoint = torch.load(weights_path, map_location=device, weights_only=False) model.load_state_dict(checkpoint["model_state"]) model.eval() print("✅ [ULTRON] 1.189B Conformer Foundation Model Loaded Successfully.") # 4. Dummy inference on 2,048-minute candlestick window (B=1, T=2048, F=6) # Features: [LogReturn, NormRange, NormBody, UpperWick, LowerWick, NormVol] x = torch.randn(1, 2048, 6, device=device) with torch.no_grad(): predictions = model(x) return_forecast = predictions["pred_returns"] # Shape: [1, 7] (1m, 3m, 5m, 10m, 15m, 30m, 60m) dir_probabilities = predictions["pred_directions"] # Shape: [1, 7] (Upward probability in [0, 1]) print(f"1-Minute Directional Upward Probability: {dir_probabilities[0, 0].item():.2%}") print(f"1-Hour Directional Upward Probability: {dir_probabilities[0, -1].item():.2%}") ``` --- ## 📚 Citation & Author Information If you utilize this foundation model, dataset pipelines, or architectural findings in your quantitative research, trading algorithms, or academic papers, please cite: ```bibtex @misc{singh2026ultronconformer, author = {Suryaansh Prithvijit Singh}, title = {ULTRON: A 1.189 Billion Parameter Multi-Scale Financial Conformer for Deep Microstructure Forecasting Across 16 Years of US Equities}, year = {2026}, publisher = {Hugging Face}, journal = {Hugging Face Model Hub}, howpublished = {\url{https://huggingface.co/Ultron28/ohlcv-transformer}} } ``` ---
**Architected by Suryaansh Prithvijit Singh // ULTRON Cybernetic Operating Systems** *Connect: [GitHub (@Ultron09)](https://github.com/Ultron09) • [Hugging Face (@Ultron28)](https://huggingface.co/Ultron28) • `connect.singha@gmail.com`*