Ultron28 commited on
Commit
89b8f39
Β·
verified Β·
1 Parent(s): 3e7f0ab

Sync README.md

Browse files
Files changed (1) hide show
  1. README.md +66 -146
README.md CHANGED
@@ -58,64 +58,17 @@ While traditional academic financial machine learning has historically relied on
58
 
59
  ---
60
 
61
- ## πŸ”¬ Empirical Breakthrough Telemetry
62
 
63
- Financial time series exhibit extreme entropy with very low Signal-to-Noise Ratios ($SNR < 0.05$). Under standard semi-strong market efficiency, a binary directional bet has a theoretical random entropy baseline of:
64
 
65
- $$\mathcal{H}_{\text{random}} = -\ln(0.5) \approx \mathbf{0.6931}$$
66
-
67
- Classical quant funds (e.g., Renaissance Technologies, Two Sigma) built quantitative dynasties on directional edges of **51.5% to 53.5%**. During active 3-Epoch training on NVIDIA A100-SXM4-80GB hardware, the ULTRON 1.189B Conformer achieved:
68
-
69
- - **Total Loss Floor**: Decreased monotonically to **`0.3273`**.
70
- - **Directional Loss**: Penetrated down to **`0.6545`** (decisively breaking below the 0.6931 random barrier).
71
- - **Multi-Horizon Directional Accuracy**:
72
- - **Overall Aggregate**: **`58.9% – 69.6%`**
73
- - **1-Minute Horizon**: **`75.0% – 87.5%`** (Peaks at `100.0%`)
74
- - **5-Minute Horizon**: **`62.5% – 75.0%`** (Peaks at `100.0%`)
75
- - **15-Minute Horizon**: **`75.0%`**
76
- - **1-Hour Horizon**: **`75.0% – 87.5%`**
77
- - **Information Coefficient (IC)**: Estimated $> 0.15$ across tactical holding horizons.
78
-
79
- <div align="center">
80
- <img src="assets/training_telemetry.png" alt="ULTRON Pre-Training Telemetry" width="95%"/>
81
- </div>
82
-
83
- ---
84
-
85
- ## πŸ“š Multi-Paper Research Agenda
86
-
87
- The findings and empirical weight trajectories from this foundation training run form the core of a three-paper academic and quantitative research portfolio:
88
-
89
- ```
90
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
91
- β”‚ 1.189B Conformer Pre-Training (2010-2023) β”‚
92
- β”‚ 90 GB OHLCV-1m | 3 Full Foundation Epochs β”‚
93
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
94
- β”‚
95
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
96
- β–Ό β–Ό β–Ό
97
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
98
- β”‚ PAPER 1 β”‚ β”‚ PAPER 2 β”‚ β”‚ PAPER 3 β”‚
99
- β”‚ Scaling Laws & Multi-Scale β”‚ β”‚ Zero-Shot Alpha & The OOS β”‚ β”‚ Mechanistic Interpretability β”‚
100
- β”‚ Inductive Biases in Financial β”‚ β”‚ Frontier (2024-2026 Quarantinedβ”‚ β”‚ of Multi-Scale Financial β”‚
101
- β”‚ Time-Series β”‚ β”‚ Regimes) β”‚ β”‚ Conformer Latents β”‚
102
- β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
103
- β”‚ β€’ Power-law scaling (1.189B) β”‚ β”‚ β€’ Zero-shot cross-regime alpha β”‚ β”‚ β€’ Inception Conv kernel anatomy β”‚
104
- β”‚ β€’ Multi-scale Inception [3-127m]β”‚ β”‚ β€’ Untouched 2024-2026 evaluationβ”‚ β”‚ (3m noise vs 127m liquidity) β”‚
105
- β”‚ β€’ ReVIN non-stationarity proof β”‚ β”‚ β€’ Sharpe, Sortino & Max DD β”‚ β”‚ β€’ Attention head specialization β”‚
106
- β”‚ β€’ Multi-horizon co-optimization β”‚ β”‚ β€’ Execution slippage dynamics β”‚ β”‚ β€’ Latent regime phase shifts β”‚
107
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€οΏ½οΏ½β”€β”€β”€β”€β”€β”€β”€β”˜
108
- ```
109
-
110
- 1. **Paper 1: Scaling Laws & Neural Architectures in Financial Time-Series**
111
- - Proves empirical neural scaling laws across 1.189 Billion parameters on continuous financial OHLCV. Demonstrates that combining multi-scale temporal convolutions with self-supervised masked candlestick objectives ($15\%$ masking) overcomes noise memorization.
112
- 2. **Paper 2: The Out-of-Sample Frontier: Zero-Shot Alpha Across Unseen Regimes (2024–2026)**
113
- - Zero-shot evaluation on pristine, held-out market regimes (post-2023 inflation, mega-cap tech dispersion, aggressive central bank rate cycles). Compares Epoch 1 vs Epoch 2 vs Epoch 3 checkpoints across real-world Sharpe ratio, Sortino ratio, and transaction-cost-adjusted execution alpha.
114
- 3. **Paper 3: Mechanistic Interpretability of Conformer Multi-Scale Latents**
115
- - Direct dissection of the 20 Conformer attention heads and multi-scale convolution filters. Proves that lower kernel scales ($K=3, 7$) act as microstructure bid-ask absorption filters, while higher scales ($K=63, 127$) identify institutional order-flow imbalances and macro continuation.
116
 
117
  <div align="center">
118
- <img src="assets/multi_horizon_forecast.png" alt="Multi-Horizon Forecast" width="95%"/>
119
  </div>
120
 
121
  ---
@@ -126,112 +79,79 @@ Permanent weight archives are versioned on Hugging Face ([`Ultron28/ohlcv-transf
126
 
127
  | Checkpoint Name | Size (BF16) | Status | Description |
128
  | :--- | :--- | :--- | :--- |
129
- | **`checkpoint_1b_epoch1.pt`** | **4.79 GB** | βœ… **Archived & Verified** | Full Epoch 1 Milestone (Steps 1–5,010 across 168 shards) |
130
- | **`checkpoint_1b_epoch2.pt`** | **4.79 GB** | βœ… **Archived & Verified** | Full Epoch 2 Milestone (Steps 5,010–10,100 across 168 shards) |
131
- | **`checkpoint_1b_epoch3.pt`** | **4.47 GB** | βœ… **Archived & Verified** | Full Foundation Pass Complete (Steps 10,100–15,030 across 168 shards) |
132
- | **`checkpoint_1b_latest.pt`** | **4.47 GB** | ⚑ **Rolling Live** | Terminal foundation checkpoint weights |
133
-
134
- ---
135
-
136
- ## πŸ“ Model Architecture Specifications
137
-
138
- | Hyperparameter | Value | Description |
139
- | :--- | :--- | :--- |
140
- | **Total Parameters** | **1,189,135,740** | **1.189 Billion Parameters** |
141
- | **Model Dimension ($d_{\text{model}}$)** | **2,048** | Latent embedding space per temporal token |
142
- | **Layers ($N_{\text{layers}}$)** | **20** | Deep alternating Conformer blocks |
143
- | **Attention Mechanism** | **Grouped-Query Attention (GQA)** | 16 Query Heads, 8 Key/Value Heads (Head Dim = 128) |
144
- | **Positional Encoding** | **RoPE (Rotary Position Embeddings)** | Dynamic rotary complex embeddings |
145
- | **FFN Dimension ($d_{\text{ff}}$)** | **5,504** | SwiGLU Gated Linear Activation |
146
- | **Context Length ($T$)** | **2,048 Minutes** | ~34 trading hours (covers full intraday + overnight gap) |
147
- | **Convolution Tokenizer** | **6 Multi-Scale Kernels** | Overlapping $[3, 7, 15, 31, 63, 127]$ with Stride $S=4$ |
148
- | **Tokens per Context** | **512 Tokens** | 4Γ— temporal downsampling with full receptive fidelity |
149
- | **Normalizer** | **Reversible Instance Normalization (RevIN)** | Dynamic non-stationary distribution tracking |
150
- | **Optimizer** | **8-Bit AdamW (bitsandbytes)** | Static VRAM: **2.38 GB** (BF16 weights) |
151
- | **Hardware** | **NVIDIA A100-SXM4-80GB** | Native Tensor Core BF16 (100% compute saturation) |
152
 
153
  ---
154
 
155
- ## πŸ’» Zero-Shot Inference in PyTorch
156
-
157
- ```python
158
- import torch
159
- from huggingface_hub import hf_hub_download
160
- from model_1b import FinancialConformer1B
161
- from config_1b import Model1BConfig
162
-
163
- device = "cuda" if torch.cuda.is_available() else "cpu"
164
-
165
- # 1. Download model weights from Hugging Face Model Hub
166
- print("⚑ [ULTRON] Fetching 1.189B Model Weights...")
167
- weights_path = hf_hub_download(
168
- repo_id="Ultron28/ohlcv-transformer",
169
- filename="checkpoints_1b/checkpoint_1b_latest.pt"
170
- )
171
-
172
- # 2. Instantiate 1.189B Conformer Architecture
173
- cfg = Model1BConfig()
174
- model = FinancialConformer1B(cfg).to(device)
175
-
176
- # 3. Load checkpoint state
177
- checkpoint = torch.load(weights_path, map_location=device, weights_only=False)
178
- model.load_state_dict(checkpoint["model_state"])
179
- model.eval()
180
- print("βœ… [ULTRON] 1.189B Conformer Foundation Model Loaded Successfully.")
181
 
182
- # 4. Inference on 2,048-minute candlestick window (B=1, T=2048, F=6)
183
- # Features: [LogReturn, NormRange, NormBody, UpperWick, LowerWick, NormVol]
184
- x = torch.randn(1, 2048, 6, device=device)
185
-
186
- with torch.no_grad():
187
- predictions = model(x)
188
- return_forecast = predictions["pred_returns"] # Shape: [1, 7] (1m, 3m, 5m, 10m, 15m, 30m, 60m)
189
- dir_probabilities = predictions["pred_directions"] # Shape: [1, 7] (Upward probability in [0, 1])
190
-
191
- print(f"1-Minute Directional Upward Probability: {dir_probabilities[0, 0].item():.2%}")
192
- print(f"1-Hour Directional Upward Probability: {dir_probabilities[0, -1].item():.2%}")
193
- ```
194
 
195
  ---
196
 
197
- ## πŸ“‘ Research Papers & Technical Manuscripts
198
 
199
- This research is documented across three formal academic manuscripts authored by **Suryaansh Prithvijit Singh** (Lead Researcher, Airborne AI / AirborneHRS):
200
 
201
- | # | Paper Title | Formats | Focus Area |
202
  | :-: | :--- | :--- | :--- |
203
- | **1** | **Scaling 1.189-Billion Parameter Financial Conformers: Empirical Foundations and Multi-Horizon Directional Alpha Across 14 Years of Market Microstructure** | [πŸ“„ PDF](papers/Paper_1_Financial_Conformer_Foundation_Model.pdf) &bull; [πŸ“ Markdown](papers/Paper_1_Financial_Conformer_Foundation_Model.md) | Foundation architecture, 14-year pre-training dynamics (168 shards), multi-horizon scaling laws, and zero-shot OOS testing on 2024–2026. |
204
- | **2** | **Continuous Market Adaptation Without Catastrophic Forgetting: Online Walk-Forward Learning in Multi-Billion Parameter Financial Foundation Models** | [πŸ“„ PDF](papers/Paper_2_Continual_Walk_Forward_Online_Adaptation.pdf) &bull; [πŸ“ Markdown](papers/Paper_2_Continual_Walk_Forward_Online_Adaptation.md) | Online walk-forward adaptation (`wf2024`, `wf2025`), preserving 14-year historical priors while capturing new volatility regimes. |
205
- | **3** | **From Microstructure Waveforms to Executable Alpha: Mechanistic Frequency Decomposition and Transaction Friction Dynamics in Neural Forecasting** | [πŸ“„ PDF](papers/Paper_3_Mechanistic_Alpha_and_Execution_Friction.pdf) &bull; [πŸ“ Markdown](papers/Paper_3_Mechanistic_Alpha_and_Execution_Friction.md) | Inception kernel ablation ($K \in [3, 7, 15, 31, 63, 127]$), dollar-neutral portfolio slippage stress testing (0, 2, 5, 10 bps). |
206
 
207
  <div align="center">
208
- <img src="papers/figures/fig2_oos_directional_accuracy.png" alt="OOS Directional Accuracy" width="85%"/>
209
  </div>
210
 
211
  ---
212
 
213
- ## πŸ† Out-of-Sample Empirical Benchmark (2024 – 2026)
 
 
214
 
215
- Evaluated across **27 strictly held-out out-of-sample monthly Parquet shards** (95,040 independent test windows):
 
 
 
 
 
 
 
 
 
 
216
 
217
- ### 1. Static Zero-Shot Directional Accuracy & Rank IC
218
- | Forecast Horizon | 2024 Held-Out Acc | 2024 Rank IC | 2025 Held-Out Acc | 2025 Rank IC | 2026 Frontier Acc | 2026 Rank IC |
219
- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
220
- | **1-Minute** | **54.22%** | +0.0106 | **54.30%** | +0.0046 | **53.90%** | +0.0074 |
221
- | **3-Minute** | 52.15% | +0.0065 | 51.65% | -0.0003 | 51.91% | +0.0108 |
222
- | **5-Minute** | 51.39% | +0.0144 | 51.09% | -0.0005 | 51.39% | -0.0012 |
223
- | **15-Minute** | 50.52% | +0.0148 | 49.81% | +0.0057 | 50.12% | +0.0014 |
224
- | **30-Minute** | 51.17% | +0.0122 | 50.06% | +0.0120 | 48.21% | -0.0009 |
225
- | **60-Minute** | 51.49% | +0.0228 | 50.73% | +0.0241 | 47.96% | +0.0053 |
226
- | **1-Day** | 53.00% | +0.0269 | 52.10% | +0.0198 | 47.71% | +0.0028 |
227
- | **Overall Average**| **51.99%** | β€” | **51.39%** | β€” | **50.17%** | β€” |
228
 
229
- ### 2. Continual Walk-Forward Learning Retention
230
- - **Online Adaptation on 2024 (`checkpoint_1b_wf2024.pt`) Tested on 2025**: **51.40%** directional accuracy (strictly exceeds zero-shot baseline of 51.39% with zero catastrophic forgetting).
231
- - **Online Adaptation on 2025 (`checkpoint_1b_wf2025.pt`) Tested on 2026**: **53.90%** 1-minute accuracy, **+0.56** gross Sharpe ratio.
 
 
 
 
232
 
233
  <div align="center">
234
- <img src="papers/figures/fig4_kernel_ablation.png" alt="Inception Kernel Ablation" width="85%"/>
235
  </div>
236
 
237
  ---
@@ -241,13 +161,13 @@ Evaluated across **27 strictly held-out out-of-sample monthly Parquet shards** (
241
  If you utilize this foundation model, dataset pipelines, or academic papers in your quantitative research or trading algorithms, please cite:
242
 
243
  ```bibtex
244
- @misc{singh2026ultronconformer,
245
  author = {Suryaansh Prithvijit Singh},
246
- title = {ULTRON: A 1.189 Billion Parameter Multi-Scale Financial Conformer for Deep Microstructure Forecasting Across 14 Years of US Equities},
247
  year = {2026},
248
- publisher = {GitHub & Hugging Face},
249
- journal = {Hugging Face Model Hub},
250
- howpublished = {\url{https://github.com/Ultron09/ohlcv-transformer}}
251
  }
252
  ```
253
 
@@ -255,7 +175,7 @@ If you utilize this foundation model, dataset pipelines, or academic papers in y
255
 
256
  <div align="center">
257
 
258
- **Architected by Suryaansh Prithvijit Singh // ULTRON Cybernetic Operating Systems**
259
- *Connect: [GitHub (@Ultron09)](https://github.com/Ultron09) β€’ [Hugging Face (@Ultron28)](https://huggingface.co/Ultron28) β€’ `connect.singha@gmail.com`*
260
 
261
  </div>
 
58
 
59
  ---
60
 
61
+ ## πŸ”¬ Verified Pre-Training Telemetry (855 Logged Steps, 2010–2023)
62
 
63
+ Financial time series exhibit extreme entropy with very low Signal-to-Noise Ratios ($SNR < 0.05$). Under standard semi-strong market efficiency, the weighted binary cross-entropy floor ($0.5 \ln 2$) is **`0.34657`** ($\ln 2 \approx 0.69315$). Across 3 full epochs ($15,030$ steps; $855$ logged telemetry checkpoints in [`logs/pretraining_telemetry_verified.json`](logs/pretraining_telemetry_verified.json)) on NVIDIA A100-SXM4-80GB hardware:
64
 
65
+ - **Epoch 1 (Steps 320–4,970, $n=183$)**: Mean Joint Loss **`0.3448 Β± 0.0079`** (Min **`0.3176`**), Mean BCE **`0.6896`**, Mean Directional Acc **`53.24%`** (Max Batch **`71.40%`**).
66
+ - **Epoch 2 (Steps 5,030–9,900, $n=204$)**: Mean Joint Loss **`0.3447 Β± 0.0071`** (Min **`0.3242`**), Mean BCE **`0.6894`**, Mean Directional Acc **`53.25%`** (Max Batch **`75.00%`**).
67
+ - **Epoch 3 (Steps 10,360–15,030, $n=468$)**: Mean Joint Loss **`0.3450 Β± 0.0067`** (Min **`0.3234`**), Mean BCE **`0.6899`**, Mean Directional Acc **`52.66%`** (Max Batch **`75.00%`**).
68
+ - **Monotonic Loss Variance Contraction**: Batch loss standard deviation contracts by **`-15.2%`** across epochs ($0.0079 \to 0.0071 \to 0.0067$), while global Frobenius parameter shift decelerates from **`10.0001%`** (`Ep1β†’Ep2`) to **`5.7305%`** (`Ep2β†’Ep3`).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
 
70
  <div align="center">
71
+ <img src="papers/figures/fig1_pretraining_curves.png" alt="ULTRON Pre-Training Telemetry" width="95%"/>
72
  </div>
73
 
74
  ---
 
79
 
80
  | Checkpoint Name | Size (BF16) | Status | Description |
81
  | :--- | :--- | :--- | :--- |
82
+ | **`checkpoint_1b_epoch1.pt`** | **4.79 GB** | βœ… **Archived & Verified** | Full Epoch 1 Milestone (Step 5,010 across 168 shards) |
83
+ | **`checkpoint_1b_epoch2.pt`** | **4.79 GB** | βœ… **Archived & Verified** | Full Epoch 2 Milestone (Step 10,020 across 168 shards) |
84
+ | **`checkpoint_1b_epoch3.pt`** | **4.47 GB** | βœ… **Archived & Verified** | Full Epoch 3 Foundation Pass (Step 15,030 across 168 shards) |
85
+ | **`checkpoint_1b_latest.pt`** | **4.47 GB** | βœ… **Foundation Release** | Terminal 14-year (2010–2023) foundation weights |
86
+ | **`checkpoint_1b_wf2024.pt`** | **4.43 GB** | βœ… **Walk-Forward 2024** | Online continual adaptation across 2024 shards (`lr=1e-5`, `0.5439%` Frobenius drift) |
87
+ | **`checkpoint_1b_wf2025.pt`** | **4.43 GB** | βœ… **Walk-Forward 2025** | Online continual adaptation across 2024+2025 shards (`lr=1e-5`, `0.7860%` cumulative drift) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
  ---
90
 
91
+ ## πŸ“ Exact Model Architecture Audit (`1,189,135,740` Parameters)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
92
 
93
+ | Subsystem / Module | Configuration & Shapes | Exact Parameters | % of Total |
94
+ | :--- | :--- | :---: | :---: |
95
+ | **Inception Conv1d Branches ($K=3,7,15,31,63,127$)** | $6 \times [\text{Conv1d}(8 \to 341, K_i, s=4) + \text{bias}]$ | `673,134` | `0.057%` |
96
+ | **Inception Linear Projection & RMSNorm** | $\mathbf{W}_{\text{proj}} (2046 \times 2048) + \text{RMSNorm}(2048)$ | `4,192,256` | `0.353%` |
97
+ | **20Γ— GQA Attention (`q, k, v, out` + Norm)** | $H_q=16, H_{kv}=8, d_{\text{head}}=128$, RoPE | `251,699,200` | `21.167%` |
98
+ | **20Γ— Depthwise Separable Conv Module** | Pointwise GLU ($2048 \to 4096$), $\text{DWConv}_{K=7}$, GroupNorm-32 | `251,985,920` | `21.191%` |
99
+ | **20Γ— SwiGLU Feed-Forward Network** | $d_{\text{model}}=2048, d_{\text{ff}}=5504$ (`w_gate, w_up, w_down`) | `676,372,480` | `56.879%` |
100
+ | **Final Backbone RMSNorm & Dual MLP Heads** | `final_norm` ($2,048$) + `return_head` & `dir_head` ($2\times 2,105,351$) | `4,212,750` | `0.354%` |
101
+ | **Total Financial Conformer (1.189B)** | **20 Conformer Blocks, $L=2048 \to T=512$ Tokens** | **`1,189,135,740`** | **`100.000%`** |
 
 
 
102
 
103
  ---
104
 
105
+ ## πŸ“‘ Tier-1 Research Papers & Technical Manuscripts
106
 
107
+ This research is documented across three formal academic manuscripts authored by **Suryaansh Prithvijit Singh** (*Lead Researcher, Airborne AI / AirborneHRS* &bull; `Suryaansh@airbornehrs.in`):
108
 
109
+ | # | Paper Title | Formats | Core Empirical & Mathematical Contributions |
110
  | :-: | :--- | :--- | :--- |
111
+ | **1** | **Scaling 1.189-Billion Parameter Financial Conformers: Empirical Foundations, Pre-Training Dynamics, and Multi-Horizon Directional Alpha Across 14 Years of Market Microstructure** | [πŸ“„ PDF](papers/Paper_1_Financial_Conformer_Foundation_Model.pdf) &bull; [πŸ“ Markdown](papers/Paper_1_Financial_Conformer_Foundation_Model.md) | Exact 1.189B architecture equations, 855-step pre-training telemetry ($-15.2\%$ loss variance contraction), 95,040-window OOS benchmark (2024–2026), microstructure flat-bar audit, and multi-epoch scaling vs. 4 classical baselines (30 refs). |
112
+ | **2** | **Continuous Market Adaptation Without Catastrophic Forgetting: Online Walk-Forward Learning and Frobenius Manifold Stability in Multi-Billion Parameter Financial Foundation Models** | [πŸ“„ PDF](papers/Paper_2_Continual_Walk_Forward_Online_Adaptation.pdf) &bull; [πŸ“ Markdown](papers/Paper_2_Continual_Walk_Forward_Online_Adaptation.md) | Trust-region walk-forward adaptation (`wf2024`, `wf2025`), exact 308-tensor Frobenius weight drift & cosine similarity audit ($0.5439\%$ global drift, $0.999920$ CosSim), plasticity hierarchy, and forward ROC-AUC/Rank IC transfer (23 refs). |
113
+ | **3** | **From Microstructure Waveforms to Executable Alpha: Mechanistic Receptive-Field Decomposition and Causal Transaction Friction Dynamics in Billion-Parameter Neural Forecasting** | [πŸ“„ PDF](papers/Paper_3_Mechanistic_Alpha_and_Execution_Friction.pdf) &bull; [πŸ“ Markdown](papers/Paper_3_Mechanistic_Alpha_and_Execution_Friction.md) | 7-horizon continuous $\ell_2$ & discrete threshold kernel ablation ($K \in \{3..127\}$), Frequency-Horizon Specialization Law, and strictly causal chronological execution backtest with rolling $z$-score confidence hurdles across $0, 1, 2, 5\text{ bps}$ slippage (22 refs). |
114
 
115
  <div align="center">
116
+ <img src="papers/figures/fig2_oos_directional_accuracy.png" alt="OOS Directional Accuracy and Baselines" width="95%"/>
117
  </div>
118
 
119
  ---
120
 
121
+ ## πŸ† Out-of-Sample Empirical Benchmark & Scientific Audit (2024 – 2026)
122
+
123
+ All empirical metrics are 100% deterministic and reproducible via [`logs/oos_evaluation_results.json`](logs/oos_evaluation_results.json), [`logs/pretraining_telemetry_verified.json`](logs/pretraining_telemetry_verified.json), and [`logs/tier1_verified_audit.json`](logs/tier1_verified_audit.json) (generated by [`scripts/tier1_scientific_audit.py`](scripts/tier1_scientific_audit.py)).
124
 
125
+ ### 1. 27-Shard Zero-Shot Out-of-Sample Benchmark (95,040 Test Windows)
126
+ | Forecast Horizon | 2024 Dir Acc (%) | 2024 Rank IC | 2025 Dir Acc (%) | 2025 Rank IC | 2026 Dir Acc (%) | 2026 Rank IC | 3-Year Weighted Acc |
127
+ | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
128
+ | **1-Minute ($\tau=1$)** | **54.22%** | $+0.0106$ | **54.30%** | $+0.0046$ | **53.90%** | $+0.0074$ | **54.22%** |
129
+ | **3-Minute ($\tau=3$)** | **52.15%** | $+0.0065$ | **51.65%** | $-0.0003$ | **51.91%** | $+0.0108$ | **51.90%** |
130
+ | **5-Minute ($\tau=5$)** | **51.39%** | $+0.0144$ | **51.09%** | $-0.0005$ | **51.39%** | $-0.0012$ | **51.26%** |
131
+ | **15-Minute ($\tau=15$)** | **50.52%** | $+0.0148$ | $49.81\%$ | $+0.0057$ | **50.12%** | $+0.0014$ | **50.16%** |
132
+ | **30-Minute ($\tau=30$)** | **51.17%** | $+0.0122$ | **50.06%** | $+0.0120$ | $48.21\%$ | $-0.0009$ | **50.35%** |
133
+ | **60-Minute ($\tau=60$)** | **51.49%** | $+0.0228$ | **50.73%** | $+0.0241$ | $47.96\%$ | $+0.0053$ | **50.76%** |
134
+ | **1-Day ($\tau=1440$)** | **53.00%** | $+0.0269$ | **52.10%** | $+0.0198$ | $47.71\%$ | $+0.0028$ | **52.01%** |
135
+ | **7-Horizon Mean** | **51.99%** | **$+0.0155$** | **51.39%** | **$+0.0093$** | **50.17%** | **$+0.0037$** | **51.52%** |
136
 
137
+ ### 2. Exact Frobenius Weight Drift & Online Walk-Forward Adaptation
138
+ - **Frobenius Plasticity Hierarchy (`Ep3 β†’ WF-2024`)**: Global Frobenius drift is **`0.5439%`** (Cosine Similarity **`0.999920`**), with the 20-layer Conformer backbone shifting by only **`0.5291%`** while the Inception Tokenizer (**`1.1773%`**) and Directional Head (**`1.0087%`**) recalibrate to current regimes.
139
+ - **Forward ROC-AUC & Rank IC Expansion**: `WF-2024` improves 2025 OOS Strictly Non-Zero Accuracy (`51.27% β†’ 51.32%`), Balanced Accuracy (`50.26% β†’ 50.31%`), and ROC-AUC (**`0.5125 β†’ 0.5154`**), while `WF-2025` elevates 2025 ROC-AUC to **`0.5167`** (`60m` Rank IC **`+0.0542`**, `1d` Rank IC **`+0.0826`**).
140
+
141
+ <div align="center">
142
+ <img src="papers/figures/fig3_walk_forward_comparison.png" alt="Frobenius Drift and Walk-Forward ROC-AUC" width="95%"/>
143
+ </div>
 
 
 
 
144
 
145
+ ### 3. Mechanistic Kernel Ablation & Strictly Causal Execution Backtest
146
+ - **Frequency-Horizon Specialization**: Masking short receptive fields (`K=7` or `Micro K=3,7`) induces **`18.23%–22.33%`** mean continuous logit $\ell_2$ perturbation and **`49.53%–63.60%`** return forecast $\ell_2$ perturbation (peaking at **`36.11%`** on 15m logits while leaving 1d logits nearly invariant at **`5.54%`**). Masking macro `K=127` drives the largest single-branch accuracy drop (**`-0.11%`**), and masking `Meso K=15,31` drives the largest ROC-AUC drop (**`-0.0019`**).
147
+ - **Post-Friction Net Sharpe Survival Under Causal Confidence Filtering**:
148
+ - **2024 `5m` ($\tau_{\text{conf}}=2.0\sigma$, $99.3\%$ turnover saved)**: **`+6.24 bps/trade`** gross return (breakeven **`3.12 bps`**), Net Sharpe **`+4.18` (0 bps), `+2.84` (1 bps), `+1.50` (2 bps)**.
149
+ - **2025 `60m` ($\tau_{\text{conf}}=1.5\sigma$, $94.4\%$ turnover saved)**: **`+5.14 bps/trade`** gross return (breakeven **`2.57 bps`**), Net Sharpe **`+0.80` (0 bps), `+0.49` (1 bps), `+0.18` (2 bps)**.
150
+ - **2025 `1d` ($\tau_{\text{conf}}=0.0\sigma$ unfiltered / $1.0\sigma$ filtered)**: Unfiltered **`+10.67 bps/trade`** (breakeven **`5.34 bps`**), Net Sharpe **`+1.11` (0 bps), `+0.90` (1 bps), `+0.69` (2 bps), `+0.07` (5 bps)**; at $\tau_{\text{conf}}=1.0\sigma$, gross return jumps to **`+19.01 bps/trade`** (breakeven **`9.51 bps`**).
151
+ - **2026 `15m` ($\tau_{\text{conf}}=2.0\sigma$, $98.9\%$ turnover saved)**: **`+4.63 bps/trade`** gross return (breakeven **`2.31 bps`**), Net Sharpe **`+0.91` (0 bps), `+0.52` (1 bps), `+0.12` (2 bps)**.
152
 
153
  <div align="center">
154
+ <img src="papers/figures/fig5_slippage_friction_curve.png" alt="Causal Confidence Hurdle and Post-Friction Net Sharpe" width="95%"/>
155
  </div>
156
 
157
  ---
 
161
  If you utilize this foundation model, dataset pipelines, or academic papers in your quantitative research or trading algorithms, please cite:
162
 
163
  ```bibtex
164
+ @article{singh2026financialconformer,
165
  author = {Suryaansh Prithvijit Singh},
166
+ title = {Scaling 1.189-Billion Parameter Financial Conformers: Empirical Foundations, Pre-Training Dynamics, and Multi-Horizon Directional Alpha Across 14 Years of Market Microstructure},
167
  year = {2026},
168
+ institution = {Airborne AI / AirborneHRS},
169
+ note = {Lead Researcher, Suryaansh@airbornehrs.in},
170
+ url = {https://github.com/Ultron09/ohlcv-transformer}
171
  }
172
  ```
173
 
 
175
 
176
  <div align="center">
177
 
178
+ **Architected by Suryaansh Prithvijit Singh // Lead Researcher, Airborne AI / AirborneHRS**
179
+ *Connect: [GitHub (@Ultron09)](https://github.com/Ultron09) β€’ [Hugging Face (@Ultron28)](https://huggingface.co/Ultron28) β€’ `Suryaansh@airbornehrs.in`*
180
 
181
  </div>