README.md

---
language:
- en
license: apache-2.0
library_name: pytorch
tags:
- time-series
- financial-forecasting
- ohlcv
- conformer
- foundation-model
- 1b-parameters
- quantitative-finance
- algorithmic-trading
- market-microstructure
- deep-learning
datasets:
- mito0o852/OHLCV-1m
metrics:
- roc-auc
- directional-accuracy
- huber-loss
pipeline_tag: time-series-forecasting
---

<div align="center">

# ⚡ ULTRON // 1.189B Parameter Financial Conformer
### A Foundation Architecture for Multi-Horizon Financial Microstructure Forecasting
**Suryaansh Prithvijit Singh** &bull; Lead Researcher, Airborne AI / AirborneHRS  
Email: `Suryaansh@airbornehrs.in` | GitHub: [@Ultron09](https://github.com/Ultron09) | HF: [@Ultron28](https://huggingface.co/Ultron28)

[![DOI](https://zenodo.org/badge/1425150644.svg)](https://doi.org/10.5281/zenodo.23466965)
[![GitHub Repository](https://img.shields.io/badge/GitHub-ohlcv--transformer-blue?logo=github)](https://github.com/AirBorneAI/ohlcv-transformer)
[![Hugging Face Model](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-1.189B%20Model%20Hub-yellow)](https://huggingface.co/Ultron28/ohlcv-transformer)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AirBorneAI/ohlcv-transformer/blob/main/OHLCV_Transformer_Colab.ipynb)
[![Parameters](https://img.shields.io/badge/Parameters-1%2C189%2C135%2C740%20(1.189B)-crimson)](#-exact-model-architecture-audit-1189135740-parameters)
[![Papers Available](https://img.shields.io/badge/Research%20Papers-Unified%20Monograph%20%2B%203%20Studies-purple)](#-tier-1-research-papers--unified-monograph)
[![Context Window](https://img.shields.io/badge/Context-2%2C048%20Minutes%20(~34h)-cyan)](#-executive-summary)

</div>

---

## 🏛️ Executive Summary

The **ULTRON Financial Conformer (`1,189,135,740` parameters across `283` parameter tensors)** is a billion-parameter hybrid convolution-attention foundation model pre-trained on 14 continuous calendar years (`2010–2023`, `168` monthly Parquet shards, `15,030` optimization steps) of 1-minute US equity OHLCV bars (`mito0o852/OHLCV-1m`) across 15 liquid US equities and ETFs (`AAPL, AMD, AMZN, BAC, GOOGL, INTC, IWM, JPM, META, MSFT, NFLX, NVDA, QQQ, SPY, TSLA`).

All empirical claims are **100% deterministic, peer-review-audited, and reproducible** via [`logs/defensible_tier1_audit.json`](logs/defensible_tier1_audit.json) and [`logs/pretraining_telemetry_verified.json`](logs/pretraining_telemetry_verified.json) (generated by [`scripts/defensible_scientific_audit.py`](scripts/defensible_scientific_audit.py)):

1. **1-Minute Base-Rate & Flat-Bar Decomposition**: Across our stratified 15-ticker contiguous evaluation cohorts (`2024`: `8,044` windows, `2025`: `8,060` windows, `2026`: `8,100` windows), exact zero-return flat bars ($y_{t,1\text{m}}=0$) account for **`7.86%` (2024), `6.85%` (2025), and `6.46%` (2026)** of observations, reducing strictly positive 1m returns $P(y_{t,1\text{m}} > 0)$ to **`46.51%`, `46.36%`, and `46.15%`**. Under BCE training, the 1m logit centers around $\bar{z}_{1\text{m}} = -0.1627$, so uncalibrated $z > 0$ thresholding fires `0.0%` of the time (`53.49%–53.85%` raw accuracy, identical to a `FlatBar-Down` heuristic). Removing this static threshold bias proves genuine discrimination: **1m All-Bar ROC-AUC of `0.5375 / 0.5262 / 0.5274`**, **1m Strict Non-Zero ROC-AUC (`NZAUC`, $y \neq 0$) of `0.5248 / 0.5138 / 0.5159`**, **Prior-Calibrated Accuracy of `52.04% / 51.23% / 51.25%`**, and active two-sided **5m Strict Non-Zero Accuracy of `51.74%` (2024) and `51.18%` (2026)**.
2. **Exact Global Cosine Geometry, Adam Arithmetic & Pre-2024 Crisis Forgetting**: Using the exact `float64` Hilbert-space Law of Cosines $\cos_{\text{exact}} = \frac{1 + r^2 - \delta_F^2}{2r}$, every module strictly satisfies the Cauchy-Schwarz lower bound $\cos_{\text{exact}} \ge \sqrt{1 - \delta_F^2}$: global Frobenius drift is **`0.5439%` ($\cos_{\text{exact}} = 0.99998521 \ge 0.99998521$)** for `Ep3 → WF-2024`, **`0.5725%` ($\cos_{\text{exact}} = 0.99998361$)** for `WF-2024 → WF-2025`, and **`0.7860%` ($\cos_{\text{exact}} = 0.99996911$)** cumulatively for `Ep3 → WF-2025`. Decomposing drift into RMS weight norm $\|\mathbf{W}\|_{\text{RMS}}$ and absolute RMS update $\|\Delta\mathbf{W}\|_{\text{RMS}}$ proves that the Inception Tokenizer has the **largest** weight norm (`0.02574` vs. `0.01432` for the backbone), yet still exhibits the highest relative drift (`1.1773%`) because its absolute RMS update (`3.030e-4`) is **4.00x larger** than the backbone (`7.575e-5`). Direct evaluation on **Pre-2024 Crisis Shards (`2020-03` COVID crash, `2022-06` Fed rate shock, `2023-12`; `7,896` windows)** confirms zero overall forgetting (`0.5130 → 0.5135` AUC; `2020-03` COVID 1m $\Delta\text{AUC} = +0.0052\ [+0.0015, +0.0084]$), while 2026 paired block-bootstrap 95% CIs confirm a significant forward 1m gain ($\Delta\text{AUC} = +0.0022\ [+0.0001, +0.0040]$).
3. **Controlled Mechanistic Ablation & Denominator Proof**: Using in-distribution mean-ablation and random 341ch/682ch controls, we prove that the apparent `15m` peak in uncentered $\|\Delta z_h\|_2 / \|z_h\|_2$ is a **denominator effect** (`||z_15m||_RMS = 0.0405` as $\bar{z}_{15\text{m}} \approx 0$, vs. `0.2041` at `1m` and `0.1285` at `1d`), whereas denominator-free $\text{RMS}(\Delta z_h)$ decays monotonically (`0.0403` at `1m` $\to$ `0.0136` at `15m` $\to$ `0.0058` at `1d` for `K=7`) and variance-normalized centered L2 sensitivity significantly exceeds random channel controls (`49.90%` for `Meso K=15,31` vs. `30.89%` for `Random 682-Ch Control`).
4. **Non-Overlapping Execution Economics & Frozen 2024 Policy**: Stepping by $S_h = \lceil \tau_h / 32 \rceil$ windows eliminates multi-period overlap, and tracking state-dependent turnover $|w_{i,k} - w_{i,k-1}|$ shows true state-hold breakeven is `1.15x–2.41x` higher than isolated `Gross / 2`. The single frozen policy selected on 2024 (`60m, τ=0.0σ`) achieves **`+4.40` Gross Sharpe (`+2.04` Net Sharpe at `1.5 bps`, `BE = 2.82 bps`, `MaxDD = 1.69%`)** in 2024 and **`+1.01` Gross Sharpe** untouched in 2025, while a fixed demeaned $\tau=1.0\sigma$ rule at `3m` achieves **`+5.38` (2024), `+4.91` (2025), and `+3.56` (2026) Gross Sharpe** across all 3 years (`BE = 0.35–0.44 bps`), establishing that standalone OHLCV signals require passive maker/execution-timing deployment rather than taker spread crossing.

<div align="center">
  <img src="assets/architecture_schematic.png" alt="ULTRON Financial Conformer Architecture" width="90%"/>
</div>

---

## 📑 Tier-1 Research Papers & Unified Monograph

This research is available both as a **Unified Flagship Monograph** (consolidating the entire checkpoint and log lineage into a single self-contained 11-page paper) and as three synchronized individual manuscripts authored by **Suryaansh Prithvijit Singh** (*Lead Researcher, Airborne AI / AirborneHRS* &bull; `Suryaansh@airbornehrs.in`):

| # | Manuscript Title | Direct PDF Link | Core Empirical & Mathematical Contributions |
| :-: | :--- | :--- | :--- |
| **★** | **The 1.189-Billion Parameter Financial Conformer: A Unified Empirical Audit of Pre-Training Scaling, Base-Rate Calibration, Continual Walk-Forward Geometry, Controlled Receptive-Field Ablation, and Non-Overlapping Execution Economics** *(Flagship Unified Monograph)* | [📄 Unified PDF (11p)](papers/Paper_Unified_Financial_Conformer_Monograph.pdf) | Complete self-contained 11-page monograph consolidating Architecture, Pre-Training Telemetry, 1m Base-Rate/Flat-Bar Decomposition, Exact float64 Law-of-Cosines Drift, Pre-2024 Crisis Forgetting, Controlled Mean-Ablation, and Non-Overlapping Frozen-Policy Backtesting. |
| **1** | **Scaling 1.189-Billion Parameter Financial Conformers: Pre-Training Dynamics, Base-Rate Decomposition, and Non-Zero Multi-Horizon Alpha in Market Microstructure** | [📄 Paper 1 PDF (6p)](papers/Paper_1_Financial_Conformer_Foundation_Model.pdf) | Exact 283-tensor architecture audit (`1,189,135,740` params), 855-step pre-training telemetry (`-15.2%` loss variance contraction), 1m flat-bar & base-rate log-odds proof, Strict Non-Zero ROC-AUC (`NZAUC`), and $|y|$-quartile stratification (`Q1..Q4`). |
| **2** | **Continual Walk-Forward Adaptation in 1.189B Financial Conformers: Exact Cosine Geometry, Adam Plasticity Arithmetic, and Pre-2024 Crisis Forgetting Audits** | [📄 Paper 2 PDF (4p)](papers/Paper_2_Continual_Walk_Forward_Online_Adaptation.pdf) | Exact `float64` Law-of-Cosines global cosine similarity ($\cos_{\text{exact}} \ge \sqrt{1-\delta_F^2}$), per-parameter $\|\mathbf{W}\|_{\text{RMS}}$ vs. $\|\Delta\mathbf{W}\|_{\text{RMS}}$ Adam arithmetic proof, Pre-2024 crisis stress tests (`2020-03` COVID, `2022-06` Rates), and 2026 paired block-bootstrap 95% CIs. |
| **3** | **Mechanistic Interpretability and Non-Overlapping Execution Economics in 1.189B Financial Conformers: Controlled Kernel Ablations, Denominator Decomposition, and Deflated Sharpe Audits** | [📄 Paper 3 PDF (5p)](papers/Paper_3_Mechanistic_Alpha_and_Execution_Friction.pdf) | In-distribution Mean-Ablation vs. Zero-Ablation with Random 341ch/682ch Controls, proof of the 15m denominator effect vs. monotonic $\text{RMS}(\Delta z_h)$, and non-overlapping contiguous backtesting with state turnover, DSR ($M=35$), and untouched Frozen 2024 Policy transfer. |

<div align="center">
  <img src="papers/figures/fig2_oos_directional_accuracy.png" alt="OOS Non-Zero ROC-AUC and 1m Return Quartiles" width="95%"/>
</div>

---

## 📐 Exact Model Architecture Audit (`1,189,135,740` Parameters across `283` Tensors)

| Subsystem / Module | Configuration & Shapes | Exact Parameters | % of Total |
| :--- | :--- | :---: | :---: |
| **Inception Conv1d Branches ($K=3,7,15,31,63,127$)** | $6 \times [\text{Conv1d}(8 \to 341, K_i, s=4) + \text{bias}]$ | `673,134` | `0.057%` |
| **Inception Linear Projection & RMSNorm** | $\mathbf{W}_{\text{proj}} (2046 \times 2048) + \text{RMSNorm}(2048)$ | `4,192,256` | `0.353%` |
| **20× GQA Attention (`q, k, v, out`)** | $H_q=16, H_{kv}=8, d_{\text{head}}=128$, RoPE | `251,658,240` | `21.163%` |
| **20× Depthwise Separable Conv Module** | Pointwise GLU ($2048 \to 4096$), $\text{DWConv}_{K=7}$, Norm | `251,985,920` | `21.191%` |
| **20× SwiGLU Feed-Forward Network** | $d_{\text{model}}=2048, d_{\text{ff}}=5504$ (`w_gate, w_up, w_down`) | `676,331,520` | `56.876%` |
| **20× Conformer Block RMSNorms** | $3 \times \text{RMSNorm}(2048)$ per block (`norm_attn, norm_conv, norm_ffn`) | `122,880` | `0.010%` |
| **Final Backbone RMSNorm & Dual MLP Heads** | `final_norm` ($2,048$) + `return_head` & `dir_head` ($2\times 2,084,871$) | `4,171,790` | `0.351%` |
| **Total Financial Conformer (1.189B)** | **20 Conformer Blocks, $L=2048 \to T=512$ Tokens (`283` Tensors)** | **`1,189,135,740`** | **`100.000%`** |

---

## 🏆 Peer-Review-Hardened Empirical Results (`logs/defensible_tier1_audit.json`)

### 1. Stratified 15-Ticker OOS Evaluation of Frozen Epoch 3 (`2024–2026`)
| Horizon ($S_h$ Step) | Year | $N_{\text{eff}}$ | Flat % | Pos % | $P(z>0)$ | Raw Acc | NZ Acc ($y \neq 0$) | Cal Acc | All AUC | NZ AUC | Non-Ovlp IC |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **1m ($S=1$)** | 2024 | 8,044 | `7.86%` | `46.51%` | `0.0%` | `53.49%` | `49.53%` | **`52.04%`** | **`0.5375`** | **`0.5248`** | `+0.0137` |
| **1m ($S=1$)** | 2025 | 8,060 | `6.85%` | `46.36%` | `0.0%` | `53.64%` | `50.23%` | **`51.23%`** | **`0.5262`** | **`0.5138`** | `+0.0210` |
| **1m ($S=1$)** | 2026 | 8,100 | `6.46%` | `46.15%` | `0.0%` | `53.85%` | `50.67%` | **`51.25%`** | **`0.5274`** | **`0.5159`** | `+0.0247` |
| **3m ($S=1$)** | 2024 | 8,044 | `4.00%` | `48.51%` | `27.0%` | `52.05%` | `50.80%` | **`51.48%`** | **`0.5256`** | **`0.5185`** | `-0.0191` |
| **3m ($S=1$)** | 2026 | 8,100 | `2.94%` | `48.27%` | `27.0%` | `52.17%` | **`51.28%`** | **`51.09%`** | **`0.5204`** | **`0.5147`** | `-0.0137` |
| **5m ($S=1$)** | 2024 | 8,044 | `3.38%` | `48.69%` | `38.8%` | `52.36%` | **`51.74%`** | **`51.32%`** | **`0.5256`** | **`0.5219`** | `+0.0150` |
| **5m ($S=1$)** | 2026 | 8,100 | `2.52%` | `48.40%` | `39.4%` | `51.64%` | **`51.18%`** | **`51.17%`** | **`0.5166`** | **`0.5134`** | `-0.0101` |
| **15m ($S=1$)** | 2024 | 8,044 | `1.60%` | `50.85%` | `62.3%` | `50.90%` | **`50.97%`** | **`50.86%`** | **`0.5170`** | **`0.5157`** | `+0.0259` |
| **60m ($S=2$)** | 2025 | 4,030 | `0.47%` | `50.30%` | `100.0%` | `50.30%` | `50.54%` | `50.56%` | **`0.5117`** | **`0.5128`** | **`+0.0411`** |
| **1d ($S=13$)** | 2024 | 627 | `0.20%` | `53.01%` | `100.0%` | `53.01%` | `53.11%` | **`51.28%`** | `0.5062` | `0.5063` | `+0.0051` |
| **1d ($S=13$)** | 2025 | 627 | `0.15%` | `52.72%` | `100.0%` | `52.72%` | `52.80%` | `50.22%` | `0.4994` | `0.4991` | **`+0.0451`** |

### 2. Exact `float64` Law-of-Cosines Geometry & Adam Plasticity Hierarchy
| Subsystem / Module | Params | `Ep3 → WF24` $\delta_F$ | `WF24 → WF25` $\delta_F$ | `Ep3 → WF25` $\delta_F$ | `Ep3 → WF24` Exact $\cos$ ($\ge \sqrt{1-\delta_F^2}$) | $\|\mathbf{W}\|_{\text{RMS}}$ | $\|\Delta\mathbf{W}_{24}\|_{\text{RMS}}$ |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **Inception Tokenizer** | `4.87M` | **`1.1773%`** | `1.1807%` | `1.6172%` | `0.99993072` (`0.99993070`) | **`0.02574`** | **`3.030e-04`** |
| **GQA Attention (`L0–19`)** | `251.66M` | **`1.1705%`** | `1.2528%` | `1.7160%` | `0.99993150` (`0.99993149`) | `0.01241` | **`1.452e-04`** |
| **Directional Head** | `2.11M` | **`1.0087%`** | `0.9400%` | `1.4724%` | `0.99994962` (`0.99994912`) | `0.01582` | `1.596e-04` |
| **All 20 Blocks Backbone** | `1180.06M` | `0.5291%` | `0.5591%` | `0.7671%` | `0.99998601` (`0.99998600`) | `0.01432` | `7.575e-05` |
| **SwiGLU FFN (`L0–19`)** | `676.37M` | `0.3616%` | `0.3472%` | `0.4778%` | `0.99999347` (`0.99999346`) | `0.01209` | `4.374e-05` |
| **Return Reg. Head** | `2.11M` | `0.2419%` | `0.2698%` | `0.2998%` | `0.99999707` (`0.99999707`) | `0.01106` | `2.676e-05` |
| **Depthwise Conv (`L0–19`)** | `251.99M` | `0.1540%` | `0.1998%` | `0.2802%` | `0.99999881` (`0.99999881`) | `0.01692` | `2.606e-05` |
| **Global Model (`283` Tensors)** | **`1,189.14M`** | **`0.5439%`** | **`0.5725%`** | **`0.7860%`** | **`0.99998521` (`0.99998521`)** | **`0.01438`** | **`7.821e-05`** |

<div align="center">
  <img src="papers/figures/fig3_walk_forward_comparison.png" alt="Plasticity Hierarchy and Pre-2024 Stress Forgetting Audit" width="95%"/>
</div>

### 3. Controlled Mechanistic Ablation & Non-Overlapping Execution Economics

<div align="center">
  <img src="papers/figures/fig4_kernel_ablation.png" alt="Controlled Mean-Ablation and 15m Denominator Decomposition" width="95%"/>
  <br/>
  <img src="papers/figures/fig5_slippage_friction_curve.png" alt="Non-Overlapping State Breakeven and Frozen 2024 Policy Transfer" width="95%"/>
</div>

---

## 📚 Citation & Author Information

If you utilize this foundation model, dataset pipelines, or academic papers in your quantitative research or trading algorithms, please cite:

```bibtex
@article{singh2026financialconformer,
  author       = {Suryaansh Prithvijit Singh},
  title        = {The 1.189-Billion Parameter Financial Conformer: A Unified Empirical Audit of Pre-Training Scaling, Base-Rate Calibration, Continual Walk-Forward Geometry, Controlled Receptive-Field Ablation, and Non-Overlapping Execution Economics},
  year         = {2026},
  institution  = {Airborne AI / AirborneHRS},
  doi          = {10.5281/zenodo.23466965},
  url          = {https://doi.org/10.5281/zenodo.23466965},
  note         = {Lead Researcher, Suryaansh@airbornehrs.in}
}
```

---

<div align="center">

**Architected by Suryaansh Prithvijit Singh // Lead Researcher, Airborne AI / AirborneHRS**  
*Connect: [GitHub (@Ultron09)](https://github.com/Ultron09) • [Hugging Face (@Ultron28)](https://huggingface.co/Ultron28) • `Suryaansh@airbornehrs.in`*

</div>