--- language: - en license: mit library_name: pytorch tags: - time-series - finance - quantitative-trading - causal-transformer - commodities - futures - technical-analysis metrics: - rank_ic - pinball_loss - cross_entropy --- # CandleLLM Commodities: Causal Transformer for Global Commodity Markets CandleLLM is a compact, high-efficiency autoregressive causal transformer (~3.6M parameters) designed for financial time series, specifically trained on **28 major global commodities** spanning 20+ years of continuous historical data. The model combines: 1. **Next-Candle Autoregressive Generation**: Using Finite Scalar Quantization (FSQ) tokenized candles (4,375 discrete tokens encoding body, range, upper wick, lower wick, and volume). 2. **Multi-Horizon Swing Quantile Prediction**: Simultaneous prediction of return distributions across **5-day, 10-day, and 20-day** forward horizons across 5 quantiles (`[0.10, 0.25, 0.50, 0.75, 0.90]`). --- ## 🌾 Commodity Universe (28 Major Markets) - **Precious Metals**: Gold (`GC=F`), Silver (`SI=F`), Platinum (`PL=F`), Palladium (`PA=F`) - **Industrial Metals**: High Grade Copper (`HG=F`), Aluminum (`ALI=F`) - **Energy**: Crude Oil WTI (`CL=F`), Brent Crude (`BZ=F`), Natural Gas (`NG=F`), Heating Oil (`HO=F`), RBOB Gasoline (`RB=F`) - **Grains & Oilseeds**: Corn (`ZC=F`), Soybeans (`ZS=F`), Soybean Oil (`ZL=F`), Soybean Meal (`ZM=F`), Chicago Wheat (`ZW=F`), KC HRW Wheat (`KE=F`), Rough Rice (`ZR=F`), Oats (`ZO=F`) - **Softs & Livestock**: Cocoa (`CC=F`), Coffee (`KC=F`), Sugar #11 (`SB=F`), Cotton #2 (`CT=F`), Orange Juice (`OJ=F`), Lumber (`LBR=F`), Live Cattle (`LE=F`), Feeder Cattle (`GF=F`), Lean Hogs (`HE=F`) --- ## 🧠 Model Architecture & Features - **Parameters**: 3,613,056 (~3.6M) - **Layers**: 4 Transformer Blocks, 6 Attention Heads, `d_model=192`, Context Window: 128 bars - **Engineered Features (54 Continuous + 12 Cyclical)**: - **Candlestick Patterns**: Continuous hammer/pinbar score $[-1, 1]$, Doji intensity, Inside bar compression ratio, Outside bar expansion ratio, Penetration depth. - **Moving Average Dynamics**: EMA 20/50 normalized spread, SMA 200 normalized distance, EMA 50 trend slope velocity. - **Oscillators & Squeeze**: Donald Lambert Commodity Channel Index (CCI), Bollinger Bandwidth squeeze ratio, Fast Stochastic %K. - **Turtle Trading Principles**: Richard Dennis System 1 (20-day) & System 2 (55-day) breakout channels, $N$-volatility expansion ratio, distance to $2N$ trailing stop. - **Seasonality**: Annual 365.25-day harmonic sine/cosine curves capturing crop cycles and seasonal heating/cooling demand. --- ## 📊 Validation Metrics (Out-of-Sample 2022–2023) - **Cross Entropy (CE)**: `6.6144` (vs Bigram `7.4244`, gap `-0.8100`) - **Pinball Quantile Loss**: `0.2268` (Skill: `+0.89%` over climatology) - **Median Rank Information Coefficient (IC)**: - **5-Day Horizon**: `+0.035` - **10-Day Horizon**: `+0.060` - **20-Day Horizon**: `+0.094` (strong positive directional predictability) --- ## 🚀 Quickstart & Inference ```python import torch from src.model import CandleLLM from src.features import compute_all_features # Load model checkpoint checkpoint = torch.load("checkpoints/commodity_swing_best.pt", map_location="cpu") config = checkpoint["config"] model = CandleLLM(**config) model.load_state_dict(checkpoint["model_state_dict"]) model.eval() print(f"Loaded CandleLLM ({sum(p.numel() for p in model.parameters())/1e6:.1f}M params)") ``` Weights and tokenizers are released under the MIT License.