pravikanth88's picture
Release CandleLLM commodity model weights and documentation
fc4932b verified
|
Raw History Blame Contribute Delete
3.63 kB
---
language:
- en
license: mit
library_name: pytorch
tags:
- time-series
- finance
- quantitative-trading
- causal-transformer
- commodities
- futures
- technical-analysis
metrics:
- rank_ic
- pinball_loss
- cross_entropy
---
# CandleLLM Commodities: Causal Transformer for Global Commodity Markets
CandleLLM is a compact, high-efficiency autoregressive causal transformer (~3.6M parameters) designed for financial time series, specifically trained on **28 major global commodities** spanning 20+ years of continuous historical data.
The model combines:
1. **Next-Candle Autoregressive Generation**: Using Finite Scalar Quantization (FSQ) tokenized candles (4,375 discrete tokens encoding body, range, upper wick, lower wick, and volume).
2. **Multi-Horizon Swing Quantile Prediction**: Simultaneous prediction of return distributions across **5-day, 10-day, and 20-day** forward horizons across 5 quantiles (`[0.10, 0.25, 0.50, 0.75, 0.90]`).
---
## 🌾 Commodity Universe (28 Major Markets)
- **Precious Metals**: Gold (`GC=F`), Silver (`SI=F`), Platinum (`PL=F`), Palladium (`PA=F`)
- **Industrial Metals**: High Grade Copper (`HG=F`), Aluminum (`ALI=F`)
- **Energy**: Crude Oil WTI (`CL=F`), Brent Crude (`BZ=F`), Natural Gas (`NG=F`), Heating Oil (`HO=F`), RBOB Gasoline (`RB=F`)
- **Grains & Oilseeds**: Corn (`ZC=F`), Soybeans (`ZS=F`), Soybean Oil (`ZL=F`), Soybean Meal (`ZM=F`), Chicago Wheat (`ZW=F`), KC HRW Wheat (`KE=F`), Rough Rice (`ZR=F`), Oats (`ZO=F`)
- **Softs & Livestock**: Cocoa (`CC=F`), Coffee (`KC=F`), Sugar #11 (`SB=F`), Cotton #2 (`CT=F`), Orange Juice (`OJ=F`), Lumber (`LBR=F`), Live Cattle (`LE=F`), Feeder Cattle (`GF=F`), Lean Hogs (`HE=F`)
---
## 🧠 Model Architecture & Features
- **Parameters**: 3,613,056 (~3.6M)
- **Layers**: 4 Transformer Blocks, 6 Attention Heads, `d_model=192`, Context Window: 128 bars
- **Engineered Features (54 Continuous + 12 Cyclical)**:
- **Candlestick Patterns**: Continuous hammer/pinbar score $[-1, 1]$, Doji intensity, Inside bar compression ratio, Outside bar expansion ratio, Penetration depth.
- **Moving Average Dynamics**: EMA 20/50 normalized spread, SMA 200 normalized distance, EMA 50 trend slope velocity.
- **Oscillators & Squeeze**: Donald Lambert Commodity Channel Index (CCI), Bollinger Bandwidth squeeze ratio, Fast Stochastic %K.
- **Turtle Trading Principles**: Richard Dennis System 1 (20-day) & System 2 (55-day) breakout channels, $N$-volatility expansion ratio, distance to $2N$ trailing stop.
- **Seasonality**: Annual 365.25-day harmonic sine/cosine curves capturing crop cycles and seasonal heating/cooling demand.
---
## 📊 Validation Metrics (Out-of-Sample 2022–2023)
- **Cross Entropy (CE)**: `6.6144` (vs Bigram `7.4244`, gap `-0.8100`)
- **Pinball Quantile Loss**: `0.2268` (Skill: `+0.89%` over climatology)
- **Median Rank Information Coefficient (IC)**:
- **5-Day Horizon**: `+0.035`
- **10-Day Horizon**: `+0.060`
- **20-Day Horizon**: `+0.094` (strong positive directional predictability)
---
## 🚀 Quickstart & Inference
```python
import torch
from src.model import CandleLLM
from src.features import compute_all_features
# Load model checkpoint
checkpoint = torch.load("checkpoints/commodity_swing_best.pt", map_location="cpu")
config = checkpoint["config"]
model = CandleLLM(**config)
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()
print(f"Loaded CandleLLM ({sum(p.numel() for p in model.parameters())/1e6:.1f}M params)")
```
Weights and tokenizers are released under the MIT License.