| --- |
| license: cc-by-nc-4.0 |
| library_name: xgboost |
| tags: |
| - finance |
| - stock-market |
| - indian-stock-market |
| - nse |
| - bse |
| - tabular |
| - time-series |
| - xgboost |
| - classification |
| - indian-market |
| --- |
| |
| # XGBoost — Indian Stock Market Prediction |
|
|
| XGBoost binary classification model trained on historical Indian equity-market data to predict whether a stock reaches a specified positive-return threshold within a **5-trading-day horizon**. |
|
|
| The current model targets a **+3.0% return threshold** and was trained across thousands of Indian equities using technical, market, cross-sectional, and macroeconomic features. |
|
|
| > **Research status:** This repository contains a research model and its out-of-sample evaluation artifacts. The reported results should not be interpreted as a guarantee of future market performance or profitability. |
|
|
| --- |
|
|
| ## Model Overview |
|
|
| | Property | Value | |
| | ----------------------- | ------------------------------------ | |
| | Model | XGBoost | |
| | Task | Binary classification | |
| | Prediction target | ≥ +3.0% return within 5 trading days | |
| | Dataset size | ~6.92 million rows | |
| | Symbols | 2,656 | |
| | Historical period | 2008-09-02 → 2026-06-30 | |
| | Features | 221 | |
| | Positive class | 36.8% | |
| | Negative class | 63.2% | |
| | `scale_pos_weight` | 1.49 | |
| | Training method | Histogram-based XGBoost | |
| | Hardware | CUDA GPU | |
| | Maximum boosting rounds | 2,000 | |
| | Learning rate | 0.02 | |
| | Maximum tree depth | 6 | |
| | Early stopping | 75 rounds | |
|
|
| --- |
|
|
| # Dataset |
|
|
| The training dataset contains approximately **6.92 million observations** covering **2,656 Indian stock symbols** from September 2008 through June 2026. |
|
|
| The model uses 221 input features derived from historical market data and broader market conditions. |
|
|
| The feature groups include: |
|
|
| * Price returns |
| * Momentum |
| * Trend indicators |
| * Volatility |
| * Technical indicators |
| * Relative strength |
| * Market breadth |
| * Cross-sectional statistics |
| * Nifty relationships |
| * Gold relationships |
| * Brent crude relationships |
| * USD/INR relationships |
| * US-market indicators |
|
|
| The target is a binary label representing whether the specified positive-return threshold is reached within the prediction horizon. |
|
|
| --- |
|
|
| # Temporal Dataset Split |
|
|
| The dataset was divided chronologically rather than randomly. |
|
|
| | Split | Period | Approx. Share | |
| | ----------------- | ----------------------- | ------------: | |
| | Training | 2008-09-02 → 2021-02-15 | 70% | |
| | Validation | 2021-02-16 → 2024-08-30 | 20% | |
| | Walk-forward test | 2024-09-01 → 2026-06-30 | 10% | |
|
|
| This temporal separation is intended to reduce leakage from randomly mixing observations from different points in time. |
|
|
| The final test period was not used for model fitting. |
|
|
| --- |
|
|
| # Validation Performance |
|
|
| The model achieved the following results on the held-out validation period: |
|
|
| **2021-02-16 → 2024-08-30** |
|
|
| | Metric | Score | |
| | ---------------------------------- | ---------: | |
| | ROC-AUC | **0.9472** | |
| | PR-AUC | **0.9042** | |
| | Precision | **0.7981** | |
| | Recall | **0.8364** | |
| | Information Coefficient (Spearman) | **0.7140** | |
| | Brier Score | **0.0835** | |
| | Brier Skill Score | **0.6068** | |
| | Mean Calibration Error | **0.0749** | |
|
|
| Validation observations: |
|
|
| **1,698,058 rows** |
|
|
| The validation results indicate strong separation between the positive and negative classes on this historical period. |
|
|
| However, validation performance alone should not be treated as evidence of future profitability. |
|
|
| --- |
|
|
| # Validation Decile Analysis |
|
|
| Predictions were divided into ten probability-ranked groups. |
|
|
| | Decile | Hit Rate | Samples | |
| | -----: | -------: | ------: | |
| | D10 | 98.4% | 169,806 | |
| | D9 | 88.8% | 169,806 | |
| | D8 | 60.5% | 169,806 | |
| | D7 | 31.1% | 169,805 | |
| | D6 | 14.6% | 169,806 | |
| | D5 | 6.6% | 169,806 | |
| | D4 | 3.1% | 169,805 | |
| | D3 | 1.7% | 169,806 | |
| | D2 | 0.9% | 169,806 | |
| | D1 | 0.4% | 169,806 | |
|
|
| The probability ranking shows strong separation across the validation sample, with substantially different observed positive rates between the lowest- and highest-ranked groups. |
|
|
| These figures describe historical classification performance and do not account for transaction costs, slippage, liquidity constraints, portfolio construction, or market impact. |
|
|
| --- |
|
|
| # Out-of-Sample Walk-Forward Evaluation |
|
|
| The model was subsequently evaluated chronologically over the final portion of the dataset. |
|
|
| The test period was divided into four chronological folds of approximately six months each. |
|
|
| | Fold | Period | PR-AUC | IC | Brier | Samples | |
| | ---- | ----------------------- | ---------: | ---------: | -----: | ------: | |
| | 1 | 2024-09-01 → 2025-02-28 | **0.9336** | **0.7619** | 0.0850 | 262,062 | |
| | 2 | 2025-03-01 → 2025-08-31 | **0.8987** | **0.7044** | 0.0829 | 266,786 | |
| | 3 | 2025-09-01 → 2026-02-28 | **0.8921** | **0.7011** | 0.0845 | 282,585 | |
| | 4 | 2026-03-01 → 2026-06-30 | **0.8962** | **0.7157** | 0.0925 | 180,837 | |
|
|
| ### Walk-Forward Summary |
|
|
| | Statistic | Value | |
| | ---------------- | ---------: | |
| | Mean PR-AUC | **0.9052** | |
| | Mean IC | **0.7208** | |
| | Mean Brier Score | **0.0862** | |
| | Minimum IC | **0.7011** | |
| | Maximum IC | **0.7619** | |
|
|
| The model maintained relatively strong classification performance across all four chronological folds. |
|
|
| Performance did, however, vary between periods. The difference between the highest and lowest observed IC is approximately **0.061**. |
|
|
| This suggests that model performance is not completely invariant across market regimes and warrants additional robustness testing. |
|
|
| --- |
|
|
| # Overfitting & Generalization |
|
|
| The model was evaluated using a chronological validation set followed by a later out-of-sample test period. |
|
|
| The results provide evidence that the model retains predictive separation outside its training period. |
|
|
| However, the current experiment does **not** establish that the model is free from overfitting. |
|
|
| In particular: |
|
|
| * Validation performance is very strong. |
| * Walk-forward test performance remains strong. |
| * Performance varies between chronological folds. |
| * The model reached the maximum configured **2,000 boosting rounds**, with the best iteration at **1,999**. |
| * Early stopping therefore did not activate before the configured maximum number of rounds. |
|
|
| The last point means that additional experiments with a larger maximum number of boosting rounds should be evaluated carefully rather than assuming that more trees will improve generalization. |
|
|
| Future experiments should compare performance across additional temporal folds and assess whether increasing model complexity improves out-of-sample performance or simply improves the validation period. |
|
|
| --- |
|
|
| # Calibration |
|
|
| The validation set produced: |
|
|
| * Brier Score: **0.0835** |
| * Brier Skill Score: **0.6068** |
| * Mean Calibration Error: **0.0749** |
|
|
| The model therefore produces probability scores that contain useful information, but the raw probabilities should not automatically be interpreted as perfectly calibrated real-world probabilities. |
|
|
| Probability calibration should be evaluated separately if the model's output is to be interpreted probabilistically. |
|
|
| --- |
|
|
| # Feature Set |
|
|
| The model uses 221 features. |
|
|
| Examples of macro and cross-market features include: |
|
|
| ```text |
| usd_inr_sma20_ratio |
| usd_inr_sma200_ratio |
| usd_inr_momentum_10 |
| usd_inr_momentum_20 |
| usd_inr_volatility_20d |
| usd_inr_trend |
| usd_inr_zscore |
| usd_inr_vol_zscore |
| usd_inr_high_vol |
| usd_inr_appreciation |
| |
| us_ret_1d |
| us_ret_5d |
| us_ret_20d |
| us_ret_60d |
| |
| us_breadth_1d |
| us_breadth_5d |
| us_breadth_20d |
| |
| us_dispersion_1d |
| us_dispersion_5d |
| us_avg_vol_ratio |
| us_pct_above_sma20 |
| us_avg_rsi |
| us_avg_dist_52w_high |
| |
| corr_nifty_20d |
| corr_nifty_60d |
| beta_nifty_60d |
| is_high_beta |
| is_low_beta |
| corr_breakdown |
| |
| rel_strength_5d |
| rel_strength_20d |
| |
| corr_gold_60d |
| corr_gold_rising |
| |
| corr_brent_60d |
| corr_brent_rising |
| |
| corr_usd_inr_60d |
| corr_usd_inr_rising |
| |
| corr_us_60d |
| ``` |
|
|
| The complete feature schema is available in `model_metadata.json`. |
|
|
| --- |
|
|
| # Model Artifacts |
|
|
| The repository contains the following artifacts: |
|
|
| | File | Description | |
| | ----------------------------- | --------------------------------------------------------------------------------- | |
| | `xgb_model.json` | Trained XGBoost model | |
| | `model_metadata.json` | Training configuration, feature schema, split information and evaluation metadata | |
| | `feature_importance.csv` | XGBoost feature-importance statistics | |
| | `val_predictions.parquet` | Validation predictions | |
| | `test_wf_predictions.parquet` | Chronological walk-forward test predictions | |
|
|
| The prediction files are provided to make the reported evaluation results independently inspectable and reproducible. |
|
|
| --- |
|
|
| # Reproducibility |
|
|
| The model metadata records: |
|
|
| * Feature names |
| * Feature count |
| * Dataset information |
| * Training period |
| * Validation period |
| * Model parameters |
| * Class weighting |
| * Best iteration |
| * Evaluation metrics |
| * Training timestamp |
|
|
| The model uses a fixed random seed of `42`. |
|
|
| --- |
|
|
| # Important Limitations |
|
|
| This model should be considered a **research artifact**, not a guaranteed trading system. |
|
|
| The reported classification metrics do not directly measure: |
|
|
| * Portfolio returns |
| * Sharpe ratio |
| * Maximum drawdown |
| * Transaction costs |
| * Brokerage |
| * Taxes |
| * Slippage |
| * Bid/ask spread |
| * Market impact |
| * Position sizing |
| * Portfolio concentration |
| * Liquidity constraints |
| * Capacity |
| * Execution latency |
|
|
| A model can achieve strong classification metrics while producing poor investment returns after these factors are considered. |
|
|
| The target is also a classification threshold rather than a direct optimization of portfolio returns. |
|
|
| Additional research is therefore required before using the model for live decision-making. |
|
|
| --- |
|
|
| # Current Research Status |
|
|
| The current experiment establishes a useful baseline for the **+3.0% / 5-trading-day** prediction task. |
|
|
| The next stages of research should include: |
|
|
| 1. Additional temporal walk-forward experiments. |
| 2. Comparison of model complexity and boosting-round counts. |
| 3. Testing alternative return thresholds and horizons. |
| 4. Feature ablation and importance stability analysis. |
| 5. Leakage and feature-timing audits. |
| 6. Performance analysis by market regime. |
| 7. Calibration analysis on completely unseen periods. |
| 8. Portfolio-level backtesting. |
| 9. Transaction-cost and slippage modelling. |
| 10. Paper-trading validation before considering live deployment. |
|
|
| --- |
|
|
| # License |
|
|
| This model is published under the **Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)** license. |
|
|
| The license permits non-commercial use subject to the terms of the license. |
|
|
| Commercial use requires appropriate licensing. |
|
|
| --- |
|
|
| # Author |
|
|
| [**Sayantan Basu**](https://sayantan-basu.vercel.app) |
|
|
