IRG XGBoost baselines, wider search (notebook 11)
Fifteen gradient-boosted classifiers (3 biosamples x 5 seeds) that predict whether a 1,000 bp window is an ENCODE candidate
cis-regulatory element (cCRE: PLS, pELS or dELS) of one biosample, against GC-matched genomic background, from 6-mer
frequencies (4,096 features). Trained on PxALabs/encode-regulatory-elements v2.1, *_gc_matched tables. Part of
github.com/PxA-Labs/interpretable-regulatory-genomics. No pretrained or language models are involved. This is the successor to
PxALabs/irg-xgboost-baselines (notebook 09): a wider hyperparameter search and the tree cap lifted from 1,500 to 8,000.
Files
weights/<biosample>/seed<N>.ubj- XGBoost native binary models (tcell,k562,gm12878; seeds 0-4); no picklexgb_features.py- the feature function (6-mer frequencies; windows containing N are skipped)manifest.json- per file: SHA-256, size, selected hyperparameters, best iteration, validation and test AUROC;MANIFEST.sha256
Usage
import xgboost as xgb
from xgb_features import kmer_features
clf = xgb.XGBClassifier(); clf.load_model("weights/k562/seed0.ubj")
p = clf.predict_proba(kmer_features(["ACGT" * 250], k=6))[:, 1] # probability of being a K562 cCRE
Use all five seeds and report the spread; do not pick a seed on test data.
Training
Selection on the validation chromosomes only (early stopping on validation AUC, patience 100). Stage A: the 3 best notebook-09
configurations per biosample re-run with up to 8,000 trees. Stage B: 8 new random configurations (space seed 777; max_depth {6, 8, 10},
learning_rate {0.05, 0.08}, subsample {0.6, 0.8, 1.0}, colsample_bytree {0.1, 0.2, 0.3}, min_child_weight {1, 5, 10}, reg_lambda
{20, 50, 100}; up to 4,000 trees). Final models: best validation configuration per biosample, 5 seeds, up to 8,000 trees.
Selected: tcell and k562 - depth 8, lr 0.05, subsample 0.6, colsample 0.1, min_child_weight 5, lambda 100; gm12878 - depth 8, lr 0.05,
subsample 1.0, colsample 0.3, min_child_weight 1, lambda 20. Best iteration 1,780-3,096 (no model hit the cap).
Split: train chr1-14, X, Y / validation chr15-18 / test chr19-22. Test chromosomes were never used for selection.
Source: Kaggle kernel purvanshj009/irg-11-xgb-wide v2, XGBoost 3.4.1 on GPU.
Evaluation (test chromosomes, mean over 5 seeds; SD in brackets)
"Matched" = test subset matched on GC and CpG observed/expected, equal positives and negatives per bin (a sub-population; not comparable with the full-task AUROC).
| biosample | test AUROC | matched AUROC | previous (nb09) |
|---|---|---|---|
| tcell | 0.8911 (0.0009) | 0.7458 (0.0033) | 0.8894 / 0.7431 |
| k562 | 0.8710 (0.0008) | 0.7724 (0.0019) | 0.8695 / 0.7686 |
| gm12878 | 0.8456 (0.0004) | 0.7575 (0.0006) | 0.8429 / 0.7526 |
Against the 5-seed means of the tuned CNNs (PxALabs/irg-cnn-baselines), these models are ahead of ShallowCNN on T cell and K562
(0.005 and 0.004 full AUROC) and about level or slightly behind on GM12878 (ShallowCNN +0.001 full, +0.007 matched). Full analysis and
caveats: docs/results/xgb_wide.md in the GitHub repository.
Intended use and limitations
- Reference baselines for sequence-based cCRE classification. Not for clinical use or variant-effect prediction.
- Each model is specific to the biosample it was trained on; cross-biosample use is not covered here.
- The negatives are GC-matched genomic windows outside the biosample's cCREs, not validated non-regulatory elements.
- The search is still truncated (3 + 8 configurations per biosample, one seed); the selected configurations again sit near the edge of the search space, so a further gain of about 0.001 is plausible. Only k=6 was tuned.
- Composition shortcuts (GC, CpG) are controlled only in the matched evaluation; repeat content is not controlled in training.
- Data note: the project's earlier tables named
k562_*were T-cell elements; these models use the corrected v2.1 tables.
Provenance and integrity
All 15 files were reloaded and their best iteration checked against the run log. All 15 were re-scored on the real test rows with
XGBoost 2.1.4 on CPU (files were written by 3.4.1) and reproduced the AUROC recorded during the run to four decimals
(tools/hf/verify_xgb_models.py). SHA-256 for every file is in manifest.json and MANIFEST.sha256.