File size: 13,793 Bytes
0cef53c 9402518 8dff74c e71731f 0cef53c 8dff74c 0cef53c 8dff74c 0cef53c 8dff74c e71731f 0cef53c df3eb75 0cef53c 9b40da9 722850b 8dff74c 9402518 8dff74c 9402518 8dff74c 0cef53c f43b1e5 0cef53c 9b40da9 0cef53c 9402518 8dff74c 9402518 0cef53c 973bea8 0cef53c 722850b 0cef53c 8dff74c 0cef53c 2f04f23 0cef53c e71731f 0821800 e71731f 0cef53c 2f04f23 0cef53c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 | # Using GNN4Colliders
GNN4Colliders trains event classifiers on reconstructed collider objects:
```text
ROOT files --prepare--> event cache (one row per object) --> graph or token batches --> model
```
## Install
Python 3.14 and PyTorch 2.14 (CUDA 13), managed by [uv](https://docs.astral.sh/uv/):
```bash
uv sync --dev # includes pytest, ruff and the ONNX packages
uv sync --dev --extra tabpfn # also TabPFN (configs/gamgam_tabpfn.yaml)
uv sync --dev --extra tabicl # also TabICL (configs/gamgam_tabicl.yaml)
uv sync --dev --extra tabdpt # also TabDPT (configs/gamgam_tabdpt.yaml)
uv sync --dev --extra plots # matplotlib, for scripts/plot_label_efficiency.py
uv run pytest # about 2 minutes; GPU tests run when CUDA is available
uv run pytest --device mps # the fixture runs on an Apple GPU
```
## Commands
Every command takes an experiment file and optional `dotted.key=value`
overrides (values are YAML):
```bash
uv run gnn4colliders prepare configs/gamgam.yaml # ROOT -> data.cache
uv run gnn4colliders train configs/gamgam.yaml # -> run_dir/{best,last}.pt
uv run gnn4colliders evaluate configs/gamgam.yaml # -> run_dir/test_metrics.json
uv run gnn4colliders predict configs/gamgam.yaml # -> run_dir/test_predictions.npz
uv run gnn4colliders export configs/gamgam.yaml # -> run_dir/model.onnx
uv run gnn4colliders probe configs/gamgam_jepa_finetune.yaml \
model.pretrained=outputs/delphes_jepa/last.pt data.train_events=1000 # -> run_dir/test_probe.json
```
`train` writes, in `run_dir` (default `outputs/<name>`):
* `config.yaml`: the resolved configuration;
* `history.jsonl`: one line per epoch with the loss and metrics of the
training and validation splits, the learning rate and the epoch time;
* `best.pt`: the epoch with the best validation `train.best_metric` (AUC by
default), `last.pt`: the latest epoch (resume with
`train.resume=<run_dir>/last.pt`), and `epoch_NNNN.pt` every
`train.save_every` epochs.
`probe` fits k-NN and linear classifiers to the frozen event embeddings of
the encoder of `model.pretrained` (JEPA, contrastive or supervised; random
without) on `data.train_events` training events and reports their metrics
on `split`, to compare pretraining without fine-tuning.
`data.eval_events=20000` evaluates (and validates during training,
predicts and probes) only that many random validation or test events,
seeded by `data.seed`: AUCs to about 0.003, in minutes, for quick runs.
`evaluate`, `predict` and `export` use `checkpoint` (default `best.pt`, else
`last.pt`) on `split` (default `test`); `output` overrides the output file.
Predictions hold the logits, scores, predicted class, label, weight, fold,
source file and entry of every event; `gnn4colliders.inference.write_root_scores`
adds scores to a copy of a ROOT tree.
## Experiments
| File | Task |
| --- | --- |
| `configs/gamgam.yaml` | ATLAS Open Data GamGam, 5 Higgs production modes, from scratch (the paper baseline and main benchmark) |
| `configs/gamgam_finetune.yaml` | the same task fine-tuned from the pretrained multiclass model |
| `configs/gamgam_transformer.yaml` | the same task with a transformer with pair attention bias (`model.pair_bias=false` for a plain transformer; same event cache) |
| `configs/gamgam_deep_sets.yaml` | the same task with Deep Sets, the baseline without object interactions (same event cache) |
| `configs/tth_cp_even_odd.yaml` | CP-even versus CP-odd ttH (Delphes, binary; local files) |
| `configs/delphes_pretraining.yaml` | 12-class pretraining on the Delphes samples |
| `configs/delphes_jepa.yaml` | JEPA pretraining of the transformer encoder on the same samples, without labels (same event cache) |
| `configs/delphes_jepa_edge.yaml` | the same for the edge network (fine-tune with `configs/gamgam_finetune.yaml model.pretrained=outputs/delphes_jepa_edge/last.pt`) |
| `configs/delphes_contrastive.yaml` | contrastive (SimCLR) pretraining of the transformer encoder on the same samples (`model.backbone=edge_network` for the edge network) |
| `configs/delphes_masked.yaml` | masked object modelling: JEPA predicting hidden objects' kinematics |
| `configs/gamgam_jepa_finetune.yaml` | GamGam with the JEPA-pretrained encoder (`model.pretrained=null`: from scratch); also takes contrastive and masked-object encoders, and `probe` |
| `configs/gamgam_tabpfn.yaml` | GamGam classified in context by TabPFN-3.5, no training (`evaluate` only) |
| `configs/gamgam_tabicl.yaml` | the same by TabICLv2 (BSD-3 weights, downloaded from Hugging Face on first use) |
| `configs/gamgam_tabdpt.yaml` | the same by TabDPT (Apache-2.0 weights; `model.tabdpt_context_size` retrieves each event's context) |
| `configs/delphes_pfn.yaml` | meta-training of our own PFN on tasks drawn from the Delphes samples (same event cache) |
| `configs/gamgam_pfn.yaml` | GamGam classified in context by that PFN, no training (`evaluate` only) |
| `configs/delphes_tabpfn.yaml` | meta-training of TabPFN-3.5 (LoRA by default) on the same tasks; evaluate with `configs/gamgam_tabpfn.yaml checkpoint=outputs/delphes_tabpfn/best.pt` |
| `configs/delphes_tabicl.yaml` | the same for TabICL; evaluate with `configs/gamgam_tabicl.yaml checkpoint=outputs/delphes_tabicl/best.pt` |
Each file documents its keys; `src/gnn4colliders/config.py` lists every key
with its default. Common overrides:
```bash
train.fast=true # torch.compile + CUDA graphs (edge network) + bfloat16 + TF32
train.seed=3 data.seed=3 # initialization and shuffling seeds
train.max_epochs=100
model.pretrained=Pretrained_GNN/multiclass_pretrained_model_12/model_epoch_71.pt
model.freeze_backbone=true # train only the new classifier
data.files=[{path: my.root, label: 0}] data.hf_repo=null # local inputs
```
### Data
Large inputs belong on scratch storage, not in the repository: set
`HF_HOME` (where Hugging Face downloads go) and pass `data.cache=...`, e.g.
on Perlmutter `HF_HOME=$SCRATCH/.cache/huggingface` and
`data.cache=$SCRATCH/gnn4colliders/delphes/events.pt` (the 105M Delphes
events take 52 GB of ROOT files and a 21 GB cache).
`data.files` lists the ROOT files and their class labels (or a label branch
name). With `data.hf_repo` the paths are inside that Hugging Face dataset at
`data.hf_revision` and are downloaded once and checked against `sha256` and
`size_bytes`.
Each object becomes a row of `pt, eta, phi, energy, btag, charge, node_type`
built from `feature_branches` (one branch or constant per object type,
`CALC_E` = pt cosh(eta), `NODE_TYPE` = object-type index) and multiplied by
`scales`. `derived_branches` select objects or threshold tagger scores before
that (the GamGam file splits `lep_*` into electrons and muons). The fold of an
event is `fold_var`, modulo `fold_modulus` when set, and
`train_folds`/`validation_folds`/`test_folds` select the splits. Events are
unweighted unless `weight_var` names a weight branch.
### In-context models and label efficiency
`data.train_events=N` keeps N training events, an equal number per class
(smaller N give subsets of larger ones). Supervised models train on them;
the in-context models (`model.name: tabpfn` or `pfn`) use them as the
labelled context and are only evaluated:
```bash
uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation data.train_events=1000
uv run gnn4colliders train configs/delphes_pfn.yaml # meta-training
uv run gnn4colliders evaluate configs/gamgam_pfn.yaml split=validation data.train_events=1000
uv run gnn4colliders train configs/gamgam.yaml data.train_events=1000 # supervised reference
```
`scripts/label_efficiency.py` runs such a sweep over several `N` and
configs (restartable) and prints validation AUC and seconds per run; the
metrics JSON records `seconds` (setup, inference) and, on CUDA,
`peak_memory_gib`:
```bash
uv run python scripts/label_efficiency.py configs/gamgam_tabpfn.yaml configs/gamgam.yaml \
--events 100 1000 10000 50000
```
TabPFN-2.5 and later weights are non-commercial (research and internal
benchmarking are allowed). tabpfn downloads them after a one-time login at
https://ux.priorlabs.ai (token in `~/.cache/tabpfn/auth_token` or
`TABPFN_TOKEN`), but a checkpoint already in its cache directory loads
without the login, so on compute nodes download it from Hugging Face:
```bash
export TABPFN_MODEL_CACHE_DIR=$SCRATCH/.cache/tabpfn # default ~/Library/Caches/tabpfn on macOS
uvx --from huggingface_hub hf download Prior-Labs/tabpfn_3_5 config.json \
tabpfn-v3.5-20260909.safetensors tabpfn-v3.5-20260909_multiclass.safetensors \
--local-dir $TABPFN_MODEL_CACHE_DIR
```
`model.tabpfn_checkpoint=tabpfn-v3.5-20260909_multiclass.safetensors`
selects the multiclass checkpoint. `model.tabpfn_version=v2` needs no login.
TabDPT retrieval (`model.tabdpt_context_size`) on macOS needs
`OMP_NUM_THREADS=1`: faiss and torch load separate OpenMP runtimes.
Tabular models can use a pretrained encoder's event embeddings instead of
or beside the rows: `model.tabular_inputs=embeddings` (or `both`) with
`model.pretrained=outputs/delphes_jepa/last.pt` and the encoder's sizes
(those of `configs/delphes_jepa.yaml`: `model.hid_size=128 model.n_layers=4
model.n_heads=8 model.ffn_size=512 model.pair_bias=true`).
TabPFN and TabICL classify the same table rows (`model.slots` objects of
each type). Two options add physics they cannot learn from a few events:
`model.relative_phi=true` measures every azimuth from the leading object's,
and `model.pair_objects=4` appends the invariant mass and Delta R of every
pair of the four leading objects (12 columns). Compare with and without,
one run name per variant:
```bash
uv run python scripts/label_efficiency.py configs/gamgam_tabicl.yaml --events 100 1000 \
--override name=gamgam_tabicl_physics model.relative_phi=true model.pair_objects=4
```
TabPFN-3.5 can also be meta-trained on the Delphes tasks, like our PFN.
By default only LoRA updates of its in-context blocks train
(`model.lora_rank`, 8; `model.lora_rank=0` trains all 219M weights), and
checkpoints hold only those (33 MB with the optimizer state, against
2.6 GB for all weights). With a `checkpoint`, TabPFN is
evaluated with its own forward pass, not the classifier:
```bash
uv run gnn4colliders train configs/delphes_tabpfn.yaml
uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation \
data.train_events=1000 checkpoint=outputs/delphes_tabpfn/best.pt
```
Activations are recomputed in the backward pass. On a 32 GB Apple M2 Max
one task of 2048 context events and 256 queries takes about 4.5 s and
16 GiB per step with all weights trained, so the Mac is for development;
meta-train on a GPU node.
### Fine-tuning
`Pretrained_GNN/` holds the pretrained models of the paper, the final
epoch of each: `multiclass_pretrained_model_12/model_epoch_71.pt` and
`multilabel_pretrained_model_41/model_epoch_44.pt` (with their training
configuration and log). With `model.pretrained` the edge network is loaded from
that checkpoint and gets a new `model.out_size` classifier; `model.hid_size`,
`n_layers` and `n_proc_steps` must match it (64, 4, 4). A JEPA checkpoint
(`configs/delphes_jepa*.yaml`) loads the same way: its context encoder
initializes the edge network or the transformer encoder.
## GPUs, Perlmutter and DDP
`train.device: auto` uses CUDA, else an Apple GPU (MPS), else the CPU. MPS
is for development on fixtures and small runs (no float64: metrics are
computed on the CPU; no DDP, CUDA graphs or TF32).
On a single GPU, `train.fast=true` is about 3x faster than float32 for the
GamGam model (see [design.md](design.md)). Set `OMP_NUM_THREADS` to the CPU
allocation and, on Perlmutter compute nodes, `UV_CACHE_DIR=$SCRATCH/.cache/uv`
(uv cannot lock its home-directory cache there).
```bash
# One training per seed, each followed by test-split evaluation:
for seed in 1 2 3 4 5; do
sbatch -A <account> --export=ALL,SEED=$seed scripts/train.sh train.fast=true
done
uv run python scripts/summarize_ensemble.py outputs/gamgam/seed* # mean +- std
```
The study of in-context models and pretraining has one wrapper per kind of
job, each on one GPU (the header of each script lists its variables):
```bash
# Label-efficiency sweep of released in-context models and supervised references:
sbatch -A <account> --export=ALL,CONFIGS="configs/gamgam_tabpfn.yaml configs/gamgam_tabicl.yaml configs/gamgam.yaml" \
scripts/label_efficiency.sh
# Meta-training, then GamGam in context with the meta-trained model:
sbatch -A <account> --export=ALL,CONFIG=configs/delphes_tabicl.yaml,EVAL_CONFIG=configs/gamgam_tabicl.yaml \
scripts/meta_train.sh task.prior_classes=[1,6,7,8,9,10,11]
# Label-free pretraining, then k-NN and linear probes on GamGam:
sbatch -A <account> --export=ALL,CONFIG=configs/delphes_contrastive.yaml scripts/pretrain.sh
# Linear CKA between the pretrained encoders, layer by layer (a login node is enough):
uv run python scripts/compare_encoders.py configs/gamgam_jepa_finetune.yaml --events 5000 \
--encoders random=random jepa=outputs/delphes_jepa/last.pt \
contrastive=outputs/delphes_contrastive/last.pt masked=outputs/delphes_masked/last.pt
# Curves and tables of every finished run:
uv run --extra plots python scripts/plot_label_efficiency.py --sweeps outputs/label_efficiency \
--probes outputs/delphes_jepa outputs/delphes_contrastive outputs/delphes_masked
```
Under `torchrun` with more than one process, training is data parallel:
`data.batch_size` is per process, training events are sharded over ranks
(padded to equal length) and metrics are computed over all ranks' events.
CUDA graphs need a single process (so `train.fast=true` does too; DDP is
tested in float32 eager). `scripts/train_ddp.sh` launches all GPUs of
one or more nodes.
|