Download docs/usage.md from HWresearch/GNN4Colliders: direct link, hf CLI and curl.
- Browser
- Download file 13.8 kB
-
https://huggingface.co/HWresearch/GNN4Colliders/resolve/main/docs/usage.md
- Command line
-
hf download hf://HWresearch/GNN4Colliders/docs/usage.md
-
curl -L -o usage.md https://huggingface.co/HWresearch/GNN4Colliders/resolve/main/docs/usage.md
Using GNN4Colliders
GNN4Colliders trains event classifiers on reconstructed collider objects:
ROOT files --prepare--> event cache (one row per object) --> graph or token batches --> model
Install
Python 3.14 and PyTorch 2.14 (CUDA 13), managed by uv:
uv sync --dev # includes pytest, ruff and the ONNX packages
uv sync --dev --extra tabpfn # also TabPFN (configs/gamgam_tabpfn.yaml)
uv sync --dev --extra tabicl # also TabICL (configs/gamgam_tabicl.yaml)
uv sync --dev --extra tabdpt # also TabDPT (configs/gamgam_tabdpt.yaml)
uv sync --dev --extra plots # matplotlib, for scripts/plot_label_efficiency.py
uv run pytest # about 2 minutes; GPU tests run when CUDA is available
uv run pytest --device mps # the fixture runs on an Apple GPU
Commands
Every command takes an experiment file and optional dotted.key=value
overrides (values are YAML):
uv run gnn4colliders prepare configs/gamgam.yaml # ROOT -> data.cache
uv run gnn4colliders train configs/gamgam.yaml # -> run_dir/{best,last}.pt
uv run gnn4colliders evaluate configs/gamgam.yaml # -> run_dir/test_metrics.json
uv run gnn4colliders predict configs/gamgam.yaml # -> run_dir/test_predictions.npz
uv run gnn4colliders export configs/gamgam.yaml # -> run_dir/model.onnx
uv run gnn4colliders probe configs/gamgam_jepa_finetune.yaml \
model.pretrained=outputs/delphes_jepa/last.pt data.train_events=1000 # -> run_dir/test_probe.json
train writes, in run_dir (default outputs/<name>):
config.yaml: the resolved configuration;history.jsonl: one line per epoch with the loss and metrics of the training and validation splits, the learning rate and the epoch time;best.pt: the epoch with the best validationtrain.best_metric(AUC by default),last.pt: the latest epoch (resume withtrain.resume=<run_dir>/last.pt), andepoch_NNNN.pteverytrain.save_everyepochs.
probe fits k-NN and linear classifiers to the frozen event embeddings of
the encoder of model.pretrained (JEPA, contrastive or supervised; random
without) on data.train_events training events and reports their metrics
on split, to compare pretraining without fine-tuning.
data.eval_events=20000 evaluates (and validates during training,
predicts and probes) only that many random validation or test events,
seeded by data.seed: AUCs to about 0.003, in minutes, for quick runs.
evaluate, predict and export use checkpoint (default best.pt, else
last.pt) on split (default test); output overrides the output file.
Predictions hold the logits, scores, predicted class, label, weight, fold,
source file and entry of every event; gnn4colliders.inference.write_root_scores
adds scores to a copy of a ROOT tree.
Experiments
| File | Task |
|---|---|
configs/gamgam.yaml |
ATLAS Open Data GamGam, 5 Higgs production modes, from scratch (the paper baseline and main benchmark) |
configs/gamgam_finetune.yaml |
the same task fine-tuned from the pretrained multiclass model |
configs/gamgam_transformer.yaml |
the same task with a transformer with pair attention bias (model.pair_bias=false for a plain transformer; same event cache) |
configs/gamgam_deep_sets.yaml |
the same task with Deep Sets, the baseline without object interactions (same event cache) |
configs/tth_cp_even_odd.yaml |
CP-even versus CP-odd ttH (Delphes, binary; local files) |
configs/delphes_pretraining.yaml |
12-class pretraining on the Delphes samples |
configs/delphes_jepa.yaml |
JEPA pretraining of the transformer encoder on the same samples, without labels (same event cache) |
configs/delphes_jepa_edge.yaml |
the same for the edge network (fine-tune with configs/gamgam_finetune.yaml model.pretrained=outputs/delphes_jepa_edge/last.pt) |
configs/delphes_contrastive.yaml |
contrastive (SimCLR) pretraining of the transformer encoder on the same samples (model.backbone=edge_network for the edge network) |
configs/delphes_masked.yaml |
masked object modelling: JEPA predicting hidden objects' kinematics |
configs/gamgam_jepa_finetune.yaml |
GamGam with the JEPA-pretrained encoder (model.pretrained=null: from scratch); also takes contrastive and masked-object encoders, and probe |
configs/gamgam_tabpfn.yaml |
GamGam classified in context by TabPFN-3.5, no training (evaluate only) |
configs/gamgam_tabicl.yaml |
the same by TabICLv2 (BSD-3 weights, downloaded from Hugging Face on first use) |
configs/gamgam_tabdpt.yaml |
the same by TabDPT (Apache-2.0 weights; model.tabdpt_context_size retrieves each event's context) |
configs/delphes_pfn.yaml |
meta-training of our own PFN on tasks drawn from the Delphes samples (same event cache) |
configs/gamgam_pfn.yaml |
GamGam classified in context by that PFN, no training (evaluate only) |
configs/delphes_tabpfn.yaml |
meta-training of TabPFN-3.5 (LoRA by default) on the same tasks; evaluate with configs/gamgam_tabpfn.yaml checkpoint=outputs/delphes_tabpfn/best.pt |
configs/delphes_tabicl.yaml |
the same for TabICL; evaluate with configs/gamgam_tabicl.yaml checkpoint=outputs/delphes_tabicl/best.pt |
Each file documents its keys; src/gnn4colliders/config.py lists every key
with its default. Common overrides:
train.fast=true # torch.compile + CUDA graphs (edge network) + bfloat16 + TF32
train.seed=3 data.seed=3 # initialization and shuffling seeds
train.max_epochs=100
model.pretrained=Pretrained_GNN/multiclass_pretrained_model_12/model_epoch_71.pt
model.freeze_backbone=true # train only the new classifier
data.files=[{path: my.root, label: 0}] data.hf_repo=null # local inputs
Data
Large inputs belong on scratch storage, not in the repository: set
HF_HOME (where Hugging Face downloads go) and pass data.cache=..., e.g.
on Perlmutter HF_HOME=$SCRATCH/.cache/huggingface and
data.cache=$SCRATCH/gnn4colliders/delphes/events.pt (the 105M Delphes
events take 52 GB of ROOT files and a 21 GB cache).
data.files lists the ROOT files and their class labels (or a label branch
name). With data.hf_repo the paths are inside that Hugging Face dataset at
data.hf_revision and are downloaded once and checked against sha256 and
size_bytes.
Each object becomes a row of pt, eta, phi, energy, btag, charge, node_type
built from feature_branches (one branch or constant per object type,
CALC_E = pt cosh(eta), NODE_TYPE = object-type index) and multiplied by
scales. derived_branches select objects or threshold tagger scores before
that (the GamGam file splits lep_* into electrons and muons). The fold of an
event is fold_var, modulo fold_modulus when set, and
train_folds/validation_folds/test_folds select the splits. Events are
unweighted unless weight_var names a weight branch.
In-context models and label efficiency
data.train_events=N keeps N training events, an equal number per class
(smaller N give subsets of larger ones). Supervised models train on them;
the in-context models (model.name: tabpfn or pfn) use them as the
labelled context and are only evaluated:
uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation data.train_events=1000
uv run gnn4colliders train configs/delphes_pfn.yaml # meta-training
uv run gnn4colliders evaluate configs/gamgam_pfn.yaml split=validation data.train_events=1000
uv run gnn4colliders train configs/gamgam.yaml data.train_events=1000 # supervised reference
scripts/label_efficiency.py runs such a sweep over several N and
configs (restartable) and prints validation AUC and seconds per run; the
metrics JSON records seconds (setup, inference) and, on CUDA,
peak_memory_gib:
uv run python scripts/label_efficiency.py configs/gamgam_tabpfn.yaml configs/gamgam.yaml \
--events 100 1000 10000 50000
TabPFN-2.5 and later weights are non-commercial (research and internal
benchmarking are allowed). tabpfn downloads them after a one-time login at
https://ux.priorlabs.ai (token in ~/.cache/tabpfn/auth_token or
TABPFN_TOKEN), but a checkpoint already in its cache directory loads
without the login, so on compute nodes download it from Hugging Face:
export TABPFN_MODEL_CACHE_DIR=$SCRATCH/.cache/tabpfn # default ~/Library/Caches/tabpfn on macOS
uvx --from huggingface_hub hf download Prior-Labs/tabpfn_3_5 config.json \
tabpfn-v3.5-20260909.safetensors tabpfn-v3.5-20260909_multiclass.safetensors \
--local-dir $TABPFN_MODEL_CACHE_DIR
model.tabpfn_checkpoint=tabpfn-v3.5-20260909_multiclass.safetensors
selects the multiclass checkpoint. model.tabpfn_version=v2 needs no login.
TabDPT retrieval (model.tabdpt_context_size) on macOS needs
OMP_NUM_THREADS=1: faiss and torch load separate OpenMP runtimes.
Tabular models can use a pretrained encoder's event embeddings instead of
or beside the rows: model.tabular_inputs=embeddings (or both) with
model.pretrained=outputs/delphes_jepa/last.pt and the encoder's sizes
(those of configs/delphes_jepa.yaml: model.hid_size=128 model.n_layers=4 model.n_heads=8 model.ffn_size=512 model.pair_bias=true).
TabPFN and TabICL classify the same table rows (model.slots objects of
each type). Two options add physics they cannot learn from a few events:
model.relative_phi=true measures every azimuth from the leading object's,
and model.pair_objects=4 appends the invariant mass and Delta R of every
pair of the four leading objects (12 columns). Compare with and without,
one run name per variant:
uv run python scripts/label_efficiency.py configs/gamgam_tabicl.yaml --events 100 1000 \
--override name=gamgam_tabicl_physics model.relative_phi=true model.pair_objects=4
TabPFN-3.5 can also be meta-trained on the Delphes tasks, like our PFN.
By default only LoRA updates of its in-context blocks train
(model.lora_rank, 8; model.lora_rank=0 trains all 219M weights), and
checkpoints hold only those (33 MB with the optimizer state, against
2.6 GB for all weights). With a checkpoint, TabPFN is
evaluated with its own forward pass, not the classifier:
uv run gnn4colliders train configs/delphes_tabpfn.yaml
uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation \
data.train_events=1000 checkpoint=outputs/delphes_tabpfn/best.pt
Activations are recomputed in the backward pass. On a 32 GB Apple M2 Max one task of 2048 context events and 256 queries takes about 4.5 s and 16 GiB per step with all weights trained, so the Mac is for development; meta-train on a GPU node.
Fine-tuning
Pretrained_GNN/ holds the pretrained models of the paper, the final
epoch of each: multiclass_pretrained_model_12/model_epoch_71.pt and
multilabel_pretrained_model_41/model_epoch_44.pt (with their training
configuration and log). With model.pretrained the edge network is loaded from
that checkpoint and gets a new model.out_size classifier; model.hid_size,
n_layers and n_proc_steps must match it (64, 4, 4). A JEPA checkpoint
(configs/delphes_jepa*.yaml) loads the same way: its context encoder
initializes the edge network or the transformer encoder.
GPUs, Perlmutter and DDP
train.device: auto uses CUDA, else an Apple GPU (MPS), else the CPU. MPS
is for development on fixtures and small runs (no float64: metrics are
computed on the CPU; no DDP, CUDA graphs or TF32).
On a single GPU, train.fast=true is about 3x faster than float32 for the
GamGam model (see design.md). Set OMP_NUM_THREADS to the CPU
allocation and, on Perlmutter compute nodes, UV_CACHE_DIR=$SCRATCH/.cache/uv
(uv cannot lock its home-directory cache there).
# One training per seed, each followed by test-split evaluation:
for seed in 1 2 3 4 5; do
sbatch -A <account> --export=ALL,SEED=$seed scripts/train.sh train.fast=true
done
uv run python scripts/summarize_ensemble.py outputs/gamgam/seed* # mean +- std
The study of in-context models and pretraining has one wrapper per kind of job, each on one GPU (the header of each script lists its variables):
# Label-efficiency sweep of released in-context models and supervised references:
sbatch -A <account> --export=ALL,CONFIGS="configs/gamgam_tabpfn.yaml configs/gamgam_tabicl.yaml configs/gamgam.yaml" \
scripts/label_efficiency.sh
# Meta-training, then GamGam in context with the meta-trained model:
sbatch -A <account> --export=ALL,CONFIG=configs/delphes_tabicl.yaml,EVAL_CONFIG=configs/gamgam_tabicl.yaml \
scripts/meta_train.sh task.prior_classes=[1,6,7,8,9,10,11]
# Label-free pretraining, then k-NN and linear probes on GamGam:
sbatch -A <account> --export=ALL,CONFIG=configs/delphes_contrastive.yaml scripts/pretrain.sh
# Linear CKA between the pretrained encoders, layer by layer (a login node is enough):
uv run python scripts/compare_encoders.py configs/gamgam_jepa_finetune.yaml --events 5000 \
--encoders random=random jepa=outputs/delphes_jepa/last.pt \
contrastive=outputs/delphes_contrastive/last.pt masked=outputs/delphes_masked/last.pt
# Curves and tables of every finished run:
uv run --extra plots python scripts/plot_label_efficiency.py --sweeps outputs/label_efficiency \
--probes outputs/delphes_jepa outputs/delphes_contrastive outputs/delphes_masked
Under torchrun with more than one process, training is data parallel:
data.batch_size is per process, training events are sharded over ranks
(padded to equal length) and metrics are computed over all ranks' events.
CUDA graphs need a single process (so train.fast=true does too; DDP is
tested in float32 eager). scripts/train_ddp.sh launches all GPUs of
one or more nodes.