|
Download docs/usage.md from HWresearch/GNN4Colliders: direct link, hf CLI and curl.
- Browser
- Download file 13.8 kB
-
https://huggingface.co/HWresearch/GNN4Colliders/resolve/main/docs/usage.md
- Command line
-
hf download hf://HWresearch/GNN4Colliders/docs/usage.md
-
curl -L -o usage.md https://huggingface.co/HWresearch/GNN4Colliders/resolve/main/docs/usage.md
13.8 kB
| # Using GNN4Colliders | |
| GNN4Colliders trains event classifiers on reconstructed collider objects: | |
| ```text | |
| ROOT files --prepare--> event cache (one row per object) --> graph or token batches --> model | |
| ``` | |
| ## Install | |
| Python 3.14 and PyTorch 2.14 (CUDA 13), managed by [uv](https://docs.astral.sh/uv/): | |
| ```bash | |
| uv sync --dev # includes pytest, ruff and the ONNX packages | |
| uv sync --dev --extra tabpfn # also TabPFN (configs/gamgam_tabpfn.yaml) | |
| uv sync --dev --extra tabicl # also TabICL (configs/gamgam_tabicl.yaml) | |
| uv sync --dev --extra tabdpt # also TabDPT (configs/gamgam_tabdpt.yaml) | |
| uv sync --dev --extra plots # matplotlib, for scripts/plot_label_efficiency.py | |
| uv run pytest # about 2 minutes; GPU tests run when CUDA is available | |
| uv run pytest --device mps # the fixture runs on an Apple GPU | |
| ``` | |
| ## Commands | |
| Every command takes an experiment file and optional `dotted.key=value` | |
| overrides (values are YAML): | |
| ```bash | |
| uv run gnn4colliders prepare configs/gamgam.yaml # ROOT -> data.cache | |
| uv run gnn4colliders train configs/gamgam.yaml # -> run_dir/{best,last}.pt | |
| uv run gnn4colliders evaluate configs/gamgam.yaml # -> run_dir/test_metrics.json | |
| uv run gnn4colliders predict configs/gamgam.yaml # -> run_dir/test_predictions.npz | |
| uv run gnn4colliders export configs/gamgam.yaml # -> run_dir/model.onnx | |
| uv run gnn4colliders probe configs/gamgam_jepa_finetune.yaml \ | |
| model.pretrained=outputs/delphes_jepa/last.pt data.train_events=1000 # -> run_dir/test_probe.json | |
| ``` | |
| `train` writes, in `run_dir` (default `outputs/<name>`): | |
| * `config.yaml`: the resolved configuration; | |
| * `history.jsonl`: one line per epoch with the loss and metrics of the | |
| training and validation splits, the learning rate and the epoch time; | |
| * `best.pt`: the epoch with the best validation `train.best_metric` (AUC by | |
| default), `last.pt`: the latest epoch (resume with | |
| `train.resume=<run_dir>/last.pt`), and `epoch_NNNN.pt` every | |
| `train.save_every` epochs. | |
| `probe` fits k-NN and linear classifiers to the frozen event embeddings of | |
| the encoder of `model.pretrained` (JEPA, contrastive or supervised; random | |
| without) on `data.train_events` training events and reports their metrics | |
| on `split`, to compare pretraining without fine-tuning. | |
| `data.eval_events=20000` evaluates (and validates during training, | |
| predicts and probes) only that many random validation or test events, | |
| seeded by `data.seed`: AUCs to about 0.003, in minutes, for quick runs. | |
| `evaluate`, `predict` and `export` use `checkpoint` (default `best.pt`, else | |
| `last.pt`) on `split` (default `test`); `output` overrides the output file. | |
| Predictions hold the logits, scores, predicted class, label, weight, fold, | |
| source file and entry of every event; `gnn4colliders.inference.write_root_scores` | |
| adds scores to a copy of a ROOT tree. | |
| ## Experiments | |
| | File | Task | | |
| | --- | --- | | |
| | `configs/gamgam.yaml` | ATLAS Open Data GamGam, 5 Higgs production modes, from scratch (the paper baseline and main benchmark) | | |
| | `configs/gamgam_finetune.yaml` | the same task fine-tuned from the pretrained multiclass model | | |
| | `configs/gamgam_transformer.yaml` | the same task with a transformer with pair attention bias (`model.pair_bias=false` for a plain transformer; same event cache) | | |
| | `configs/gamgam_deep_sets.yaml` | the same task with Deep Sets, the baseline without object interactions (same event cache) | | |
| | `configs/tth_cp_even_odd.yaml` | CP-even versus CP-odd ttH (Delphes, binary; local files) | | |
| | `configs/delphes_pretraining.yaml` | 12-class pretraining on the Delphes samples | | |
| | `configs/delphes_jepa.yaml` | JEPA pretraining of the transformer encoder on the same samples, without labels (same event cache) | | |
| | `configs/delphes_jepa_edge.yaml` | the same for the edge network (fine-tune with `configs/gamgam_finetune.yaml model.pretrained=outputs/delphes_jepa_edge/last.pt`) | | |
| | `configs/delphes_contrastive.yaml` | contrastive (SimCLR) pretraining of the transformer encoder on the same samples (`model.backbone=edge_network` for the edge network) | | |
| | `configs/delphes_masked.yaml` | masked object modelling: JEPA predicting hidden objects' kinematics | | |
| | `configs/gamgam_jepa_finetune.yaml` | GamGam with the JEPA-pretrained encoder (`model.pretrained=null`: from scratch); also takes contrastive and masked-object encoders, and `probe` | | |
| | `configs/gamgam_tabpfn.yaml` | GamGam classified in context by TabPFN-3.5, no training (`evaluate` only) | | |
| | `configs/gamgam_tabicl.yaml` | the same by TabICLv2 (BSD-3 weights, downloaded from Hugging Face on first use) | | |
| | `configs/gamgam_tabdpt.yaml` | the same by TabDPT (Apache-2.0 weights; `model.tabdpt_context_size` retrieves each event's context) | | |
| | `configs/delphes_pfn.yaml` | meta-training of our own PFN on tasks drawn from the Delphes samples (same event cache) | | |
| | `configs/gamgam_pfn.yaml` | GamGam classified in context by that PFN, no training (`evaluate` only) | | |
| | `configs/delphes_tabpfn.yaml` | meta-training of TabPFN-3.5 (LoRA by default) on the same tasks; evaluate with `configs/gamgam_tabpfn.yaml checkpoint=outputs/delphes_tabpfn/best.pt` | | |
| | `configs/delphes_tabicl.yaml` | the same for TabICL; evaluate with `configs/gamgam_tabicl.yaml checkpoint=outputs/delphes_tabicl/best.pt` | | |
| Each file documents its keys; `src/gnn4colliders/config.py` lists every key | |
| with its default. Common overrides: | |
| ```bash | |
| train.fast=true # torch.compile + CUDA graphs (edge network) + bfloat16 + TF32 | |
| train.seed=3 data.seed=3 # initialization and shuffling seeds | |
| train.max_epochs=100 | |
| model.pretrained=Pretrained_GNN/multiclass_pretrained_model_12/model_epoch_71.pt | |
| model.freeze_backbone=true # train only the new classifier | |
| data.files=[{path: my.root, label: 0}] data.hf_repo=null # local inputs | |
| ``` | |
| ### Data | |
| Large inputs belong on scratch storage, not in the repository: set | |
| `HF_HOME` (where Hugging Face downloads go) and pass `data.cache=...`, e.g. | |
| on Perlmutter `HF_HOME=$SCRATCH/.cache/huggingface` and | |
| `data.cache=$SCRATCH/gnn4colliders/delphes/events.pt` (the 105M Delphes | |
| events take 52 GB of ROOT files and a 21 GB cache). | |
| `data.files` lists the ROOT files and their class labels (or a label branch | |
| name). With `data.hf_repo` the paths are inside that Hugging Face dataset at | |
| `data.hf_revision` and are downloaded once and checked against `sha256` and | |
| `size_bytes`. | |
| Each object becomes a row of `pt, eta, phi, energy, btag, charge, node_type` | |
| built from `feature_branches` (one branch or constant per object type, | |
| `CALC_E` = pt cosh(eta), `NODE_TYPE` = object-type index) and multiplied by | |
| `scales`. `derived_branches` select objects or threshold tagger scores before | |
| that (the GamGam file splits `lep_*` into electrons and muons). The fold of an | |
| event is `fold_var`, modulo `fold_modulus` when set, and | |
| `train_folds`/`validation_folds`/`test_folds` select the splits. Events are | |
| unweighted unless `weight_var` names a weight branch. | |
| ### In-context models and label efficiency | |
| `data.train_events=N` keeps N training events, an equal number per class | |
| (smaller N give subsets of larger ones). Supervised models train on them; | |
| the in-context models (`model.name: tabpfn` or `pfn`) use them as the | |
| labelled context and are only evaluated: | |
| ```bash | |
| uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation data.train_events=1000 | |
| uv run gnn4colliders train configs/delphes_pfn.yaml # meta-training | |
| uv run gnn4colliders evaluate configs/gamgam_pfn.yaml split=validation data.train_events=1000 | |
| uv run gnn4colliders train configs/gamgam.yaml data.train_events=1000 # supervised reference | |
| ``` | |
| `scripts/label_efficiency.py` runs such a sweep over several `N` and | |
| configs (restartable) and prints validation AUC and seconds per run; the | |
| metrics JSON records `seconds` (setup, inference) and, on CUDA, | |
| `peak_memory_gib`: | |
| ```bash | |
| uv run python scripts/label_efficiency.py configs/gamgam_tabpfn.yaml configs/gamgam.yaml \ | |
| --events 100 1000 10000 50000 | |
| ``` | |
| TabPFN-2.5 and later weights are non-commercial (research and internal | |
| benchmarking are allowed). tabpfn downloads them after a one-time login at | |
| https://ux.priorlabs.ai (token in `~/.cache/tabpfn/auth_token` or | |
| `TABPFN_TOKEN`), but a checkpoint already in its cache directory loads | |
| without the login, so on compute nodes download it from Hugging Face: | |
| ```bash | |
| export TABPFN_MODEL_CACHE_DIR=$SCRATCH/.cache/tabpfn # default ~/Library/Caches/tabpfn on macOS | |
| uvx --from huggingface_hub hf download Prior-Labs/tabpfn_3_5 config.json \ | |
| tabpfn-v3.5-20260909.safetensors tabpfn-v3.5-20260909_multiclass.safetensors \ | |
| --local-dir $TABPFN_MODEL_CACHE_DIR | |
| ``` | |
| `model.tabpfn_checkpoint=tabpfn-v3.5-20260909_multiclass.safetensors` | |
| selects the multiclass checkpoint. `model.tabpfn_version=v2` needs no login. | |
| TabDPT retrieval (`model.tabdpt_context_size`) on macOS needs | |
| `OMP_NUM_THREADS=1`: faiss and torch load separate OpenMP runtimes. | |
| Tabular models can use a pretrained encoder's event embeddings instead of | |
| or beside the rows: `model.tabular_inputs=embeddings` (or `both`) with | |
| `model.pretrained=outputs/delphes_jepa/last.pt` and the encoder's sizes | |
| (those of `configs/delphes_jepa.yaml`: `model.hid_size=128 model.n_layers=4 | |
| model.n_heads=8 model.ffn_size=512 model.pair_bias=true`). | |
| TabPFN and TabICL classify the same table rows (`model.slots` objects of | |
| each type). Two options add physics they cannot learn from a few events: | |
| `model.relative_phi=true` measures every azimuth from the leading object's, | |
| and `model.pair_objects=4` appends the invariant mass and Delta R of every | |
| pair of the four leading objects (12 columns). Compare with and without, | |
| one run name per variant: | |
| ```bash | |
| uv run python scripts/label_efficiency.py configs/gamgam_tabicl.yaml --events 100 1000 \ | |
| --override name=gamgam_tabicl_physics model.relative_phi=true model.pair_objects=4 | |
| ``` | |
| TabPFN-3.5 can also be meta-trained on the Delphes tasks, like our PFN. | |
| By default only LoRA updates of its in-context blocks train | |
| (`model.lora_rank`, 8; `model.lora_rank=0` trains all 219M weights), and | |
| checkpoints hold only those (33 MB with the optimizer state, against | |
| 2.6 GB for all weights). With a `checkpoint`, TabPFN is | |
| evaluated with its own forward pass, not the classifier: | |
| ```bash | |
| uv run gnn4colliders train configs/delphes_tabpfn.yaml | |
| uv run gnn4colliders evaluate configs/gamgam_tabpfn.yaml split=validation \ | |
| data.train_events=1000 checkpoint=outputs/delphes_tabpfn/best.pt | |
| ``` | |
| Activations are recomputed in the backward pass. On a 32 GB Apple M2 Max | |
| one task of 2048 context events and 256 queries takes about 4.5 s and | |
| 16 GiB per step with all weights trained, so the Mac is for development; | |
| meta-train on a GPU node. | |
| ### Fine-tuning | |
| `Pretrained_GNN/` holds the pretrained models of the paper, the final | |
| epoch of each: `multiclass_pretrained_model_12/model_epoch_71.pt` and | |
| `multilabel_pretrained_model_41/model_epoch_44.pt` (with their training | |
| configuration and log). With `model.pretrained` the edge network is loaded from | |
| that checkpoint and gets a new `model.out_size` classifier; `model.hid_size`, | |
| `n_layers` and `n_proc_steps` must match it (64, 4, 4). A JEPA checkpoint | |
| (`configs/delphes_jepa*.yaml`) loads the same way: its context encoder | |
| initializes the edge network or the transformer encoder. | |
| ## GPUs, Perlmutter and DDP | |
| `train.device: auto` uses CUDA, else an Apple GPU (MPS), else the CPU. MPS | |
| is for development on fixtures and small runs (no float64: metrics are | |
| computed on the CPU; no DDP, CUDA graphs or TF32). | |
| On a single GPU, `train.fast=true` is about 3x faster than float32 for the | |
| GamGam model (see [design.md](design.md)). Set `OMP_NUM_THREADS` to the CPU | |
| allocation and, on Perlmutter compute nodes, `UV_CACHE_DIR=$SCRATCH/.cache/uv` | |
| (uv cannot lock its home-directory cache there). | |
| ```bash | |
| # One training per seed, each followed by test-split evaluation: | |
| for seed in 1 2 3 4 5; do | |
| sbatch -A <account> --export=ALL,SEED=$seed scripts/train.sh train.fast=true | |
| done | |
| uv run python scripts/summarize_ensemble.py outputs/gamgam/seed* # mean +- std | |
| ``` | |
| The study of in-context models and pretraining has one wrapper per kind of | |
| job, each on one GPU (the header of each script lists its variables): | |
| ```bash | |
| # Label-efficiency sweep of released in-context models and supervised references: | |
| sbatch -A <account> --export=ALL,CONFIGS="configs/gamgam_tabpfn.yaml configs/gamgam_tabicl.yaml configs/gamgam.yaml" \ | |
| scripts/label_efficiency.sh | |
| # Meta-training, then GamGam in context with the meta-trained model: | |
| sbatch -A <account> --export=ALL,CONFIG=configs/delphes_tabicl.yaml,EVAL_CONFIG=configs/gamgam_tabicl.yaml \ | |
| scripts/meta_train.sh task.prior_classes=[1,6,7,8,9,10,11] | |
| # Label-free pretraining, then k-NN and linear probes on GamGam: | |
| sbatch -A <account> --export=ALL,CONFIG=configs/delphes_contrastive.yaml scripts/pretrain.sh | |
| # Linear CKA between the pretrained encoders, layer by layer (a login node is enough): | |
| uv run python scripts/compare_encoders.py configs/gamgam_jepa_finetune.yaml --events 5000 \ | |
| --encoders random=random jepa=outputs/delphes_jepa/last.pt \ | |
| contrastive=outputs/delphes_contrastive/last.pt masked=outputs/delphes_masked/last.pt | |
| # Curves and tables of every finished run: | |
| uv run --extra plots python scripts/plot_label_efficiency.py --sweeps outputs/label_efficiency \ | |
| --probes outputs/delphes_jepa outputs/delphes_contrastive outputs/delphes_masked | |
| ``` | |
| Under `torchrun` with more than one process, training is data parallel: | |
| `data.batch_size` is per process, training events are sharded over ranks | |
| (padded to equal length) and metrics are computed over all ranks' events. | |
| CUDA graphs need a single process (so `train.fast=true` does too; DDP is | |
| tested in float32 eager). `scripts/train_ddp.sh` launches all GPUs of | |
| one or more nodes. | |