- data.eval_events: evaluate (validation in training, evaluate, predict,
probe) only that many seeded random validation or test events.
- scripts/label_efficiency.sh, meta_train.sh, pretrain.sh: one-GPU Slurm
wrappers for the sweep, meta-training then in-context sweep, and
pretraining then probes.
- scripts/plot_label_efficiency.py: AUC against labelled events from sweeps
and probes, as PDF/PNG and Markdown/LaTeX tables; optional extra `plots`.
Wrappers smoke-tested outside Slurm on fixtures; tests pass on CPU and MPS.
No change to existing models' results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: tabular in-context models, LoRA meta-training, pretraining objectives and probes
In-context models (tabular.py):
- TabPFN-3.5 meta-trained on the PFN's Delphes tasks (model.name: tabpfn,
task.name: pfn), LoRA on its 24 in-context blocks by default
(model.lora_rank; 0 trains all weights); checkpoints hold only trained
weights. configs/delphes_tabpfn.yaml.
- TabICLv2 (BSD-3) as released and meta-trained the same way
(configs/gamgam_tabicl.yaml, configs/delphes_tabicl.yaml); LoRA is a
weight parametrization, as TabICL reads fused projections directly.
- TabDPT (Apache-2.0) as released, with optional per-event retrieval of the
context (configs/gamgam_tabdpt.yaml).
- Row options: azimuths from the leading object (model.relative_phi), pair
masses and Delta R (model.pair_objects); encoder embeddings instead of or
beside rows (model.tabular_inputs).
- model.tabpfn_checkpoint; evaluate records setup/inference seconds and
CUDA peak memory.
Pretraining (all label-free, encoders load as JEPA's):
- contrastive.py: SimCLR with physics augmentations
(configs/…
feat: TabPFN and a collider PFN as in-context models
- data.train_events: an equal number of training events per class (nested
subsets), for supervised training and as the in-context models' context.
- tabular.py: TabPFN (default 3.5) on fixed-width event rows; optional
extra `tabpfn`. configs/gamgam_tabpfn.yaml.
- pfn.py: a prior-fitted network over events (object-transformer
embeddings, label embedding, attention to context events only),
meta-trained on tasks drawn from the Delphes samples (task.name: pfn).
configs/delphes_pfn.yaml, configs/gamgam_pfn.yaml.
- evaluate/predict run in-context models without a training step.
- EncoderBlock split into project/attend so context keys are cached.
Tested on CPU fixtures only; GPU study pending (docs/design.md). Existing
models' features, splits, losses and metrics are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
model.backbone selects the JEPA encoder (transformer or edge_network).
The edge network's object encodings are its node states averaged over
the encoder and processing steps (its last states alone are nearly equal
within an event at initialization), its global state is a predictor
token, and model.variance_weight adds a VICReg variance term against
collapse. A JEPA checkpoint fine-tunes through gamgam_finetune.yaml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
model.name: jepa with task.name: jepa pretrains the transformer encoder
without labels (I-JEPA): a predictor told only the type and type index of
hidden objects predicts their encodings by an EMA target encoder.
configs/delphes_jepa.yaml uses the 12 Delphes samples and cache of the
supervised pretraining; configs/gamgam_jepa_finetune.yaml fine-tunes the
context encoder on GamGam (model.pretrained=null: from scratch).
TransformerClassifier now wraps an ObjectTransformer encoder.
The Slurm scripts put Hugging Face downloads on scratch (HF_HOME).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: transformer pair attention bias and a Deep Sets model
model.pair_bias adds a per-head attention bias from an MLP of the pair
features [deta, dphi, dR, log(1 + m^2)] (Particle Transformer style); it is
on in configs/gamgam_transformer.yaml. configs/gamgam_deep_sets.yaml adds
Deep Sets, the order-independent baseline without object interactions. Both
share an ObjectEmbedding and export to ONNX.
The transformer now uses its own pre-norm blocks on scaled_dot_product_attention:
nn.TransformerEncoder's inference fast path returns NaN with a per-head mask.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(transformer): type embedding instead of positions, pre-norm, ONNX export
Objects are stored by type, so the position embedding mostly relearned the
object type and made the output depend on object order. Tokens are now a
linear map of the kinematics (phi as cos/sin) plus a node-type embedding;
model.max_seq_len is gone. The transformer exports to ONNX with tokens and
token_mask inputs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chore: move test fixtures into tests/, flatten scripts/, drop paper source
Fine-tuning loads a single checkpoint, so drop the 114 intermediate
model_epoch_N.pt files and keep multiclass epoch 71 and multilabel
epoch 44.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
refactor: flat modules, one YAML per experiment, no Hydra
Replace the 14 subpackages with 13 flat modules (6.7k -> 2.7k lines),
the Hydra config groups with typed dataclasses and one self-contained
YAML file per experiment (configs/), and the docs with docs/usage.md and
docs/design.md. All capabilities stay: GamGam, ttH CP, Delphes, binary
and multiclass tasks, transformer, fine-tuning from Pretrained_GNN, DDP,
torch.compile/CUDA graphs/bfloat16, predict and ONNX export.
Behavior: prepared caches, pretrained-model logits and two CPU training
epochs are bit-identical to the previous code on the fixtures and a 60k
event GamGam subset. Energies are now computed with one vectorized cosh
(<= 2.4e-7 relative difference on 969 of the full GamGam objects). The
ttH and Delphes configs now use Number % 10 as the fold (the raw event
index selected only the first 10 events), and use the paper recipe
(64/4/4, Adam 1e-4, decay 0.99). The ROOT score writer handles jagged
trees. Prediction files store source file and entry instead of sample
ids, and checkpo…
chore: remove legacy parity, validation harness, benchmarks and release tooling
The rewrite is complete; the frozen DGL/legacy references, the validation
harness, benchmarks, CI, agent skills, release tooling, migration history
and changelog are no longer needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: upgrade to Python 3.14 and PyTorch 2.14; add torch.compile and bfloat16
- Python 3.14, torch 2.14.1 (cu130 wheels), NumPy 2; frozen parity
references pass unchanged
- trainer.compile (dynamic-shape torch.compile) and trainer.precision
(bfloat16 autocast; graph sums, loss, optimizer in float32)
- CUDA graph steps use device-tensor learning rates, so one capture per
bucket follows the schedule; checkpoints store portable optimizer state
- trainer=paper_fast: compile + CUDA graphs + bfloat16, 5.9 vs 19.2 ms per
GamGam batch for float32 eager on an A100
- loader workers fork explicitly (Python 3.14 defaults to forkserver);
ONNX export keeps the TorchScript exporter
Correction to 5b73bcb: the pre-change float32 eager step was 21.3 ms per
batch (not ~29 ms; that profile used unshuffled ttH-only batches), so the
default path improved by about 1.2x and the large gains are opt-in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs: make the ATLAS Open Data GamGam task the main benchmark
Document the GamGam workflow, the new data, checkpoint, loader, and speed
options, and the intentional differences in docs/migration.md. Slurm
wrappers now use a scratch uv cache, which compute nodes can lock. Add an
ensemble summary script.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: replace DGL with shared packed events and native PyTorch graphs
Track the documented Slurm wrappers that an unanchored slurm/ ignore rule
excluded, move graph-cache preparation from config.application into
graphs.preparation (outputs verified identical to main), archive rewrite
task specs under docs/history, move the paper source under docs/paper,
remove empty scaffolding, and refresh stale transformer documentation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- validation: use pinned public Hugging Face fixture (528346c0a2dcf57bc6f47e0e929113ae0f5ec47a)
- data: resolve training sources from Hugging Face (02920c2576f35911f7068c0379c06585ae9c8b57)