probe.linear_cka and probe.layer_embeddings, and
scripts/compare_encoders.py: CKA of the event embeddings of several
encoders of one architecture (pretrained with JEPA, contrastive, masked
objects or supervision, or random), and layer by layer against the first,
as in Ho et al. (2026). Tested on fixtures for both backbones.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: evaluation subsample, Slurm wrappers for the study and label-efficiency plots
- data.eval_events: evaluate (validation in training, evaluate, predict,
probe) only that many seeded random validation or test events.
- scripts/label_efficiency.sh, meta_train.sh, pretrain.sh: one-GPU Slurm
wrappers for the sweep, meta-training then in-context sweep, and
pretraining then probes.
- scripts/plot_label_efficiency.py: AUC against labelled events from sweeps
and probes, as PDF/PNG and Markdown/LaTeX tables; optional extra `plots`.
Wrappers smoke-tested outside Slurm on fixtures; tests pass on CPU and MPS.
No change to existing models' results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: tabular in-context models, LoRA meta-training, pretraining objectives and probes
In-context models (tabular.py):
- TabPFN-3.5 meta-trained on the PFN's Delphes tasks (model.name: tabpfn,
task.name: pfn), LoRA on its 24 in-context blocks by default
(model.lora_rank; 0 trains all weights); checkpoints hold only trained
weights. configs/delphes_tabpfn.yaml.
- TabICLv2 (BSD-3) as released and meta-trained the same way
(configs/gamgam_tabicl.yaml, configs/delphes_tabicl.yaml); LoRA is a
weight parametrization, as TabICL reads fused projections directly.
- TabDPT (Apache-2.0) as released, with optional per-event retrieval of the
context (configs/gamgam_tabdpt.yaml).
- Row options: azimuths from the leading object (model.relative_phi), pair
masses and Delta R (model.pair_objects); encoder embeddings instead of or
beside rows (model.tabular_inputs).
- model.tabpfn_checkpoint; evaluate records setup/inference seconds and
CUDA peak memory.
Pretraining (all label-free, encoders load as JEPA's):
- contrastive.py: SimCLR with physics augmentations
(configs/…
feat: TabPFN and a collider PFN as in-context models
- data.train_events: an equal number of training events per class (nested
subsets), for supervised training and as the in-context models' context.
- tabular.py: TabPFN (default 3.5) on fixed-width event rows; optional
extra `tabpfn`. configs/gamgam_tabpfn.yaml.
- pfn.py: a prior-fitted network over events (object-transformer
embeddings, label embedding, attention to context events only),
meta-trained on tasks drawn from the Delphes samples (task.name: pfn).
configs/delphes_pfn.yaml, configs/gamgam_pfn.yaml.
- evaluate/predict run in-context models without a training step.
- EncoderBlock split into project/attend so context keys are cached.
Tested on CPU fixtures only; GPU study pending (docs/design.md). Existing
models' features, splits, losses and metrics are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
model.backbone selects the JEPA encoder (transformer or edge_network).
The edge network's object encodings are its node states averaged over
the encoder and processing steps (its last states alone are nearly equal
within an event at initialization), its global state is a predictor
token, and model.variance_weight adds a VICReg variance term against
collapse. A JEPA checkpoint fine-tunes through gamgam_finetune.yaml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
model.name: jepa with task.name: jepa pretrains the transformer encoder
without labels (I-JEPA): a predictor told only the type and type index of
hidden objects predicts their encodings by an EMA target encoder.
configs/delphes_jepa.yaml uses the 12 Delphes samples and cache of the
supervised pretraining; configs/gamgam_jepa_finetune.yaml fine-tunes the
context encoder on GamGam (model.pretrained=null: from scratch).
TransformerClassifier now wraps an ObjectTransformer encoder.
The Slurm scripts put Hugging Face downloads on scratch (HF_HOME).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: transformer pair attention bias and a Deep Sets model
model.pair_bias adds a per-head attention bias from an MLP of the pair
features [deta, dphi, dR, log(1 + m^2)] (Particle Transformer style); it is
on in configs/gamgam_transformer.yaml. configs/gamgam_deep_sets.yaml adds
Deep Sets, the order-independent baseline without object interactions. Both
share an ObjectEmbedding and export to ONNX.
The transformer now uses its own pre-norm blocks on scaled_dot_product_attention:
nn.TransformerEncoder's inference fast path returns NaN with a per-head mask.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(transformer): type embedding instead of positions, pre-norm, ONNX export
Objects are stored by type, so the position embedding mostly relearned the
object type and made the output depend on object order. Tokens are now a
linear map of the kinematics (phi as cos/sin) plus a node-type embedding;
model.max_seq_len is gone. The transformer exports to ONNX with tokens and
token_mask inputs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chore: move test fixtures into tests/, flatten scripts/, drop paper source
Fine-tuning loads a single checkpoint, so drop the 114 intermediate
model_epoch_N.pt files and keep multiclass epoch 71 and multilabel
epoch 44.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
refactor: flat modules, one YAML per experiment, no Hydra
Replace the 14 subpackages with 13 flat modules (6.7k -> 2.7k lines),
the Hydra config groups with typed dataclasses and one self-contained
YAML file per experiment (configs/), and the docs with docs/usage.md and
docs/design.md. All capabilities stay: GamGam, ttH CP, Delphes, binary
and multiclass tasks, transformer, fine-tuning from Pretrained_GNN, DDP,
torch.compile/CUDA graphs/bfloat16, predict and ONNX export.
Behavior: prepared caches, pretrained-model logits and two CPU training
epochs are bit-identical to the previous code on the fixtures and a 60k
event GamGam subset. Energies are now computed with one vectorized cosh
(<= 2.4e-7 relative difference on 969 of the full GamGam objects). The
ttH and Delphes configs now use Number % 10 as the fold (the raw event
index selected only the first 10 events), and use the paper recipe
(64/4/4, Adam 1e-4, decay 0.99). The ROOT score writer handles jagged
trees. Prediction files store source file and entry instead of sample
ids, and checkpo…