kurquia commited on
Commit
c01ae27
·
verified ·
1 Parent(s): d1b047a

Add ShellFlow-B and ShellGATr checkpoints; model card table

Browse files

This PR adds the two networks from App. C of *Discovering Z Bosons Inside of a Neural Network*:

- `shellflow_b_2to4lep/model.ckpt`: ShellFlow-B, 3,016,837 parameters, trained 100 epochs on 2to4lep. It loads with the current ShellFlow `main`.
- `shellgatr_2to4lep/model.ckpt`: ShellGATr, 3,087,498 parameters, L-GATr-slim blocks. It needs https://github.com/pomidori/ShellFlow/pull/1 and `uv sync --extra gatr`.

The optimizer state is stripped, and distribution matching is off in the stored hyperparameters, as in the existing files.

The model card gains a table for the two networks and corrects the parameter count of the first five models: 3.0M, not ~6.0M, which counted the EMA copy as well.

README.md CHANGED
@@ -18,7 +18,7 @@ ShellFlow is a transformer generative model of LHC collision events, trained wit
18
 
19
  ## Models
20
 
21
- All models share the same backbone (d_model 128, 6 layers, 4 heads, ~6.0M parameters) and were trained on the union of the ATLAS Open Data `1LMET30` and `2to4lep` skims (events with ≤ 8 reconstructed particles, de-duplicated by run/event number).
22
 
23
  | File | Model | Chart | Aux head | Training |
24
  |---|---|---|---|---|
@@ -28,6 +28,17 @@ All models share the same backbone (d_model 128, 6 layers, 4 heads, ~6.0M parame
28
  | `e1_v2_manifold_noaux/model.ckpt` | Ablation E1-V2 | on-shell (manifold) | no | 2 epochs, 48k steps |
29
  | `e1_v3_euclidean_noaux/model.ckpt` | Ablation E1-V3 | free (Euclidean) | no | 2 epochs, 48k steps |
30
 
 
 
 
 
 
 
 
 
 
 
 
31
  Checkpoint selection: `union_1LMET30_2to4lep/model.ckpt` holds the weights at the end of the last training epoch (stored `epoch` = 29, `global_step` = 717210). It is not the checkpoint with the lowest validation loss, which occurred at epoch 26 (`val/loss/total` = 1.1685). Epochs are counted from 0, as in PyTorch Lightning.
32
 
33
  Checkpoint format: PyTorch Lightning (weights + EMA weights + hyperparameters; optimizer state stripped). The hyperparameters carry `architecture: legacy_type_film`, which selects the paper architecture in the ShellFlow code. Distribution matching (a training-time reweighting) is disabled in the stored hyperparameters so the checkpoints load without any dataset present; it has no effect on generation.
 
18
 
19
  ## Models
20
 
21
+ The five models of arXiv:2607.16144 share the same backbone (d_model 128, 6 layers, 4 heads, 3.0M parameters; each file also holds an EMA copy of the weights) and were trained on the union of the ATLAS Open Data `1LMET30` and `2to4lep` skims (events with ≤ 8 reconstructed particles, de-duplicated by run/event number).
22
 
23
  | File | Model | Chart | Aux head | Training |
24
  |---|---|---|---|---|
 
28
  | `e1_v2_manifold_noaux/model.ckpt` | Ablation E1-V2 | on-shell (manifold) | no | 2 epochs, 48k steps |
29
  | `e1_v3_euclidean_noaux/model.ckpt` | Ablation E1-V3 | free (Euclidean) | no | 2 epochs, 48k steps |
30
 
31
+ ### Models of the steering paper
32
+
33
+ Two more networks, trained on the `2to4lep` skim alone (268M events with at most 8 reconstructed particles), appear in Appendix C of *Discovering Z Bosons Inside of a Neural Network* (arXiv link to follow).
34
+
35
+ | File | Model | Backbone | Parameters | Training |
36
+ |---|---|---|---|---|
37
+ | `shellflow_b_2to4lep/model.ckpt` | ShellFlow-B | transformer with per-type input and output layers | 3,016,837 | 100 epochs, 1,636,500 steps |
38
+ | `shellgatr_2to4lep/model.ckpt` | ShellGATr | Lorentz-equivariant L-GATr-slim blocks, 32 vectors and 96 scalars per token | 3,087,498 | 100 epochs, 1,636,500 steps |
39
+
40
+ Both files hold the weights at the end of the last training epoch (stored `epoch` = 99). ShellFlow-B loads with the current architecture in `riemannian_DiTs_model.py` and was trained with `configs/train_riemannian_DiTs_2to4lep.yaml`. ShellGATr needs the optional `gatr` extra (`uv sync --extra gatr`) and was trained with `configs/train_riemannian_DiTs_2to4lep_lgatr_slim.yaml`. As for the other files, the optimizer state is stripped and distribution matching is disabled in the stored hyperparameters.
41
+
42
  Checkpoint selection: `union_1LMET30_2to4lep/model.ckpt` holds the weights at the end of the last training epoch (stored `epoch` = 29, `global_step` = 717210). It is not the checkpoint with the lowest validation loss, which occurred at epoch 26 (`val/loss/total` = 1.1685). Epochs are counted from 0, as in PyTorch Lightning.
43
 
44
  Checkpoint format: PyTorch Lightning (weights + EMA weights + hyperparameters; optimizer state stripped). The hyperparameters carry `architecture: legacy_type_film`, which selects the paper architecture in the ShellFlow code. Distribution matching (a training-time reweighting) is disabled in the stored hyperparameters so the checkpoints load without any dataset present; it has no effect on generation.
shellflow_b_2to4lep/model.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9552c9a453c2137e6f89612c853b0854c57c0af8411265c345a151185ef441c
3
+ size 24239627
shellgatr_2to4lep/model.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:34be8d5601b6577e8a231b62f63e95fde4068fc3607ffaaf090cb5ff17ef91c1
3
+ size 24805931