Commit ·
0cbc1bb
1
Parent(s): d1b047a
Add ShellFlow-B and ShellGATr checkpoints; model card table (#1)
Browse files- Add ShellFlow-B and ShellGATr checkpoints; model card table (c01ae277c4d37e41def5ec0f1487806632077533)
Co-authored-by: Kevin Urquía <kurquia@users.noreply.huggingface.co>
- README.md +12 -1
- shellflow_b_2to4lep/model.ckpt +3 -0
- shellgatr_2to4lep/model.ckpt +3 -0
README.md
CHANGED
|
@@ -18,7 +18,7 @@ ShellFlow is a transformer generative model of LHC collision events, trained wit
|
|
| 18 |
|
| 19 |
## Models
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
| File | Model | Chart | Aux head | Training |
|
| 24 |
|---|---|---|---|---|
|
|
@@ -28,6 +28,17 @@ All models share the same backbone (d_model 128, 6 layers, 4 heads, ~6.0M parame
|
|
| 28 |
| `e1_v2_manifold_noaux/model.ckpt` | Ablation E1-V2 | on-shell (manifold) | no | 2 epochs, 48k steps |
|
| 29 |
| `e1_v3_euclidean_noaux/model.ckpt` | Ablation E1-V3 | free (Euclidean) | no | 2 epochs, 48k steps |
|
| 30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
Checkpoint selection: `union_1LMET30_2to4lep/model.ckpt` holds the weights at the end of the last training epoch (stored `epoch` = 29, `global_step` = 717210). It is not the checkpoint with the lowest validation loss, which occurred at epoch 26 (`val/loss/total` = 1.1685). Epochs are counted from 0, as in PyTorch Lightning.
|
| 32 |
|
| 33 |
Checkpoint format: PyTorch Lightning (weights + EMA weights + hyperparameters; optimizer state stripped). The hyperparameters carry `architecture: legacy_type_film`, which selects the paper architecture in the ShellFlow code. Distribution matching (a training-time reweighting) is disabled in the stored hyperparameters so the checkpoints load without any dataset present; it has no effect on generation.
|
|
|
|
| 18 |
|
| 19 |
## Models
|
| 20 |
|
| 21 |
+
The five models of arXiv:2607.16144 share the same backbone (d_model 128, 6 layers, 4 heads, 3.0M parameters; each file also holds an EMA copy of the weights) and were trained on the union of the ATLAS Open Data `1LMET30` and `2to4lep` skims (events with ≤ 8 reconstructed particles, de-duplicated by run/event number).
|
| 22 |
|
| 23 |
| File | Model | Chart | Aux head | Training |
|
| 24 |
|---|---|---|---|---|
|
|
|
|
| 28 |
| `e1_v2_manifold_noaux/model.ckpt` | Ablation E1-V2 | on-shell (manifold) | no | 2 epochs, 48k steps |
|
| 29 |
| `e1_v3_euclidean_noaux/model.ckpt` | Ablation E1-V3 | free (Euclidean) | no | 2 epochs, 48k steps |
|
| 30 |
|
| 31 |
+
### Models of the steering paper
|
| 32 |
+
|
| 33 |
+
Two more networks, trained on the `2to4lep` skim alone (268M events with at most 8 reconstructed particles), appear in Appendix C of *Discovering Z Bosons Inside of a Neural Network* (arXiv link to follow).
|
| 34 |
+
|
| 35 |
+
| File | Model | Backbone | Parameters | Training |
|
| 36 |
+
|---|---|---|---|---|
|
| 37 |
+
| `shellflow_b_2to4lep/model.ckpt` | ShellFlow-B | transformer with per-type input and output layers | 3,016,837 | 100 epochs, 1,636,500 steps |
|
| 38 |
+
| `shellgatr_2to4lep/model.ckpt` | ShellGATr | Lorentz-equivariant L-GATr-slim blocks, 32 vectors and 96 scalars per token | 3,087,498 | 100 epochs, 1,636,500 steps |
|
| 39 |
+
|
| 40 |
+
Both files hold the weights at the end of the last training epoch (stored `epoch` = 99). ShellFlow-B loads with the current architecture in `riemannian_DiTs_model.py` and was trained with `configs/train_riemannian_DiTs_2to4lep.yaml`. ShellGATr needs the optional `gatr` extra (`uv sync --extra gatr`) and was trained with `configs/train_riemannian_DiTs_2to4lep_lgatr_slim.yaml`. As for the other files, the optimizer state is stripped and distribution matching is disabled in the stored hyperparameters.
|
| 41 |
+
|
| 42 |
Checkpoint selection: `union_1LMET30_2to4lep/model.ckpt` holds the weights at the end of the last training epoch (stored `epoch` = 29, `global_step` = 717210). It is not the checkpoint with the lowest validation loss, which occurred at epoch 26 (`val/loss/total` = 1.1685). Epochs are counted from 0, as in PyTorch Lightning.
|
| 43 |
|
| 44 |
Checkpoint format: PyTorch Lightning (weights + EMA weights + hyperparameters; optimizer state stripped). The hyperparameters carry `architecture: legacy_type_film`, which selects the paper architecture in the ShellFlow code. Distribution matching (a training-time reweighting) is disabled in the stored hyperparameters so the checkpoints load without any dataset present; it has no effect on generation.
|
shellflow_b_2to4lep/model.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e9552c9a453c2137e6f89612c853b0854c57c0af8411265c345a151185ef441c
|
| 3 |
+
size 24239627
|
shellgatr_2to4lep/model.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:34be8d5601b6577e8a231b62f63e95fde4068fc3607ffaaf090cb5ff17ef91c1
|
| 3 |
+
size 24805931
|