| --- |
| license: apache-2.0 |
| language: |
| - en |
| library_name: transformers |
| --- |
| # AnalysisObjectTransformer Model |
|
|
| This repository contains the implementation of the AnalysisObjectTransformer model, a deep learning architecture designed for event classification with data from the CERN LHC. |
| The model operates reconstructed-object and event-level features. MultiHeadAttention is used to extract the correlation between reconstructed objects such as jets (hadrons) or leptons in the final state, while event-level features capture the event summary, such as total hadronic energy or missing transverse energy. |
| Achieves state-of-the-art performance on final states which can be summarized as jets accompanied by missing transverse energy. |
|
|
| ## Model Overview |
|
|
| The AnalysisObjectTransformer model is structured to process object-level features, in the case of jets: energy, mass, area, btag score, in any order (permutation invariance) and event-level features (HT, MET) to classify signal from background processes to enhance the sensitivity to rare BSM signatures. |
|
|
| ### Components |
|
|
|
|
| See [**here**](https://excalidraw.com/#json=tCXGu1s6Az9wh4md45JU6,A3ezTIoqB10HVxOt4hhRSA) for complete architecure. |
|
|
| - **Embedding Layers**: Transform input data into a higher-dimensional space for subsequent processing. |
| - **Attention Blocks (AttBlock)**: Utilize multi-head attention to capture dependencies between different elements of the input data. |
| - **Class Blocks (ClassBlock)**: Extend attention mechanisms to incorporate class tokens, enabling the model to focus on class-relevant features. Implementation based on "Going deeper with transformers": https://arxiv.org/abs/2103.17239 |
| - **MLP Head**: A sequence of fully connected layers that maps the output of the transformer blocks to the final prediction targets. |
|
|
|
|
| ## Usage |
|
|
| Firstly, clone the repository: |
| ```bash |
| git clone https://huggingface.co/maciek-g/AnalysisObjectTransformer |
| ``` |
| You can then import the model object and use within standard PyTorch and PyTorch lightning training workflows. |
|
|
| ```python |
| from particle_transformer import AnalysisObjectTransformer |
| |
| model = AnalysisObjectTransformer(input_dim_obj=..., input_dim_event=..., embed_dims=..., linear_dims1=..., linear_dims2=..., mlp_hidden_1=..., mlp_hidden_2=..., num_heads=...) |
| ``` |
|
|
| ## Parameter definitions |
|
|
| `input_dim_obj`: The number of features associated with each event object (features per jet, lepton etc..) |
|
|
|
|
| `input_dim_events`: The number of features associated with the event (Number of jets, total hadronic energy, total missing transverse energy etc..) |
|
|
| `embed_dims`: Sequence embedding dims |
|
|
|
|
