metadata
license: apache-2.0
tags:
- bilinear
- efficientformer
- gelu-tanh
- kaiming-normal
- linear-warmup
- matching
- nano
- rmsnorm
- rmsprop
- sparse
eval.py
Model Overview
A nano-scale implementation of the efficientformer architecture, built for matching tasks.
Architecture
- Architecture: efficientformer
- Scale: nano
- Attention: sparse
- Fusion strategy: bilinear
- Task head: matching
- Activation: gelu tanh
- Normalization: rmsnorm
- Initialization: kaiming normal
Training
- Optimizer: rmsprop
- LR scheduler: linear warmup
Files
eval.py— main artifact of this repository
License
See the license field above.