| In phase 1, the model was first trained to completion while keeping the bigbird frozen. Later in phase 2 the bigbird weights were set as trainable and trained using warmup. |
| In phase 1, the model was first trained to completion while keeping the bigbird frozen. Later in phase 2 the bigbird weights were set as trainable and trained using warmup. |