GEM_Banking77 Model Card

This model card provides an overview of the GEM_Banking77 model, a fine-tuned implementation of the GEM architecture designed for the Banking77 dataset.

Purpose

The GEM_Banking77 model was developed to evaluate the performance of the GEM architecture on domain-specific datasets, particularly in the banking and financial sector. The Banking77 dataset, a benchmark for intent classification, was chosen to assess the model’s effectiveness.

Key Details

• License: Apache-2.0
• Dataset: legacy-datasets/banking77
• Language: English
• Metrics: Accuracy: 92.56%
• Base Model: bert-base-uncased
• Pipeline: GEM_pipeline

Model Details

The GEM_Banking77 model is built on the GEM architecture and fine-tuned from bert-base-uncased using the Banking77 dataset. The model configuration is as follows:

• Number of epochs: 10
• Batch size: Dynamic scaling: 32 * number of GPUs
• Learning rate: 2e-5
• Maximum sequence length: 128
• Gradient accumulation steps: 2
• Cluster size: 256
• Number of domains: 8
• Number of classes: 77
• Number of attention heads: 12

Training & Evaluation

The model was trained using the GEM_pipeline and evaluated using accuracy, achieving a score of 92.56%.