| **Fine-Tuning GPT-2 and Using Transformer-Explainer for Analysis** |
|
|
| **Project Overview** |
| This project involves fine-tuning the GPT-2 model with the assigned language and analyzing its behavior using the Transformer-Explainer package. The objective is to observe how the fine-tuned model processes language inputs and generates outputs, gaining insights into its inner workings. |
|
|
| **Objectives** |
| 1. Fine-tune GPT-2 with a specific language dataset. |
| 2. Use the Transformer-Explainer package to analyze the behavior of the custom-trained GPT-2 model. |
| 3. Generate 10 examples and observe the model’s performance using the explainer. |
|
|
| **Steps to Complete the Project** |
| 1. Fine-Tuning GPT-2 |
| Used HuggingFace Transformers to fine-tune the GPT-2 model on finnish language. |
| Prepared the dataset ensuring it represents the linguistic characteristics and diversity of the assigned language. |
| 2. Running Transformer-Explainer |
| Clone the Transformer-Explainer GitHub repository. |
| Integrated the custom GPT-2 model with the explainer package. |
| 3. Observed and analyzed the output on: |
| Attention patterns. |
| Token relationships. |
|
|
| **Resources** |
| 1. Transformer-Explainer GitHub Repository |
| 2. HuggingFace Transformers Documentation |
| 3. Dataset in the assigned language |
