| --- |
| language: |
| - is |
| --- |
| README |
|
|
| Overview |
|
|
| This project implements a language translation model using GPT-2, capable of translating between Icelandic and English. The pipeline includes data preprocessing, model training, evaluation, and an interactive user interface for translations. |
|
|
| Features |
|
|
| Text Preprocessing: Tokenization and padding for uniform input size. |
|
|
| Model Training: Training a GPT-2 model on paired Icelandic-English sentences. |
|
|
| Evaluation: Perplexity-based validation of model performance. |
|
|
| Interactive Interface: An easy-to-use widget for real-time translations. |
|
|
| Installation |
|
|
| Prerequisites |
|
|
| Ensure you have the following installed: |
|
|
| Python (>= 3.8) |
|
|
| PyTorch |
|
|
| Transformers library by Hugging Face |
|
|
| ipywidgets (for the translation interface) |
|
|
| Steps |
|
|
| Clone the repository: |
|
|
| git clone <repository_url> |
| cd <repository_name> |
|
|
| Install the required libraries: |
|
|
| pip install -r requirements.txt |
|
|
| Ensure GPU availability for faster training (optional but recommended). |
|
|
| Usage |
|
|
| Training the Model |
|
|
| Prepare your dataset with English-Icelandic sentence pairs. |
|
|
| Run the script to preprocess the data and train the model: |
|
|
| python train_model.py |
| |
| The trained model and tokenizer will be saved in the ./trained_gpt2 directory. |
|
|
| Evaluating the Model |
|
|
| Evaluate the trained model using validation data: |
|
|
| python evaluate_model.py |
| |
| The script computes perplexity to measure model performance. |
| |
| Running the Interactive Interface |
| |
| Launch a Jupyter Notebook or Jupyter Lab. |
| |
| Open the file interactive_translation.ipynb. |
|
|
| Enter a sentence in English or Icelandic, and view the translation in real-time. |
|
|
| File Structure |
|
|
| train_model.py: Contains code for data preprocessing, model training, and saving. |
| |
| evaluate_model.py: Evaluates model performance using perplexity. |
|
|
| interactive_translation.ipynb: Interactive interface for testing translations. |
| |
| requirements.txt: List of required Python packages. |
| |
| trained_gpt2/: Directory to save trained model and tokenizer. |
|
|
| Key Parameters |
|
|
| Max Length: Maximum token length for inputs (default: 128). |
|
|
| Learning Rate: . |
|
|
| Batch Size: 4 (both training and validation). |
|
|
| Epochs: 10. |
|
|
| Beam Search: Used for generating translations, with a beam size of 5. |
|
|
| Future Improvements |
|
|
| Expand dataset to include additional language pairs. |
|
|
| Optimize the model for faster inference. |
|
|
| Integrate the application into a web-based interface. |
|
|
| Acknowledgements |
|
|
| Hugging Face for providing the GPT-2 model and libraries. |
|
|
| PyTorch for enabling seamless implementation and training. |
|
|
| License |
|
|
| This project is licensed under the MIT License. See the LICENSE file for details. |
