data/README.md

Data Folder

This data folder contains datasets and resources used for the Mini-Transformer project.

Structure

• corpus/: Text corpora for training and evaluation.
• raw/: Original, unprocessed datasets.
• processed/: Cleaned and preprocessed data ready for model training.
• README.md: Documentation for the data folder.

Usage

Place your datasets in the appropriate subfolders. Ensure data is properly formatted before use.

Used Datasets:

• small dataset: "ahazeemi/iwslt14-en-fr"
• medium dataset: "Helsinki-NLP/europarl", "en-fr"
• large dataset: "wmt/wmt14", "fr-en"

Notes

• Do not commit sensitive or proprietary data.
• Update this README if the folder structure changes.
• For questions about data formats, refer to the main project documentation.