File size: 792 Bytes
b22e8ba
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# Data Folder

This `data` folder contains datasets and resources used for the Mini-Transformer project.

## Structure
- **corpus/**: Text corpora for training and evaluation.
- **raw/**: Original, unprocessed datasets.
- **processed/**: Cleaned and preprocessed data ready for model training.
- **README.md**: Documentation for the data folder.

## Usage

Place your datasets in the appropriate subfolders. Ensure data is properly formatted before use.
## Used Datasets:
- small dataset: "ahazeemi/iwslt14-en-fr"
- medium dataset: "Helsinki-NLP/europarl", "en-fr"
- large dataset: "wmt/wmt14", "fr-en"

## Notes

- Do not commit sensitive or proprietary data.
- Update this README if the folder structure changes.
- For questions about data formats, refer to the main project documentation.