Neural Network-based Next Token Prediction Model Overview
This project implements a neural network-based model to predict the next token in a sequence using both English and Georgian datasets. The model utilizes LSTM layers for handling sequential text data and is designed to generate text by predicting the next word/token based on previous input.
Model Architecture
Embedding Layer: Captures word embeddings. LSTM Layers: Two LSTM layers (150 and 100 units) to process sequential data. Dense Layer: Fully connected layer with softmax activation for predicting the next token.
Datasets
Languages: English and Georgian text combined for multilingual prediction. Preprocessing: Tokenization and padding applied to input sequences.
Training Details
Optimizer: RMSprop Loss Function: Categorical Crossentropy Training Epochs: 100 Batch Size: 16
Results
The model achieves a test accuracy of . Future work could include improving the dataset size and experimenting with different architectures to enhance performance.
- Downloads last month
- 3