# 🧠 CooperLM-354M
**CooperLM-354M** is a 354 million parameter GPT-2 based language model trained from scratch on a filtered subset of Wikipedia, BookCorpus, and OpenWebText. It was created as a toy project to explore end-to-end LLM training using Hugging Face’s Transformers and Datasets libraries.
---
## 🧱 Architecture
- GPT-2 with 24 layers, 16 heads, 1024 hidden size
- 256-token context window
- Trained for 1 epoch on 100k samples (~1.2M sequences)
---
## 📊 Training Details
| Setting | Value |
|------------------------|-----------------|
| Model Type | GPT2LMHeadModel |
| Epochs | 1 |
| Precision | fp16 |
| Batch Size (effective) | 16 |
| GPU | RTX 4080 |
| Final Eval Loss | 5.63 |
| Perplexity | ~263 |
---
## 📥 Usage
```python
from transformers import GPT2LMHeadModel, GPT2TokenizerFast
model = GPT2L…
