| --- |
| license: mit |
| language: |
| - en |
| tags: |
| - text-generation |
| - gpt |
| - from-scratch |
| - conversational |
| --- |
| |
| # Cortex 2 chat preview |
|
|
| A small GPT model for conversational chat, trained completely from scratch (custom PyTorch architecture, no `transformers` library involved). |
|
|
| > ⚠️ **Note:** this is a **preview** release and may communicate worse than [Cortex 1 Chat](https://huggingface.co/VADRK155/Cortex-1-Chat). |
|
|
| - **Parameters**: 346m |
| - **Weights format**: fp16 |
| - **Architecture**: Decoder-only Transformer (custom implementation) |
| - **Training**: from scratch, no pretrained weights used |
|
|
| > ⚠️ **Note:** the context window is only **256 tokens**, so the model will forget earlier parts of the conversation quickly. |
|
|
| ## How to run |
|
|
| 2. Install dependencies: |
|
|
| pip install -r requirements.txt |
|
|
|
|
| 3. Run: |
|
|
| python chat.py |
|
|
|
|
| ## Limitations |
|
|
| This is a small model, so it may lose coherence, mix up facts, or cut off mid-thought — expected behavior for this parameter count. |
