Update README.md

README.md

@@ -7,51 +7,159 @@ tags:
 - small-language-model
 ---
 
+---
+
+library_name: talos
+tags:
+
+* talos
+* causal-language-model
+* custom-architecture
+* small-language-model
+* promethean-studios
+
+---
+
 # Talos 10M
 
-Talos 10M is a custom causal language model developed by Promethean Studios.
+> **A 9.95M-parameter language model built from scratch by Promethean Studios.**
+
+Talos 10M is a small, custom causal language model developed as part of the **Talos** project at **Promethean Studios**.
+
+This release contains the weights from a **100,000-step training run**, along with the tokenizer and configuration used during training.
+
+---
+
+## ✦ Model Overview
+
+| Property         |         Value |
+| ---------------- | ------------: |
+| **Parameters**   | **9,952,320** |
+| Architecture     |      TalosGPT |
+| Hidden size      |           448 |
+| Layers           |             3 |
+| Attention heads  |            28 |
+| KV heads         |            14 |
+| Head dimension   |            16 |
+| FFN size         |         1,792 |
+| Context length   |    512 tokens |
+| Model vocabulary |         1,024 |
+| Attention        |          Full |
+| RoPE theta       |        10,000 |
+
+---
+
+## ⚙ Training
+
+Talos 10M was trained for **100,000 steps**.
+
+| Training setting      |       Value |
+| --------------------- | ----------: |
+| Training steps        |     100,000 |
+| Validation loss       |  **1.1524** |
+| Batch size            |           8 |
+| Sequence length       |         512 |
+| Maximum learning rate |    3 × 10⁻⁴ |
+| Minimum learning rate |    3 × 10⁻⁵ |
+| Warmup                | 1,000 steps |
+| Weight decay          |         0.1 |
+| Seed                  |        1337 |
+
+The model was trained using a Tesla T4 GPU environment.
+
+---
+
+## 🧠 What is Talos?
+
+Talos is an experimental language-model project focused on building and understanding custom AI systems from the ground up.
+
+Rather than starting with a large pretrained model, Talos is developed as its own architecture, tokenizer, training pipeline, inference system, and evaluation workflow.
+
+The 10M model is one step in that progression.
+
+---
+
+## 📦 Repository Contents
+
+### `talos_10m_step_100000.pt`
+
+The trained PyTorch checkpoint containing the Talos 10M model weights and training state.
 
-## Model
+### `tokenizer.json`
 
-- Parameters: 9,952,320
-- Hidden size: 448
-- Layers: 3
-- Attention heads: 28
-- KV heads: 14
-- Head dimension: 16
-- FFN intermediate size: 1,792
-- Context length: 512 tokens
-- Model vocabulary: 1,024
-- Training tokenizer vocabulary: 512
-- Architecture: TalosGPT
-- Attention: Full attention
-- RoPE theta: 10,000
+The Talos-native ByteLevelBPE tokenizer used to produce the training data.
 
-## Training
+### `config.json`
 
-- Training steps: 100,000
-- Validation loss: 1.1523912596702575
-- Batch size: 8
-- Sequence length: 512
-- Maximum learning rate: 3e-4
-- Minimum learning rate: 3e-5
-- Warmup steps: 1,000
-- Weight decay: 0.1
-- Seed: 1337
+The model architecture and training configuration for this release.
 
-## Files
+---
+
+## 🔤 Tokenizer
+
+The model architecture contains **1,024 vocabulary slots**.
+
+The training run used a **512-vocabulary tokenizer**. The training data contained token IDs through **509**, leaving the remaining model vocabulary slots unused during this run.
+
+The tokenizer included in this repository is the exact tokenizer used to create the training tokens.
+
+---
+
+## 📈 100K Checkpoint
+
+This release represents the model after:
+
+**100,000 training steps**
+
+with a recorded validation loss of:
+
+**1.1523912596702575**
+
+At this stage, Talos is capable of producing readable, multi-paragraph text and following simple conversational formatting. Its generations can still be inconsistent, repetitive, or semantically incorrect, as expected from a model of this scale.
+
+---
+
+## 🛠 Architecture
 
-`talos_10m_step_100000.pt`
-- PyTorch training checkpoint containing the trained Talos 10M weights.
+Talos 10M uses a dense GPT-style architecture with:
 
-`tokenizer.json`
-- Talos-native ByteLevelBPE tokenizer used to produce the training tokens.
+* 3 transformer layers
+* 448-dimensional hidden states
+* 28 attention heads
+* 14 key/value heads
+* 16-dimensional attention heads
+* 1,792-dimensional feed-forward layers
+* Full attention
+* Rotary positional embeddings
+* 512-token maximum context
+
+Total parameter count:
+
+**9,952,320**
+
+---
+
+## 🚀 Using the Model
+
+Talos is a custom architecture and is **not currently a native Transformers model**.
+
+Inference is performed using the Talos codebase and its native tokenizer/generation implementation.
+
+The included checkpoint can be loaded into `TalosGPT` using the corresponding Talos configuration.
+
+---
+
+##  Project
+
+**Talos** is developed by **Promethean Studios**.
+
+This repository represents the trained 10M-parameter checkpoint from the Talos project.
+
+---
 
-`config.json`
-- Model and training configuration.
+### Release
 
-## Notes
+**Talos 10M — 100K**
 
-Talos is a custom architecture and is not a Transformers-native model.
+**9,952,320 parameters · 100,000 steps · 512-token context**
 
-The model was trained with a 512-vocabulary tokenizer while the model architecture contains 1,024 vocabulary slots. Training data used token IDs through 509.