[DOCS] Clean LaTeX formatting for Hugging Face markdown compatibility
Browse files
README.md
CHANGED
|
@@ -23,10 +23,10 @@ pipeline_tag: text-generation
|
|
| 23 |
|
| 24 |
**ScaledGPT** is an autoregressive Transformer model engineered to overcome the empirical limits of standard self-attention on sequential state machines and multi-tool agent execution.
|
| 25 |
|
| 26 |
-
By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight-Tied Recurrent Depth (
|
| 27 |
-
- **100.0% Exact Match Accuracy** on 1 to 50 digit column arithmetic length extrapolation (including 50-digit cascading carry
|
| 28 |
- **100.0% Multi-Tool Routing Accuracy** across 9 tool schemas (single-tool, multi-tool chaining, nested JSON schemas, and error recovery).
|
| 29 |
-
- **Dual NVIDIA T4 GPU Efficiency**: Native Turing
|
| 30 |
|
| 31 |
---
|
| 32 |
|
|
@@ -36,14 +36,14 @@ By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight
|
|
| 36 |
| :--- | :--- |
|
| 37 |
| **Total Parameters** | 18,944,512 (~18.95M) |
|
| 38 |
| **Non-Embedding Parameters** | 18,887,680 (~18.89M) |
|
| 39 |
-
| **Physical Layers (
|
| 40 |
-
| **Recurrent Passes (
|
| 41 |
-
| **Hidden Dimension (
|
| 42 |
-
| **Attention Heads (
|
| 43 |
-
| **Max Sequence Length (
|
| 44 |
-
| **Positional Embedding** | Decoupled 2D-RoPE (
|
| 45 |
| **Vocabulary Size** | 115 (Special discourse markers + ASCII characters) |
|
| 46 |
-
| **Weight Tying** |
|
| 47 |
|
| 48 |
---
|
| 49 |
|
|
@@ -55,7 +55,7 @@ By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight
|
|
| 55 |
| **16-25 Digit Intermediate Extrapolation** | Exact Answer Match | 100.0% (6/6) | [PASS] |
|
| 56 |
| **30-40 Digit High Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
|
| 57 |
| **45-50 Digit Ultra Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
|
| 58 |
-
| **50-Digit Cascade Carry (
|
| 59 |
| **Multi-Tool Routing Suite (9 Schemas)** | Correct Tool Selection | 100.0% (26/26) | [PASS] |
|
| 60 |
| **Multi-Tool Exact Argument Extraction** | Exact Parameter Copy Rate | 96.2% (25/26) | [PASS] |
|
| 61 |
|
|
|
|
| 23 |
|
| 24 |
**ScaledGPT** is an autoregressive Transformer model engineered to overcome the empirical limits of standard self-attention on sequential state machines and multi-tool agent execution.
|
| 25 |
|
| 26 |
+
By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight-Tied Recurrent Depth (R = 3)**, ScaledGPT achieves:
|
| 27 |
+
- **100.0% Exact Match Accuracy** on 1 to 50 digit column arithmetic length extrapolation (including 50-digit cascading carry 99...99 + 1).
|
| 28 |
- **100.0% Multi-Tool Routing Accuracy** across 9 tool schemas (single-tool, multi-tool chaining, nested JSON schemas, and error recovery).
|
| 29 |
+
- **Dual NVIDIA T4 GPU Efficiency**: Native Turing sm_75 FP16 AMP and SDPA memory-efficient attention.
|
| 30 |
|
| 31 |
---
|
| 32 |
|
|
|
|
| 36 |
| :--- | :--- |
|
| 37 |
| **Total Parameters** | 18,944,512 (~18.95M) |
|
| 38 |
| **Non-Embedding Parameters** | 18,887,680 (~18.89M) |
|
| 39 |
+
| **Physical Layers (L)** | 6 |
|
| 40 |
+
| **Recurrent Passes (R)** | 3 (Effective Depth = 18 layers) |
|
| 41 |
+
| **Hidden Dimension (d_model)** | 512 |
|
| 42 |
+
| **Attention Heads (H)** | 8 (Head dimension = 64) |
|
| 43 |
+
| **Max Sequence Length (T)** | 1024 |
|
| 44 |
+
| **Positional Embedding** | Decoupled 2D-RoPE (d/2 macro column, d/2 micro step) |
|
| 45 |
| **Vocabulary Size** | 115 (Special discourse markers + ASCII characters) |
|
| 46 |
+
| **Weight Tying** | W_te == W_lm_head |
|
| 47 |
|
| 48 |
---
|
| 49 |
|
|
|
|
| 55 |
| **16-25 Digit Intermediate Extrapolation** | Exact Answer Match | 100.0% (6/6) | [PASS] |
|
| 56 |
| **30-40 Digit High Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
|
| 57 |
| **45-50 Digit Ultra Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
|
| 58 |
+
| **50-Digit Cascade Carry (99...99 + 1)** | 50 Sequential Transitions | 100.0% (1/1) | [PASS] |
|
| 59 |
| **Multi-Tool Routing Suite (9 Schemas)** | Correct Tool Selection | 100.0% (26/26) | [PASS] |
|
| 60 |
| **Multi-Tool Exact Argument Extraction** | Exact Parameter Copy Rate | 96.2% (25/26) | [PASS] |
|
| 61 |
|