Neithanvarvas commited on
Commit
a8c16c3
·
verified ·
1 Parent(s): db7b729

[DOCS] Clean LaTeX formatting for Hugging Face markdown compatibility

Browse files
Files changed (1) hide show
  1. README.md +11 -11
README.md CHANGED
@@ -23,10 +23,10 @@ pipeline_tag: text-generation
23
 
24
  **ScaledGPT** is an autoregressive Transformer model engineered to overcome the empirical limits of standard self-attention on sequential state machines and multi-tool agent execution.
25
 
26
- By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight-Tied Recurrent Depth ($R=3$)**, ScaledGPT achieves:
27
- - **100.0% Exact Match Accuracy** on 1 to 50 digit column arithmetic length extrapolation (including 50-digit cascading carry $99\dots99 + 1$).
28
  - **100.0% Multi-Tool Routing Accuracy** across 9 tool schemas (single-tool, multi-tool chaining, nested JSON schemas, and error recovery).
29
- - **Dual NVIDIA T4 GPU Efficiency**: Native Turing `sm_75` FP16 AMP and SDPA memory-efficient attention.
30
 
31
  ---
32
 
@@ -36,14 +36,14 @@ By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight
36
  | :--- | :--- |
37
  | **Total Parameters** | 18,944,512 (~18.95M) |
38
  | **Non-Embedding Parameters** | 18,887,680 (~18.89M) |
39
- | **Physical Layers ($L$)** | 6 |
40
- | **Recurrent Passes ($R$)** | 3 (Effective Depth = 18 layers) |
41
- | **Hidden Dimension ($d_{\text{model}}$)** | 512 |
42
- | **Attention Heads ($H$)** | 8 (Head dimension = 64) |
43
- | **Max Sequence Length ($T$)** | 1024 |
44
- | **Positional Embedding** | Decoupled 2D-RoPE ($d/2$ macro column, $d/2$ micro step) |
45
  | **Vocabulary Size** | 115 (Special discourse markers + ASCII characters) |
46
- | **Weight Tying** | $W_{\text{te}} == W_{\text{lm\_head}}$ |
47
 
48
  ---
49
 
@@ -55,7 +55,7 @@ By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight
55
  | **16-25 Digit Intermediate Extrapolation** | Exact Answer Match | 100.0% (6/6) | [PASS] |
56
  | **30-40 Digit High Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
57
  | **45-50 Digit Ultra Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
58
- | **50-Digit Cascade Carry ($99\dots99 + 1$)** | 50 Sequential Transitions | 100.0% (1/1) | [PASS] |
59
  | **Multi-Tool Routing Suite (9 Schemas)** | Correct Tool Selection | 100.0% (26/26) | [PASS] |
60
  | **Multi-Tool Exact Argument Extraction** | Exact Parameter Copy Rate | 96.2% (25/26) | [PASS] |
61
 
 
23
 
24
  **ScaledGPT** is an autoregressive Transformer model engineered to overcome the empirical limits of standard self-attention on sequential state machines and multi-tool agent execution.
25
 
26
+ By combining **Decoupled 2D Rotary Position Embeddings (2D-RoPE)** with **Weight-Tied Recurrent Depth (R = 3)**, ScaledGPT achieves:
27
+ - **100.0% Exact Match Accuracy** on 1 to 50 digit column arithmetic length extrapolation (including 50-digit cascading carry 99...99 + 1).
28
  - **100.0% Multi-Tool Routing Accuracy** across 9 tool schemas (single-tool, multi-tool chaining, nested JSON schemas, and error recovery).
29
+ - **Dual NVIDIA T4 GPU Efficiency**: Native Turing sm_75 FP16 AMP and SDPA memory-efficient attention.
30
 
31
  ---
32
 
 
36
  | :--- | :--- |
37
  | **Total Parameters** | 18,944,512 (~18.95M) |
38
  | **Non-Embedding Parameters** | 18,887,680 (~18.89M) |
39
+ | **Physical Layers (L)** | 6 |
40
+ | **Recurrent Passes (R)** | 3 (Effective Depth = 18 layers) |
41
+ | **Hidden Dimension (d_model)** | 512 |
42
+ | **Attention Heads (H)** | 8 (Head dimension = 64) |
43
+ | **Max Sequence Length (T)** | 1024 |
44
+ | **Positional Embedding** | Decoupled 2D-RoPE (d/2 macro column, d/2 micro step) |
45
  | **Vocabulary Size** | 115 (Special discourse markers + ASCII characters) |
46
+ | **Weight Tying** | W_te == W_lm_head |
47
 
48
  ---
49
 
 
55
  | **16-25 Digit Intermediate Extrapolation** | Exact Answer Match | 100.0% (6/6) | [PASS] |
56
  | **30-40 Digit High Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
57
  | **45-50 Digit Ultra Extrapolation** | Exact Answer Match | 100.0% (4/4) | [PASS] |
58
+ | **50-Digit Cascade Carry (99...99 + 1)** | 50 Sequential Transitions | 100.0% (1/1) | [PASS] |
59
  | **Multi-Tool Routing Suite (9 Schemas)** | Correct Tool Selection | 100.0% (26/26) | [PASS] |
60
  | **Multi-Tool Exact Argument Extraction** | Exact Parameter Copy Rate | 96.2% (25/26) | [PASS] |
61