[UPDATE] Update CBT-18.95M paper, model card, and 16-100d random extrapolation results

RESEARCH_PAPER.md

@@ -6,13 +6,13 @@
 
 ## Abstract
 
-Small autoregressive Transformers fail on multi-digit addition beyond the sequence lengths seen during training, with standard 1D positional baselines collapsing to 0.0% accuracy on unseen lengths. We find an effective solution in two-dimensional positional coordinates (**Decoupled 2D-RoPE**), which preserve the place-value structure of digits invariant to problem length and eliminate positional phase drift. Hypothesizing that this structural principle extends to text and discourse, we apply 2D coordinates directly to atomic characters and multi-turn discourse tags rather than standard subword tokens. Combining character-level 2D coordinates, coordinate shift masking, and weight-tied recurrence into an 18.95M-parameter model (**CBT-18.95M**), we achieve strong algorithmic and tool-routing results at small scale: solving 100% of seen 1–15 digit sums, generalizing zero-shot to 98.8% (237/240) on unseen 16–50 digit random multi-carry additions, and maintaining 100% (26/26) structured tool dispatch within a compact 36 MB edge footprint.
+Character-level Transformers, which avoid subword digit merging, still fail to extrapolate multi-digit addition under standard 1D positional encodings (scoring 4.6% on 14–16 digits and collapsing to 0.0% on 18- to 20-digit sums in our experiments). We show that supplying place-value coordinates through **Decoupled 2D-RoPE** removes this failure at small scale, and that the same coordinate scheme extends to multi-turn tool-calling dialogues without degrading them. In the 18.95M-parameter **Character-Based Transformer (CBT-18.95M)**, coordinates are assigned by a deterministic rule-based parser within the forward pass, coupled with coordinate shift masking and weight-tied recurrence ($R=3$, adopted as an architectural choice for parameter-efficient depth rather than an addition-specific mechanism). Evaluated on single training runs, CBT-18.95M achieves 100% on seen 1–15 digit additions, 100.0% (180/180) zero-shot exact match on unseen 16–40 digit random multi-carry sums within shifted coordinate support ($p \le 47$), 95.0% (57/60) at the support boundary (45 and 50 digits; pooled 98.8%, 237/240 on 16–50 digits), and 100% (26/26) structured tool dispatch on template-matched phrasing (50.0% exact match on out-of-distribution entity copying) within a 36 MB FP16 footprint.
 
 ---
 
 ## 1. Introduction
 
-A foundational question in machine learning is whether autoregressive neural networks can internalize and execute exact sequential algorithms invariant to sequence length. The prevailing paradigm in contemporary language modeling scales autoregressive Transformers [1, 4] to tens or hundreds of billions of parameters over subword Byte-Pair Encoding (BPE) vocabularies. While these models exhibit remarkable fluency on conversational tasks, small Transformers trained from scratch notoriously fail on basic algorithmic tasks like multi-digit addition when tested beyond the sequence lengths seen during training [5, 6, 7, 8, 13, 14]. Frontier models with chain-of-thought or tool access do considerably better; our statements concern small models trained from scratch. While such a Transformer can memorize or interpolate addition patterns up to its training ceiling $N_{\text{train}}$, its accuracy collapses to 0.0% as the number of digits $N$ exceeds that horizon.
+A foundational question in machine learning is whether autoregressive neural networks can internalize and execute exact sequential algorithms invariant to sequence length. The prevailing paradigm in contemporary language modeling scales autoregressive Transformers [1, 4] to tens or hundreds of billions of parameters over subword Byte-Pair Encoding (BPE) vocabularies. While these models exhibit remarkable fluency on conversational tasks, small Transformers trained from scratch notoriously fail on basic algorithmic tasks like multi-digit addition when tested beyond the sequence lengths seen during training [5, 6, 7, 8, 13, 14]. Frontier models with chain-of-thought or tool access do considerably better; our statements concern small models trained from scratch. While such a Transformer can memorize or interpolate addition patterns up to its training ceiling $N_{\text{train}}$, its accuracy drops precipitously beyond that boundary (falling to 4.6% on 14–16 digits and collapsing to 0.0% on 18- to 20-digit sums in our experiments).
 
 ### 1.1 Algorithmic Arithmetic and the Success of 2D Embeddings
 
@@ -22,7 +22,7 @@ $$s_i = (a_i + b_i + c_{i-1}) \bmod 10, \quad c_i = \left\lfloor \frac{a_i + b_i
 
 where $a_i, b_i \in \{0, \dots, 9\}$ are the single-digit operands at column $i$, $c_i \in \{0, 1\}$ is the binary carry bit emitted to column $i+1$, and $c_{-1} = 0$. Because the entire transition table consists of only $10 \times 10 \times 2 = 200$ discrete rules, executing addition requires $\mathcal{O}(N)$ identical state transitions regardless of whether $N=4$, $N=15$, or $N=50$.
 
-When we trained a standard 18.91M Transformer ($d_{\text{model}}=512, L=6, n_{\text{head}}=8$) with standard one-dimensional rotary embeddings (1D-RoPE [2, 14]) on 1- to 12-digit additions with explicit column scratchpads [9, 10], it failed completely beyond its training horizon: achieving 98.4% on seen 1–12 digits and 4.6% on moderate 14–16 digits, before falling to **0.0% accuracy on unseen 18- to 20-digit sums** (Table 7). Diagnosing these runs revealed that 1D positional encodings assign a flat, monotonically increasing index $t = 0, \dots, T-1$. At unseen sequence lengths ($T \approx 600$ characters for 50 digits), relative distances $(m - n)$ between an answer column and its operand digits take values never encountered during training, inducing positional phase drift and repetitive zero loops where the model repeats blocks like `(0+0+0)=00` indefinitely:
+When we trained a standard 18.91M Transformer ($d_{\text{model}}=512, L=6, n_{\text{head}}=8$) with standard one-dimensional rotary embeddings (1D-RoPE [2, 14]) on 1- to 12-digit additions with explicit column scratchpads [9, 10], it failed completely beyond its training horizon: achieving 98.4% on seen 1–12 digits and 4.6% on moderate 14–16 digits, before falling to **0.0% accuracy on unseen 18- to 20-digit sums** (Table 7). Diagnosing these runs revealed that 1D positional encodings assign a flat, monotonically increasing index $t = 0, \dots, T-1$. At unseen sequence lengths ($T \approx 715$ characters for 50 digits, scaling at $\approx 14.3$ tokens per digit), relative distances $(m - n)$ between an answer column and its operand digits take values never encountered during training, inducing positional phase drift and repetitive zero loops where the model repeats blocks like `(0+0+0)=00` indefinitely:
 
 $$P\big(\texttt{"(0+0+0)=00"} \;\big|\; \texttt{"(0+0+0)=00"}^{\times m}\big) \approx 1 \tag{2}$$
 
@@ -32,8 +32,6 @@ To overcome this failure, we structured the problem onto a **two-dimensional (2D
 
 ### 1.2 From 2D Digits to 2D Characters: The Core Hypothesis
 
-Having found that 2D embeddings succeed at arithmetic because they prevent losing the internal place-value structure of digits across sequence length, we formed a central hypothesis: *Could the exact same structural principle be applied to atomic characters and natural text/tool discourse?*
-
 Standard language models rely on subword BPE tokenization, which clumps arbitrary character and digit substrings into irregular multi-character tokens:
 
 ```text
@@ -41,11 +39,15 @@ Standard language models rely on subword BPE tokenization, which clumps arbitrar
 "87654321" --BPE--> ["8", "7", "65", "43", "21"]
 ```
 
-A single token like `"45"` fuses the $10^3$ place (`4`) and the $10^2$ place (`5`) into one vector at one 1D index, breaking 1-to-1 place-value alignment between operands (Table 6). More broadly, subword merges obscure morphological and entity boundaries in natural language. We hypothesized that if we:
-1. decompose inputs into **atomic characters** paired with 14 structural discourse tags (e.g., `<USER>`, `<THINK>`, `<CALL>`, `<RESULT>`, `<ANSWER>`), and
-2. place these characters onto a **2D discourse matrix** (Table 1B), where macro row $p \in \{0..4\}$ indexes the functional discourse turn and micro column $\tau$ tracks character position within that turn,
+A single token like `"45"` fuses the $10^3$ place (`4`) and the $10^2$ place (`5`) into one vector at one 1D index, breaking 1-to-1 place-value alignment between operands (Table 6).
+
+While character-level Transformers avoid these subword merging artifacts and preserve character-level boundaries, they still fail to extrapolate multi-digit addition under standard 1D positional encodings, succumbing to identical phase drift. Having established that Decoupled 2D-RoPE fixes addition collapse by maintaining invariant decimal place-value coordinates, we formulated our central hypothesis: *Supplying place-value coordinates through Decoupled 2D-RoPE directly to atomic characters removes this extrapolation collapse in character-level models, and this exact coordinate scheme can be extended to multi-turn tool-calling dialogues without degrading them.*
+
+To test this, we unify arithmetic and structured dialogue into a single character representation:
+1. We decompose inputs into **atomic characters** paired with 14 structural discourse tags (e.g., `<USER>`, `<THINK>`, `<CALL>`, `<RESULT>`, `<ANSWER>`).
+2. In arithmetic mode, characters follow decimal column coordinates $(p, \tau)$ (Table 1A). In dialogue mode, characters are organized onto a **2D discourse matrix** (Table 1B), where macro row $p \in \{0..4\}$ indexes the functional discourse turn and micro column $\tau$ tracks character position within that turn.
 
-then 2D coordinates would preserve the fine-grained structural integrity of words, entity boundaries, and tool syntax without positional degradation, just as they preserve decimal place-value structure in multi-digit sums.
+We emphasize that discourse-mode coordinates ($p = \text{turn index}$) represent an architectural design choice to permit unified forward execution across arithmetic and tools—not a claim that 2D coordinates improve natural language text or tool routing over 1D. Indeed, our tokenizer ablations (Table 6) show that a standard 1D atomic-character model already scores 100% (50/50) on structured tool calling. Evaluating tool agency serves to demonstrate that Decoupled 2D-RoPE does not compromise standard character-level capabilities, allowing a single compact model to perform both tasks without mode switching.
 
 ---
 
@@ -75,7 +77,7 @@ To test this hypothesis and achieve maximal algorithmic and tool performance wit
 1. **Atomic Character Tokenization ($V=115$)**: Preserves exact character boundaries, eliminating subword tokenization chunking and enabling faithful 1-hop pointer copying for tool arguments.
 2. **Decoupled 2D-RoPE**: Rotates half of each attention head with macro row $p$ and the other half with micro column $\tau$, anchoring cross-attention invariant to sequence length in both arithmetic and discourse modes.
 3. **Coordinate Shift Masking ($k=32$)**: Randomly shifts macro coordinates during training, exposing rotary frequencies to an extended coordinate horizon ($p_{\max} = N_{\text{train}} + k = 15 + 32 = 47$) to enable zero-shot extrapolation up to 50 digits.
-4. **Weight-Tied Recurrent Depth ($R=3$)**: Unrolls $L=6$ physical layers across $R=3$ recurrent passes, effectively tripling depth to 18 layers without adding weights, providing the non-linear computational capacity needed for multi-step tool reasoning and schema dispatch within a 36 MB footprint.
+4. **Weight-Tied Recurrent Depth ($R=3$)**: Unrolls $L=6$ physical Transformer blocks across $R=3$ recurrent passes ($L_{\text{eff}}=18$), adopted as an architectural choice to provide deep non-linear representation within a compact 36 MB FP16 footprint without parameter inflation. As detailed in Section 1.4 and our 100k ablations (Table 8), recurrence is not claimed as an empirically necessary mechanism for addition (which extrapolates under 2D-RoPE alone at $R=1$) or tool routing.
 
 ### 1.4 Scope, Limitations, and Utility on Low-Resource Devices
 
@@ -85,7 +87,7 @@ To test this hypothesis and achieve maximal algorithmic and tool performance wit
 1. The coordinates $(p,\tau)$ are hand-designed and assigned by a rule-based parser, not learned; the length-generalization result is therefore conditional on correct coordinate assignment (Section 3.3) and demonstrates column-aligned addition with a fixed-template scratchpad rather than general algorithmic induction.
 2. Tool calling is bound to a small set of templates that overlap with training data, the OOD set contains only 18 prompts, near-exact copies are functionally failures, and we do not compare against a fine-tuned small off-the-shelf model.
 3. Tool and 12d-ablation test sets are small (26 and 18 for tools, 14 for Table 8; for arithmetic we supplement our 20 structured cases with a 315-problem random sweep on CBT-18.95M in Table 3B and a 12,800-problem random sweep across four equalized 1–15d, $T=1024$ models in Table 8B), every configuration is a single training run, and we report no seed variance.
-4. In Table 7, the 18.91M 1D baselines were trained on 1–12 digits without position-offset augmentation (to address this confound directly, Table 8B trains equalized 100k 1D-RoPE, 1D-Offset, Ruoss et al. randomized-position, and 2D-RoPE models on identical 1–15 digit data with $T=1024$), several Table 7 baselines are strawmen for arithmetic (e.g., naive BPE without digit splitting), and our Abacus baseline (0.0% in-distribution) reflects an under-tuned implementation rather than the method of McLeish et al. [13].
+4. In Table 7, the 18.91M 1D baselines were trained on 1–12 digits with a 512-token context window; because scratchpad traces require $\approx 14.3$ characters per digit, traces for 25 digits ($\approx 360$ tokens) and 35 digits ($\approx 500$ tokens) fit within this 512-token window, where the baselines fail from positional drift rather than context clipping, whereas 50-digit traces ($\approx 715$ tokens) exceed 512 tokens (Table 8B addresses this with equalized 1024-token models). At 70 digits ($\approx 1000$ tokens) and 80 digits ($\approx 1149$ tokens), traces approach or exceed the 1024 context window, confounding arithmetic failure with context truncation. Several Table 7 baselines are strawmen for arithmetic (e.g., naive BPE without digit splitting), and our Abacus baseline (0.0% in-distribution) reflects an under-tuned implementation rather than the method of McLeish et al. [13].
 5. The only algorithmic task is addition; we do not test tasks where coordinates cannot be assigned by a simple parser (such as multiplication or sorting).
 6. On a laptop CPU (Intel Core Ultra 5 125H, FP32), batched KV-cached inference runs at 141–394 tokens/sec (1.1–5.3 tokens/sec uncached single-stream), and we do not benchmark mobile ARM hardware or INT8 quantization.
 7. Recurrent depth showed no benefit in our addition ablation (Table 8), and the 1.56M HRM-RoPE model (Table 3) also extrapolates to 20 digits using Decoupled 2D-RoPE alone, confirming that the 2D coordinate mapping—not recurrence or the multi-stage curriculum��is the primary driver of addition extrapolation.
@@ -122,7 +124,7 @@ Scores are raw accuracies with 95% Wilson confidence intervals (no pass/fail thr
 ---
 
 **Table 3: Itemized Arithmetic Length Extrapolation Benchmark (1 to 50 Digits), Separating Seen Training Lengths from Unseen Zero-Shot Extrapolation Lengths**  
-Standard 1D-Char and 50k-BPE models (18.91M) and Dual-Loop HRM-RoPE (1.56M) were trained on **1–12 digits** with a 512-token context window; our **CBT-18.95M** was trained on **1–15 digits** with a 1024-token context window. Part II (16–20 digits) is the clean comparison within the 512-token window: HRM-RoPE (1.56M) extrapolates to 20 digits using Decoupled 2D-RoPE without CBT's three-stage curriculum, whereas the 1D models fail. In Part III (25–50 digits), all 512-token models exceed their training context length and are marked `N/A (Context Limit)`.
+Standard 1D-Char and 50k-BPE models (18.91M) and Dual-Loop HRM-RoPE (1.56M) were trained on **1–12 digits** with a 512-token context window; our **CBT-18.95M** was trained on **1–15 digits** with a 1024-token context window. Part II (16–20 digits) is the clean comparison within the 512-token window: HRM-RoPE (1.56M) extrapolates to 20 digits using Decoupled 2D-RoPE without CBT's three-stage curriculum, whereas the 1D models fail. In Part III (25–50 digits), 25-digit traces ($\approx 360$ tokens) and 35-digit traces ($\approx 500$ tokens) fit within the 512-token context window of the 12d baselines, where 1D-Char and HRM-RoPE fail due to positional drift rather than context clipping; only 50-digit traces ($\approx 715$ tokens) exceed 512 tokens. Extending 1D-Char's frequency table to $T=1024$ still fails due to drift; for strictly equalized 1–15 digit, $T=1024$ training comparisons (including Ruoss et al. randomized positions), see Table 8B.
 
 | Digit Length | Training Exposure Status | Representative Test Problem | Expected Output | 18.91M 1D-Char | 18.91M 50k-BPE | 1.56M HRM-RoPE | CBT-18.95M (Ours) |
 | :--- | :--- | :--- | :---: | :---: | :---: | :---: | :---: |
@@ -142,11 +144,11 @@ Standard 1D-Char and 50k-BPE models (18.91M) and Dual-Loop HRM-RoPE (1.56M) were
 | 20 Digits | **Unseen OOD (+5d / +8d)** | `1234...90 + 8765...09=` (20d) | $20\times\texttt{'9'}$ | `Drift [FAIL]` | `Drift [FAIL]` | $20\times\texttt{'9'}$ `[PASS]` | $20\times\texttt{'9'}$ `[PASS]` |
 | 20 Digits | **Unseen OOD (+5d / +8d)** | $10^{19} + 10^{19}=$ | $2\times 10^{19}$ | `Loops 000... [FAIL]` | `Drift [FAIL]` | $2\times 10^{19}$ `[PASS]` | $2\times 10^{19}$ `[PASS]` |
 | 20 Digits | **Unseen OOD (+5d / +8d)** | $\underbrace{\texttt{999...999}}_{20} + \texttt{1}=$ | $10^{20}$ (21 digits) | `Loops 000... [FAIL]` | `Drift [FAIL]` | $10^{20}$ `[PASS]` | $10^{20}$ `[PASS]` |
-| **Part III: Unseen Ultra-Long Horizon Extrapolation ($25 \le N \le 50$ Digits — Exceeds 512-Token Training Context of 12d Models)** | | | | | | | |
-| 25 Digits | **Unseen OOD (+10 Extra Digits)** | $\underbrace{\texttt{999...999}}_{25} + \texttt{1}=$ | $10^{25}$ (26 digits) | `Drift [FAIL at T=1024]` | `N/A (Context Limit)` | `N/A (Context Limit)` | $10^{25}$ `[PASS]` |
-| 35 Digits | **Unseen OOD (+20 Extra Digits)** | $\underbrace{\texttt{111...111}}_{35} + \underbrace{\texttt{888...888}}_{35}=$ | $35\times\texttt{'9'}$ | `Drift [FAIL at T=1024]` | `N/A (Context Limit)` | `N/A (Context Limit)` | $35\times\texttt{'9'}$ `[PASS]` |
-| 50 Digits | **Unseen OOD (+35 Extra Digits)** | `123...890 + 876...109=` (50d) | $50\times\texttt{'9'}$ | `Drift [FAIL at T=1024]` | `N/A (Context Limit)` | `N/A (Context Limit)` | $50\times\texttt{'9'}$ `[PASS]` |
-| 50 Digits | **Unseen OOD (+35 Extra Digits)** | $\underbrace{\texttt{999...999}}_{50} + \texttt{1}=$ | $10^{50}$ (51 digits) | `Drift [FAIL at T=1024]` | `N/A (Context Limit)` | `N/A (Context Limit)` | $10^{50}$ `[PASS]` |
+| **Part III: Unseen Ultra-Long Horizon Extrapolation ($25 \le N \le 50$ Digits)** | | | | | | | |
+| 25 Digits | **Unseen OOD (+10 Extra Digits)** | $\underbrace{\texttt{999...999}}_{25} + \texttt{1}=$ | $10^{25}$ (26 digits) | `Drift [FAIL]` | `Hallucinates [FAIL]` | `Drift [FAIL]` | $10^{25}$ `[PASS]` |
+| 35 Digits | **Unseen OOD (+20 Extra Digits)** | $\underbrace{\texttt{111...111}}_{35} + \underbrace{\texttt{888...888}}_{35}=$ | $35\times\texttt{'9'}$ | `Drift [FAIL]` | `Hallucinates [FAIL]` | `Drift [FAIL]` | $35\times\texttt{'9'}$ `[PASS]` |
+| 50 Digits | **Unseen OOD (+35 Extra Digits)** | `123...890 + 876...109=` (50d) | $50\times\texttt{'9'}$ | `Drift [FAIL at T=1024]` | `N/A (Context: 715 > 512)` | `N/A (Context: 715 > 512)` | $50\times\texttt{'9'}$ `[PASS]` |
+| 50 Digits | **Unseen OOD (+35 Extra Digits)** | $\underbrace{\texttt{999...999}}_{50} + \texttt{1}=$ | $10^{50}$ (51 digits) | `Drift [FAIL at T=1024]` | `N/A (Context: 715 > 512)` | `N/A (Context: 715 > 512)` | $10^{50}$ `[PASS]` |
 
 ---
 
@@ -446,7 +448,7 @@ Table 10 summarizes the three-stage training protocol that produced `champion_fi
 
 ### 3.6 Hyperparameters and Release
 
-Defaults in the training code (`train_and_eval.py` and `sft_search.py` under `champion_model/scaled_t4/`): AdamW with weight decay 0.01, peak learning rate $10^{-3}$ with cosine decay, gradient-norm clipping at 1.0, per-GPU batch size 32 on 2 GPUs, context length (block size) 1024 tokens with up to 4096 coordinate indices, $k=32$ shift range, $R=3$, FP16 autocast with gradient scaling. Stage 3 SFT uses a learning rate from $5\times10^{-5}$ to $5\times10^{-6}$. Context length is not the limiting factor at 50 digits (about 10 characters per scratchpad column gives roughly 600 tokens), but the 1D baselines were trained with a 512-token block, so their 20+ digit failures are partly confounded with context length; their failures at 14–16 digits are not. Arithmetic and tool data are produced by procedural generators in the repository (`src/arithmetic.py` and the dataset modules); the code, data generators, and checkpoints are available in the project repository.
+Defaults in the training code (`train_and_eval.py` and `sft_search.py` under `champion_model/scaled_t4/`): AdamW with weight decay 0.01, peak learning rate $10^{-3}$ with cosine decay, gradient-norm clipping at 1.0, per-GPU batch size 32 on 2 GPUs, context length (block size) 1024 tokens with up to 4096 coordinate indices, $k=32$ shift range, $R=3$, FP16 autocast with gradient scaling. Stage 3 SFT uses a learning rate from $5\times10^{-5}$ to $5\times10^{-6}$. At $\approx 14.3$ characters per digit, a 50-digit trace requires $\approx 715$ tokens (fitting within CBT's 1024 context window, but exceeding the 512-token window of the 12-digit baselines). Traces for 25 digits ($\approx 360$ tokens) and 35 digits ($\approx 500$ tokens) fit within 512 tokens, where the 12-digit baselines fail from positional drift rather than context clipping. At 70 digits ($\approx 1000$ tokens) and 80 digits ($\approx 1149$ tokens), traces approach or exceed 1024 tokens, confounding arithmetic accuracy with context truncation. The code, data generators, and evaluation scripts are available at [github.com/NeithanT/CBT](https://github.com/NeithanT/CBT), and model checkpoints are hosted at [huggingface.co/Neithanvarvas/CBT](https://huggingface.co/Neithanvarvas/CBT) (primary checkpoint `champion_final.pt` SHA256: `79d908d3...ccaa`; 18.91M 1D-Char `model_1_final.pt` SHA256: `06411d79...0c1f`; 1.56M HRM-RoPE `hrm_rope_1000_examples.pt` SHA256: `18515686...f6ee`).
 
 ---
 
@@ -481,7 +483,7 @@ Starting from the failure of a standard 18.91M-parameter Transformer and eight a
 
 ## Disclaimer: Use of AI and Compute Infrastructure
 
-- **Use of AI Assistance**: Portions of the experimental engineering, code implementation, data pipeline structuring, and manuscript drafting/formatting in this research were conducted with the assistance of **Gemini 3.8 Flash**. All experimental hypotheses, architectural selections, empirical benchmarks, and final scientific conclusions were verified and validated by the author.
+- **Use of AI Assistance**: Portions of the experimental engineering, code implementation, data pipeline structuring, and manuscript drafting/formatting in this research were conducted with the assistance of **Google Gemini** via the Antigravity agentic coding environment. All experimental hypotheses, architectural selections, empirical benchmarks, and final scientific conclusions were verified and validated by the author.
 - **Compute Hardware**: All model pretraining, supervised fine-tuning (SFT), and isolated component ablation runs reported in this paper were executed on **Kaggle's NVIDIA Tesla T4 GPUs** (16 GB GDDR6 VRAM, Turing `sm_75` architecture) using PyTorch native FP16 automatic mixed precision (`torch.float16`).
 
 ---

RESEARCH_PAPER.pdf

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8504337ea49b5fa4edd1428fdd23560ded91c8621e5c36ce8a5e0f35e634fd5
-size 462235
+oid sha256:6dce8fcc3bd88c30999e185bf6f2cf1c9a05c13e1c18b24ed34e3269ee1d1c07
+size 463749