Update README.md

README.md

@@ -11,10 +11,13 @@
   <a href="https://tinker.thinkingmachines.ai/playground">Launch Blog</a>
 </p>
 
-We introduce **Ripple-65B Preview**, a 65B parameter dense Transformer model pretrained on 12 trillion tokens that scores [PLACEHOLDER] on TerminalBench 2.1,
-surpassing significantly-larger US open-weight models such as Reflection Beam 501B (80.1), Muse Spark Open 816B (76.4), Inkling-975B (63.8), and Nemotron 3 Ultra 550B (56.4). 
+Ricursive Intelligence introduces **Ripple-65B Preview**, a 65B parameter dense Transformer model pretrained on 12 trillion tokens that scores **80.52 on TerminalBench 2.1**,
+surpassing significantly-larger US open-weight models such as Reflection Beam 501B (**80.1**), Meta Muse Spark Open 816B (**76.4**), 
+Thinking Machines Inkling 975B (**63.8**), and NVIDIA Nemotron 3 Ultra 550B (**56.4**). 
 
-Starting from an initial Olmo 3 baseline, we iteratively improved all aspects of the training recipe, including the pretraining scaling laws, model architecture, and data.
+Our Preview release includes an instruction-tuned model in BF16 and NVFP4 numerics, and a BF16 base model. **This repo contains the BF16 base model.**
+
+Starting from an initial Olmo 3 baseline, we iteratively improved all aspects of the training recipe including pretraining scaling laws, model architecture, data, and posttraining stack.
 Our final Ripple v1 model ladder improves Compute Efficiency (CE) on code by **16x** relative to the Olmo 3 ladder, with the advantage further increasing with scale up to 
 **55x** for the full hero run. 
 
@@ -22,25 +25,45 @@ Ripple-65B was codesigned to be trained and served efficiently on NVIDIA hardwar
 For our first model, a key objective was to make design decisions which reduce pretraining and posttraining
 infrastructure complexity while still enabling strong reasoning and agentic performance.
 Therefore, we chose to train a modernized dense Transformer model rather than use Mixture-of-Experts (MoE) like other labs.
-This decision quickly paid off, and we were able to train a strong model from scratch with a small team with only 5 people 
-while maintaining solid MFU throughout pretraining and posttraining.
+This decision quickly paid off, and we were able to train a strong model from scratch with a small team that scaled from 1 to 5 people during development, 
+while maintaining solid MFU (40%) throughout pretraining and posttraining with simple sharding, no custom kernels, and no quantization (BF16).
 
-Starting from the Olmo llama-like baseline, we switch to a more Gemma-like architecture to improve training stability at scale. 
 Ripple-65B uses a **thin-and-deep** model aspect ratio, with a narrow d_model (5120) and 80 total layers.
+We switch from the Olmo 3 llama-like architecture to a Gemma 3N-like architecture to improve training stability at scale. 
 To improve reasoning performance, we **overprovision the number of attention heads** while increasing the SWA-to-global layer ratio 
 and reducing SWA window size to compensate for the increased cost.
 In addition, we added FFN activation clipping, gated attention, global NoPE, and value residuals.
 
-The end result is a model that is especially capable at reasoning and agentic tasks given its model size, competing with other 
+The end result is a model that is especially capable at reasoning and agentic tasks given its model size, matching or exceeding other 
 US open-weight models with more than 10x the number of total parameters.
 At only 65B total parameters, the model can fit on consumer hardware such as the NVIDIA DGX Spark. 
 To encourage local model development, we provide an NVFP4 checkpoint and colab to help you get started. 
-The model is also fast at token generation when running on datacenter GPUs, easily reaching 250+ tok/s with DFlash2 on 4xB200 (batch=16) prior to quantization. 
+The model is also fast at token generation when running on datacenter GPUs, easily reaching 500+ tok/s on 4xB200 without quantization. 
 
-**Pre-training.** Ripple-65B Preview is pretrained from scratch on a 12T text-only corpus at 8K sequence length. 
+**Pre-training.** Ripple-65B Preview is pretrained from scratch on our 12T text-only predominantly-English and code corpus at 8K sequence length. 
 Ripple-65B Base Preview is a **pretrain-only checkpoint** without any midtraining or synthetic QA data.
 
-**Post-training.** We use a standard post-training recipe consisting of midtraining, long context, SFT, and mixed domain RL. 
+**Post-training.** We use a standard post-training recipe consisting of midtraining, long context, SFT, long context SFT, and mixed domain RL. 
+Our Preview release has gone through relatively light posttraining, with a focus on agentic coding.
+
+We show a brief overview of each stage below:
+
+<div align="center">
+
+| Stage | Tokens | Step | Seqlen | Batch | 
+| ----- | ------ | ------ | ---- | ----- |
+| Pretrain | 12T | 877,500 | 8K | 1408 -> 3072 |
+| Midtrain | 200B | 8,000 | 64K | 384 |
+| Long Context | 6.5B | 389 | 2M | 8 |
+| SFT | 183B | 7,274 | 128K | 192 |
+| LC SFT | 13.5B | 802 | 2M | 8 | 
+| RL  | N/A  | 700 | 128K | 2048 |
+
+</div>
+
+## Ripple-65B Base Preview
+
+This repo contains the unquantized base model checkpoint directly after pretraining. 
 
 ## Instruct Model Benchmarks
 
@@ -60,42 +83,42 @@ Therefore, our results represent a best-effort attempt at a fair comparison.
 
 <div align="center">
 
-  | Benchmark | Ripple-65B<br>Preview | Nemotron 3<br>Ultra | Inkling | Muse Spark<br>Open | Reflection<br>Beam |
+  | Benchmark | Ripple-65B<br>Preview | Nemotron 3<br>Ultra 550B | Inkling<br>975B | Muse Spark<br>Open 816B | Reflection<br>Beam 501B |
   |---|---|---|---|---|---|
   | **Model** | | | | | |
   | Architecture | Dense | MoE | MoE | MoE | MoE |
-  | Total params | 65B | 550B | 975B | 816B | 501B |
-  | Active params | 65B | 55B | 41B | 42B | 23B |
-  | Pre-training tokens | 12T | 20T | 45T | N/A | 23.8T |
-  | Training FLOPs (6ND) | 4.7e24 | 6.6e24 | 1.1e25 | N/A | 3.3e24 |
+  | Total params | 65B | 550B | **975B** | 816B | 501B |
+  | Active params | **65B** | 55B | 41B | 42B | 23B |
+  | Pre-training tokens | 12T | 20T | **45T** | N/A | 23.8T |
+  | Training FLOPs (6ND) | 4.7e24 | 6.6e24 | **1.1e25** | N/A | 3.3e24 |
   | **Knowledge** | | | | | |
-  | MMLU | — | *92.0* | *91.3* | *92.6* | *92.4* |
-  | MMLU-Pro | — | 86.8 | *87.3* | *87.95* | *76.5* |
-  | GPQA Diamond | — | 87.0 | 87.2[^h] | 89.0 | **90.6** |
+  | MMLU | *89.60* | *92.0* | *91.3* | **92.6** | *92.4* |
+  | MMLU-Pro | *84.15* | 86.8 | *87.3* | **87.95** | *76.5* |
+  | GPQA Diamond | *85.19* | 87.0 | 87.2[^h] | 89.0 | **90.6** |
   | **Math** | | | | | |
-  | AIME 2024 | — | *94.17* | *97.5* | *97.92* | *96.67* | 
-  | AIME 2025 | — | *94.17* | *91.67* | *100.0* | *99.58* |
-  | AIME 2026 | — | *95.42* | 97.1 | *98.33* | **97.7** |
-  | HMMT Feb 2026 | — | *80.3* | *78.79* | *93.94* | *93.94* | 
+  | AIME 2024 | *97.08* | *94.17* | *97.5* | **97.92** | *96.67* | 
+  | AIME 2025 | *98.33* | *94.17* | *91.67* | **100.0** | *99.58* |
+  | AIME 2026 | *95.00* | *95.42* | 97.1 | **98.33** | *97.7* |
+  | HMMT Feb 2026 | *90.53* | *80.3* | *78.79* | **93.94** | **93.94** | 
   | **Coding & Agentic** | | | | | |
-  | Terminal-Bench 2.1 (reported) | — | 56.4 | 63.8 | 76.4 | **80.1** |
-  | Terminal-Bench 2.1 (Terminus 2) | — | *51.31* | *55.43* | *79.78* | *67.79* |
-  | SWE-bench Verified | — | 70.7[^b] | 77.6[^f] | *78.67* | **81.6**[^n] |
+  | Terminal-Bench 2.1 (reported) | **80.52** | 56.4 | 63.8 | 76.4 | *80.1* |
+  | Terminal-Bench 2.1 (Terminus 2) | **80.52** | *51.31* | *55.43* | *79.78* | *67.79* |
+  | SWE-bench Verified | — | 71.9 | 77.6[^f] | *78.67* | **81.6**[^n] |
   | SWE-bench Pro v1.0 | — | *46.92* | 54.3[^g] | — | **66.1** |
   | SWE-bench Pro v2.0 Hard | — | *30.72* | *1.96* | *52.29* | 76.5 |
   | DeepSWE 1.1 | — | *0.22* | *1.96* | **60**[^j] | 43.3 |
-  | SciCode | — | 44.6[^c] | *39.18* | **58.5** | 49.2 | 
-  | τ³-bench banking | — | 22.6 | 23.7[^h] | *14.43* | **36.1** |
+  | SciCode | *39.40* | 44.6 | *39.18* | **58.5** | 49.2 | 
+  | τ³-bench banking | —| 22.6 | 23.7[^h] | *14.43* | **36.1** |
   | AutomationBench (Public) | — | *8.39* | *14.56* | *42.44* | 39.3 | 
   | MCP Atlas | — | *62.2* | 76.0[^i] | **89.2** | 80.8 |
   | **Safety** | | | | | |
-  | FORTRESS Adversarial | — | *65.80* | 78.0 | *89.07* | *81.33* |
-  | FORTRESS Benign | — | *89.13* | 95.9 | *88.13* | *86.40* |
-  | StrongREJECT (direct prompts) | — | *100.0* | 98.6 | *96.59* | *100.0* |
+  | FORTRESS Adversarial | *83.83* | *65.80* | 78.0 | *89.07* | *81.33* |
+  | FORTRESS Benign | *95.47* | *89.13* | 95.9 | *88.13* | *86.40* |
+  | StrongREJECT | *97.47* | *100.0* | 98.6 | *96.59* | *100.0* |
   | **Instruction Following**  | | | | | |
-  | IFBench | — | **81.7**[^d] | 79.8 | *78.56* | 79.6 |
+  | IFBench | *79.22* | **81.7** | 79.8 | *78.56* | 79.6 |
   | **Long Context**  | | | | | |
-  | AA-LCR | — | 65.4 | *78.33* | *80.67* | **78.2** |
+  | AA-LCR | *75.00* | 65.4 | *78.33* | *80.67* | **78.2** |
 
 </div>
 
@@ -119,14 +142,14 @@ For our own model and for missing evals, we use our internal fork of [Olmes](htt
 
 <div align="center">
 
-  | Benchmark (Metric) | # Shots | Ripple-65B<br>Preview | Laguna<br>M.1 | DeepSeek<br>V3.1 | DeepSeek<br>V4 Flash | Nemotron 3<br>Ultra |
+  | Benchmark (Metric) | # Shots | Ripple-65B<br>Preview | Laguna<br>M.1 225B | DeepSeek<br>V3.1 671B | DeepSeek<br>V4 Flash 284B | Nemotron 3<br>Ultra 550B |
   |---|---|---|---|---|---|---|
   | **Model** | | | | | | |
   | Architecture | — | Dense | MoE | MoE | MoE | MoE |
-  | Total params | — | 65B | 225B | 671B | 284B | 550B |
-  | Active params | — | 65B | 23B | 37B | 13B | 55B |
-  | Training tokens (T) | — | 12T | 30T+ | 15.6T | 32T+ | 20T |
-  | Training FLOPs (6ND) | — | 4.7e24 | 4.2e24 | 3.5e24 | 2.5e24 | 6.6e24 |
+  | Total params | — | 65B | 225B | **671B** | 284B | 550B |
+  | Active params | — | **65B** | 23B | 37B | 13B | 55B |
+  | Training tokens (T) | — | 12T | 30T+ | 15.6T | **32T+** | 20T |
+  | Training FLOPs (6ND) | — | 4.7e24 | 4.2e24 | 3.5e24 | 2.5e24 | **6.6e24** |
   | **World Knowledge** | | | | | | |
   | MMLU (MC acc) | 5-shot | *83.7* | *76.6* | *88.3* | 88.7 | **89.1** |
   | MMLU-Pro (CoT EM) | 5-shot | *61.5* | *52.2* | *57.8* | 68.3 | **79.1** |