docs: note 256k-in continued training; keep default weights (math hold)

README.md

@@ -34,13 +34,14 @@ base_model:
 
 | Done in this preview | Still cooking |
 | --- | --- |
-| Long-context continued pretraining | More preference / alignment optimization |
+| Long-context continued pretraining (up to 64k packs) | More preference / alignment optimization |
 | Supervised chat, thinking, and tools | Broader RL beyond math/code verifiers |
 | RL on verifiable math (and code-oriented) rewards | Reliable **64k** free-form generation (in progress) |
 | Long-output continuation (16k→32k response budgets) | Multimodal later |
-| 256k RoPE capacity retained | Broader public eval suite |
+| **256k input** continued training (experimental; see below) | Merge long-ctx without math regression |
+| 256k RoPE capacity retained on default weights | Multikey / hard long-ctx retrieval |
 
-**Practical limits today:** input capacity is configured to **256k**. Generation defaults target long answers (up to 64k in `generation_config`), but **stable 64k-out training is still underway** — expect shorter generations to be more reliable in this preview.
+**Practical limits today:** default **weights** keep strong chat/math quality with RoPE capacity **256k**. A separate **256k-input continued-training** checkpoint exists (needle retrieval strong) but is **not** the default download yet — math quality dropped after that pass. Generation defaults target long answers (up to 64k in `generation_config`), but **stable 64k-out training is still underway**.
 
 ## How it was trained (high level)
 
@@ -138,9 +139,23 @@ print(resp.choices[0].message.content)
 
 ## Context & generation
 
-- **Input capacity:** up to **256k** tokens (RoPE `max_position_embeddings=262144`)
+- **Input capacity (config):** up to **256k** tokens (RoPE `max_position_embeddings=262144`)
 - **Generation target:** moving toward **64k** free-form outputs; this preview is strongest at moderate-to-long answers (multi-k tokens). Treat 64k-out as aspirational until a later drop.
 
+## Long-context continued training (256k)
+
+We completed an experimental **256k-input** continued-training curriculum (128k → 256k packs) after the chat/RL path. Results:
+
+| Capability | Status |
+| --- | --- |
+| Single-needle retrieval @64k / 128k / 256k | Strong (pass rate **1.0** in internal needle smokes) |
+| Multi-key needle @128k | **Weak** (~0.4) — follow-up recommended |
+| Chat behaviour / tools format smoke | Holds (10/10 internal cases) |
+| Verifiable math (internal private probe) | **Regressed** vs the default chat checkpoint — therefore **not** promoted to default weights |
+| 64k free-form **output** training | Still **in progress** (orthogonal track) |
+
+**Default weights on this repo** remain the stronger chat + math snapshot (FP8). The 256k-in experimental checkpoint lives in our training volume as `teutonic-ii-lc256k` (alias `teutonic-ii-longctx-256k`); a sibling 128k checkpoint is `teutonic-ii-lc128k`. We will publish a dedicated long-context weight drop only after math recovers (co-training / replay mix).
+
 ## Evaluation honesty
 
 Internal math/reasoning probes improved substantially after verifiable-reward RL relative to the supervised merge. We **do not** claim contaminated public-bench SOTA (e.g. treat GSM8K-style numbers with caution). Prefer private or carefully decontaminated suites when comparing. Behavioural chat smokes (format, tools, basic instruction following) pass on this snapshot; broader alignment and preference quality are still in progress.