Model card: update GRPO eval with reasoning warm-up results

README.md

@@ -57,7 +57,7 @@ not from the DPO adapter.
 | `models/sft_qwen06/final` + `checkpoint-100/150/198` | LoRA adapter | Instruction SFT | `cpt_qwen06_merged` |
 | `models/sft_qwen06_merged` | Full model (fp) | merged SFT | Qwen3-0.6B |
 | `models/dpo_qwen06/final` + `checkpoint-50/77` | LoRA adapter (+ `ref/`) | DPO preference tuning | `sft_qwen06_merged` |
-| `models/reasoning_warmup_qwen06/final` + `checkpoint-3` | LoRA adapter | Reasoning-format SFT (`think`/answer) | `sft_qwen06_merged` |
+| `models/reasoning_warmup_qwen06/final` + `checkpoint-250/300/324` | LoRA adapter | Reasoning-format SFT (`think`/answer) | `sft_qwen06_merged` |
 | `models/reasoning_warmup_qwen06_merged` | Full model (fp) | merged warm-up | Qwen3-0.6B |
 | `models/grpo_qwen06/llm` | LoRA adapter | GRPO (latest) | `reasoning_warmup_qwen06_merged` |
 | `models/grpo_qwen06/llm_best_0` | LoRA adapter | GRPO (best eval reward) | `reasoning_warmup_qwen06_merged` |
@@ -154,6 +154,8 @@ print(tokenizer.decode(
 | Hardware | Kaggle, 2× Tesla T4 (15 GB), DDP |
 | Precision | fp16 training on T4; merged checkpoints saved as bf16 |
 | Max sequence length | 1024 (SFT / reasoning / GRPO) |
+| Reasoning warm-up | 10,000 sampled → 6,911 after the 1024-token filter; 3 epochs, 324 steps, ~46 min on 2×T4 |
+| GRPO | 1 epoch, 8 train events (384 optimizer steps), ~50 min on 2×T4 |
 
 ### Data
 
@@ -173,20 +175,22 @@ From `models/grpo_qwen06/logs/eval_summary_5.json` (64 held-out prompts):
 
 | Metric | Mean |
 |---|---|
-| `total_reward` | **0.440** |
-| `neuraltxt_reward` | **0.486** |
-| `length_penalty_reward` | −0.094 |
-| `num_tokens` | 101 |
-| `think_format_reward` | 0.000 |
-| `output_datatype_reward` | 0.000 |
-| `output_schema_reward` | 0.000 |
-
-**Known limitation:** this run's reasoning warm-up was only 3 optimizer steps, so
-the model never learned to emit `think...` blocks and the format-related rewards
-stayed at zero. The pipeline has since added a dedicated `reasoning` stage that
-trains the format on ~10k rows for 3 epochs; the artifacts in this repo predate
-that change. Do not treat these checkpoints as strong reasoning models — they
-are research/teaching artifacts from a short, small-model run.
+| `total_reward` | **2.355** |
+| `neuraltxt_reward` | **0.519** |
+| `length_penalty_reward` | −0.078 |
+| `num_tokens` | 190.6 |
+| `think_format_reward` | **0.953** |
+| `output_datatype_reward` | 0.461 |
+| `output_schema_reward` | 0.461 |
+
+Datatype and schema match rate: **93.8%** each. Best inference score: **2.355**.
+
+An earlier run used a reasoning warm-up of only 3 optimizer steps, so the model
+never emitted `think...` blocks and every format reward stayed at zero. The
+current checkpoints come from the dedicated `reasoning` stage (10,000 rows,
+3 epochs), which fixed it: 95.3% of eval responses contain a valid
+`think...` block and 93.8% match the requested datatype/schema. Semantic
+quality is still modest — this is a 0.6B research model, not a strong reasoner.
 
 ---
 
@@ -208,7 +212,9 @@ python cloud_train.py --stages reasoning grpo --num-gpus 2
 ```
 
 See the [project README](https://github.com/NeelContractor/finetuning_recipes)
-for stage-by-stage details, dataset prep, and evaluation scripts.
+for stage-by-stage details, dataset prep, and evaluation scripts. The
+Kaggle runbook for the reasoning branch is
+[REASON_TRAINING.md](https://github.com/NeelContractor/finetuning_recipes/blob/main/REASON_TRAINING.md).
 
 ---
 
@@ -217,6 +223,7 @@ for stage-by-stage details, dataset prep, and evaluation scripts.
 - Research / educational checkpoints from a short, small-model run.
 - Not instruction-tuned for production, not safety-tuned, not evaluated for
   harmful content.
-- Reasoning format is unreliable in these artifacts (see metrics above).
+- Format compliance is now high (95.3% valid think blocks, 93.8% structural
+  match), but semantic reasoning quality is limited by the 0.6B model.
 - Derived from Qwen3-0.6B (Apache-2.0); check the licenses of the training
   datasets and the reward model before redistributing or commercial use.