v2: SFT + DPO + GRPO (R7); two-judge eval; model card updated

README.md

@@ -20,26 +20,32 @@ One repo, every variant: transformers bf16 at the root; llama.cpp GGUF (Q8_0 / Q
 
 Rewrites AI-written drafts so they read like a person wrote them, keeping every fact, number, name and date. Fine-tune of `google/gemma-4-E4B`.
 
-**Trained on data that teaches the model how people write. No reinforcement learning or optimisation against any AI detector; it passes detectors naturally, and detector numbers below are only an external check.**
+**v2 (2026-09-14): SFT + DPO + GRPO. Trained on data that teaches the model how people write, then reinforced on fidelity only. No detector was used as a reward, filter, signal or selection criterion at any stage; it passes detectors naturally, and detector numbers below are only an external check.**
 
 ## Training
 
 * **SFT, 28.6k pairs** of *(AI draft → real human text)*. The human side is always genuine human writing (scientific abstracts, government reports, student essays, email, forum posts, Chinese prose); the AI side is a draft of the same content written by a current frontier model from the human text.
 * **DPO, 2 rounds, 1,898 pairs.** Candidates written by the model itself; a separate LLM judge grades *fidelity only* (facts kept, meaning unchanged, nothing added, greeting/sign-off kept, not a verbatim copy). Chosen = fidelity-clean with the least verbatim overlap; rejected = critical fidelity error or near-copy. No style, length or detector signal enters selection.
+* **GRPO, 300 steps (v2).** LoRA on the merged SFT + DPO model, 8 samples per draft. Reward = LLM-judged fidelity against an atomic fact list of the draft (critical −3.0, minor −0.15, invented −2.0, reversed −2.0, dropped format element / greeting −1.5), −1.0 if the draft's register is normalised, and a superlinear reuse ramp (reuse = max(verbatim 5-gram copy, content-masked syntactic-skeleton recall); free below 0.31, −4 at full copy). Errors are credited to the sentence that carries them. Release checkpoint picked from a 25-step ladder on the held-out set. No detector anywhere in the reward.
 * **Inference guard:** if a sample copies > 35 % of the draft's 5-grams it is resampled once with a logits penalty on draft n-grams (digits exempt).
 
 ## Evaluation (39-case daily-use set, 2 samples each)
 
-| 62 English samples | this model | baseline 4B rewriter |
-|---|---|---|
-| samples copying > 35 % of draft | 0 | 33 / 93 |
-| critical fidelity errors (judge) | 10 % | ≈ 30 % |
-| Chinese cases passing judge | 10 / 16 | 4 / 16 |
-| Originality.ai "human" *(external check only)* | 79 % | 57 % |
+Two independent judges (GLM-5.3 and gpt-5.6); an error counts only if both report it. v1 re-graded under the same protocol.
+
+| 62 English samples | **v2** | v1 (SFT + DPO) | baseline 4B rewriter |
+|---|---|---|---|
+| samples copying > 35 % of draft | 0 | 1 | 33 / 93 |
+| reuse (verbatim ∨ syntactic skeleton), median | 0.29 | 0.34 | — |
+| critical fidelity errors | **0 / 62** | 3 / 62 | ≈ 30 % |
+| minor / clean | 20 / 42 | 15 / 44 | — |
+| format element dropped | 5 / 62 | 12 / 62 | — |
+| Chinese cases passing judge | 13 / 16 | 11 / 16 | 4 / 16 |
+| Originality.ai "human" *(external check only)* | **85 %** | 81 % | 57 % |
 
 GGUF files (Q8_0 / Q6_K / bf16; Q5_K_M withheld at 17/62 critical errors, Q4_K_M withheld as gibberish) are in the `gguf/` folder of this repo; their quality on the same set is in `docs/QUALITY.md` of the GitHub repo. MLX 4-/6-bit quantisation of this model is not usable (Gemma 4 PLE layers); use the GGUF quants on a Mac.
 
-Known failure modes: ~1 in 10 outputs has a meaning flip (who did what, ordered vs received, a metric renamed); subject lines/greetings occasionally dropped; Chinese weaker than English; short drafts (< 120 words) less reliable. **Proofread numbers, dates and the direction of every claim.**
+Known failure modes (v2): dropped qualifiers in about 1 in 3 outputs ("an estimated 4.2 %" → "4.2 %", "suggest" → "conclude"); meaning flips were v1's main failure (~1 in 10) and did not appear in v2's 62 samples, but the set is small; an occasional garbled sentence — resample; Chinese weaker than English (13/16); short drafts (< 120 words) less reliable. **Proofread numbers, dates and the direction of every claim.** GGUF fidelity numbers in `docs/QUALITY.md` were measured on v1; v2 quants were built with the same recipe.
 
 ## Prompt format (must match exactly)
 
@@ -74,7 +80,7 @@ out = model.generate(**ids, max_new_tokens=900, do_sample=True, temperature=0.85
 print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
 ```
 
-The full inference path with the anti-copy resample, an MLX server, the evaluation set and the training scripts are on GitHub: `sgaofen/humanizer`.
+Version history: v2 (2026-09-14, this revision) adds the GRPO stage; v1 (2026-09-10, SFT + DPO) is in the commit history of this repo. The full inference path with the anti-copy resample, an MLX server, the evaluation set and the training scripts are on GitHub: `sgaofen/humanizer`.
 
 ## Licence
 

config.json

@@ -62,7 +62,6 @@
     "eos_token_id": 1,
     "expert_intermediate_size": null,
     "final_logit_softcapping": 30.0,
-    "global_head_dim": 512,
     "head_dim": 256,
     "hidden_activation": "gelu_pytorch_tanh",
     "hidden_size": 2560,
@@ -115,13 +114,36 @@
     ],
     "max_position_embeddings": 131072,
     "model_type": "gemma4_text",
+    "moe_intermediate_size": null,
     "num_attention_heads": 8,
     "num_experts": null,
-    "num_global_key_value_heads": null,
     "num_hidden_layers": 42,
     "num_key_value_heads": 2,
     "num_kv_shared_layers": 18,
     "pad_token_id": 0,
+    "per_layer_config": {
+      "05": {
+        "head_dim": 512
+      },
+      "11": {
+        "head_dim": 512
+      },
+      "17": {
+        "head_dim": 512
+      },
+      "23": {
+        "head_dim": 512
+      },
+      "29": {
+        "head_dim": 512
+      },
+      "35": {
+        "head_dim": 512
+      },
+      "41": {
+        "head_dim": 512
+      }
+    },
     "rms_norm_eps": 1e-06,
     "rope_parameters": {
       "full_attention": {
@@ -144,7 +166,7 @@
     "vocab_size_per_layer_input": 262144
   },
   "tie_word_embeddings": true,
-  "transformers_version": "5.5.0.dev0",
+  "transformers_version": "5.15.0",
   "video_token_id": 258884,
   "vision_config": {
     "_name_or_path": "",

generation_config.json

@@ -6,5 +6,5 @@
   "temperature": 1.0,
   "top_k": 64,
   "top_p": 0.95,
-  "transformers_version": "5.5.0.dev0"
+  "transformers_version": "5.15.0"
 }

model.safetensors → model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:519f1a0f586ec262ac70c383d8e2671ebfa57541b2fe6ba14d3dfaf260188a8c
-size 15992596739
+oid sha256:4736d5ae6bf7832256619d48c8768e1fc269d8a291532ad85067c61ca0e6b7f5
+size 5637144744

model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72c4fe414e96b2aa6a5c1987ef76efd8187c71e234a39328eb0cc16553c8c1dc
+size 4994573962

model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e906359b61e36e187b59d7ef8abba0479ca225c72b8776892f1201b08b716901
+size 4996204630

model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8080cb1ab73a91528449fc238665f6a2938583e5f1dc877c099292a1f31f6144
+size 254550436

tokenizer_config.json

@@ -17,8 +17,30 @@
     "<|video|>"
   ],
   "image_token": "<|image|>",
+  "is_local": false,
+  "local_files_only": false,
   "mask_token": "<mask>",
   "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
   "pad_token": "<pad>",
   "padding_side": "left",
   "processor_class": "Gemma4Processor",