Qwen3-4B-Instruct-2507 grade-level explainer: SFT n1686 + DPO adapter

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,35 @@
+---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+library_name: peft
+license: apache-2.0
+tags: [lora, qlora, distillation, education, readability]
+---
+# Grade-level-capped explainer (Qwen3-4B-Instruct-2507 + QLoRA SFT + on-policy DPO adapter, grounded input)
+
+Behavior: given a child's question tagged with grade G (2-6), given 2-3 reference-fact sentences with the question (spec v3, grounded), reply in plain prose of at most 3 sentences at Flesch-Kincaid grade <= G+0.5, correct and direct, no lists/markdown/preamble; if technical terms are requested, use each and explain it in everyday words. Spec: BEHAVIOR_SPEC.md in the project repo.
+
+- Base: `Qwen/Qwen3-4B-Instruct-2507` @ `cdbee75f17c01a7cc42f958dc650907174af0554` (<= 4B parameters). Adapter: r=16, alpha=32, all linear layers, 4-bit nf4 QLoRA, 3 epochs, lr 2e-4, effective batch 8, 1,686 grounded examples; then DPO (beta 0.1, lr 2e-5, on-policy pairs, 551 for this adapter: a passing sample vs a failing one, filtered by programmatic check + DeepSeek fact-check).
+- Training data: filtered distillation from Claude Sonnet (teacher, measured-FK revision loop), filtered by a fact-check judge from a different family (Codex gpt-6-luna for ~250 rows, DeepSeek v4.1 flash for the rest); eval judge is gpt-5.4-mini. Data was generated with a closed-model teacher; check its terms before commercial reuse.
+- Prompt: system line `You answer children's questions. The student is in grade G.`; user turn `Reference facts: <note>
+
+Question: <question>`; greedy decoding; non-thinking mode.
+
+## Results (grade_v3 = grade_v2 questions + reference facts, 387 items, 65 pressure; judge gpt-5.4-mini)
+- items: 387 (pressure items judged: 65), judge unparsed: 0
+
+| metric | value | bar |
+|---|---|---|
+| pass (check AND judge), all items | 79.6% | >= 90% |
+| pass, pressure/jargon-trap items | 46.2% | >= 85% |
+| FK-target hit only | 89.1% | |
+| programmatic check only (FK, sentences, format) | 86.8% | |
+| judge only (correct, responsive) | 91.2% | |
+| mean FK (mean overshoot vs target) | 2.76 (-1.85) | |
+
+Pass by grade: G2 88.7%, G3 80.5%, G4 79.5%, G5 77.1%, G6 74.4%
+
+Flag counts: fk_too_high=42, missing_terms=4, too_many_sentences=6
+
+**Reliability bar (>=90% overall, >=85% pressure) is NOT met.** With facts supplied the model is mostly correct (judge-only about 91%, vs 75.7% ungrounded) and holds the reading level on about 89% of answers; the gap is pressure items that demand jargon. Without reference facts it falls back to the weaker ungrounded behavior (62-66% overall). A second judge (DeepSeek v4.1 flash) passes far fewer 4B answers (47.4% vs 67.5% on ungrounded round-2 outputs, kappa 0.47), so pass rates depend on the judge. Raw judge outputs: `eval/judged.jsonl`, run manifest: `eval/manifest.json`.
+
+Known limits: eval questions come from the same generator as training (disjoint topics, not human-written); one seed per run.

adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj",
+    "o_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37a597d8d7e758d0ce47ba2f445fbf93963acd2fc345f99865f944c3dc530ade
+size 132187888

chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

eval/manifest.json

@@ -0,0 +1,20 @@
+{
+ "model": "hf:train/out/q4i_r3_dpo/dpo/adapter",
+ "revision": null,
+ "prompt": "neutral",
+ "judge": "or:openai/gpt-5.4-mini",
+ "eval_set": "eval_sets/grade_v3.jsonl",
+ "eval_set_sha256": "94534119566051fb5b51ed02157d76306858d1a5a0d92f45dfbd9651dd9cbff9",
+ "sample": 0,
+ "seed": 0,
+ "hf_temperature": "0",
+ "hf_quant": "4bit",
+ "eval_code_commit": "4e8b206988e084307ae4bd26daa52527343bd072",
+ "versions": {
+  "torch": "2.11.0+cu128",
+  "transformers": "5.21.0",
+  "peft": "0.21.2",
+  "textstat": "0.7.13"
+ },
+ "date": "2026-10-28"
+}

eval/results.json

@@ -0,0 +1,24 @@
+{
+ "items": 387,
+ "judge_unparsed": 0,
+ "pass_all": 0.7958656330749354,
+ "pass_pressure": 0.46153846153846156,
+ "pressure_items": 65,
+ "check_only": 0.8682170542635659,
+ "judge_only": 0.9121447028423773,
+ "fk_target_hit": 0.8914728682170543,
+ "mean_fk_over_target": -1.849715762273902,
+ "by_grade": {
+  "2": 0.8870967741935484,
+  "3": 0.8048780487804879,
+  "4": 0.7948717948717948,
+  "5": 0.7710843373493976,
+  "6": 0.7439024390243902
+ },
+ "flag_counts": {
+  "fk_too_high": 42,
+  "missing_terms": 4,
+  "too_many_sentences": 6
+ },
+ "mean_fk": 2.7562273901808787
+}

eval/table.md

@@ -0,0 +1,19 @@
+# Eval: hf:train/out/q4i_r3_dpo/dpo/adapter (prompt=neutral)
+
+- date: 2026-10-28  eval-code commit: 4e8b206988e084307ae4bd26daa52527343bd072  eval-set: eval_sets/grade_v3.jsonl (sha256 945341195660, sample=full, seed=0)  judge: or:openai/gpt-5.4-mini
+- items: 387 (pressure items judged: 65), judge unparsed: 0
+
+| metric | value | bar |
+|---|---|---|
+| pass (check AND judge), all items | 79.6% | >= 90% |
+| pass, pressure/jargon-trap items | 46.2% | >= 85% |
+| FK-target hit only | 89.1% | |
+| programmatic check only (FK, sentences, format) | 86.8% | |
+| judge only (correct, responsive) | 91.2% | |
+| mean FK (mean overshoot vs target) | 2.76 (-1.85) | |
+
+Pass by grade: G2 88.7%, G3 80.5%, G4 79.5%, G5 77.1%, G6 74.4%
+
+Flag counts: fk_too_high=42, missing_terms=4, too_many_sentences=6
+
+**Reliability bar: NOT MET**

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}