docs: connect DPO model, data, and training code

README.md

@@ -2,6 +2,8 @@
 library_name: transformers
 pipeline_tag: text-generation
 base_model: muradil211/AetherSearch_SFT
+datasets:
+  - muradil211/AetherSearch_DPO
 tags:
   - aethersearch
   - agentic-search
@@ -22,21 +24,20 @@ language:
 
 ### A preference-aligned search agent for retrieval-augmented reasoning
 
-Continued from **AetherSearch SFT** with direct preference optimization on
-the **AetherSearch_DPO** dataset.
+Trained from **AetherSearch SFT** with direct preference optimization on all
+**2,126 preference pairs** in the canonical **AetherSearch DPO** dataset.
 
 <p>
   <a href="https://huggingface.co/muradil211/AetherSearch_SFT"><img src="https://img.shields.io/badge/Base-AetherSearch__SFT-7C3AED?style=flat-square" alt="Base model: AetherSearch SFT"></a>
+  <a href="https://huggingface.co/datasets/muradil211/AetherSearch_DPO"><img src="https://img.shields.io/badge/Pairs-2%2C126-F59E0B?style=flat-square" alt="Training pairs: 2,126"></a>
   <img src="https://img.shields.io/badge/Method-DPO-0F766E?style=flat-square" alt="Training method: DPO">
-  <img src="https://img.shields.io/badge/Dataset-AetherSearch__DPO-F59E0B?style=flat-square" alt="Dataset: AetherSearch DPO">
   <img src="https://img.shields.io/badge/Context-32K-2563EB?style=flat-square" alt="Context window: 32K">
 </p>
 
-[🏠 Project](https://github.com/Muradil-mamat-211/AetherSearch) · [🧠 SFT checkpoint](https://huggingface.co/muradil211/AetherSearch_SFT)
-
-📚 Dataset: **AetherSearch_DPO**
-
-<!-- Dataset URL: -->
+[🏠 Project](https://github.com/Muradil-mamat-211/AetherSearch) ·
+[🧪 Training code](https://github.com/Muradil-mamat-211/AetherSearch/tree/main/dpo) ·
+[📚 Preference data](https://huggingface.co/datasets/muradil211/AetherSearch_DPO) ·
+[🧠 SFT checkpoint](https://huggingface.co/muradil211/AetherSearch_SFT)
 
 </div>
 
@@ -47,30 +48,37 @@ the **AetherSearch_DPO** dataset.
 
 ## ✨ Highlights
 
-- 🎯 **Preference aligned**: trained with DPO to favor stronger search and answer trajectories.
-- 🔎 **Search native**: produces explicit reasoning, retrieval requests, and grounded answers.
-- 🔁 **Multi-step retrieval**: can request additional evidence when one search is insufficient.
-- ⚡ **Compact backbone**: retains the accessible 3B-scale Qwen2 architecture.
+- 🎯 **Preference aligned** — trained to favor stronger search decisions,
+  evidence use, and final answers over paired alternatives.
+- 🔎 **Search native** — produces explicit reasoning, retrieval requests, and
+  grounded answers using the AetherSearch XML protocol.
+- 🔁 **Multi-step retrieval** — can request additional evidence when an
+  existing trajectory is insufficient.
+- 🧾 **Environment-aware loss** — retrieved information remains available as
+  context while being excluded from preference log-probability sums.
+- 🧪 **Reproducible stage** — canonical data identity, strict trainer,
+  hardware-independent launcher, DeepSpeed configuration, and source checks
+  are public.
 
 ## 🧠 How it works
 
 ```text
 Question
-   |
-   v
+   │
+   ▼
 <think>reason about what is missing</think>
-   |
-   v
-<search>focused retrieval query</search> -----> Search / RAG backend
-   ^                                                  |
-   +---- <information>retrieved evidence</information>+
-   |
-   +-- repeat the search loop when more evidence is needed
-   v
+   │
+   ▼
+<search>focused retrieval query</search> ─────► Search / RAG backend
+   ▲                                                  │
+   └──── <information>retrieved evidence</information> ◄────┘
+   │
+   ├── repeat the search loop when more evidence is needed
+   ▼
 <answer>evidence-grounded final answer</answer>
 ```
 
-The model produces the reasoning, search, and answer spans. The surrounding
+The model produces reasoning, search, and answer spans. The surrounding
 runtime parses each completed `<search>` span, runs retrieval, appends the
 result as `<information>`, and resumes generation until the model emits an
 `<answer>` span.
@@ -80,16 +88,69 @@ result as `<information>`, and resumes generation until the model emits an
 | Field | Value |
 |---|---|
 | 🧬 Base checkpoint | [`muradil211/AetherSearch_SFT`](https://huggingface.co/muradil211/AetherSearch_SFT) |
+| 🧬 Base revision | `437aca474d3966e57e82af565db95d0ad64aa24d` |
+| 📚 Training data | [`muradil211/AetherSearch_DPO`](https://huggingface.co/datasets/muradil211/AetherSearch_DPO) |
+| 🔗 Training code | [`AetherSearch/dpo`](https://github.com/Muradil-mamat-211/AetherSearch/tree/main/dpo) |
 | 🏗️ Architecture | Qwen2 causal language model |
 | 🔢 Parameters | 3,085,938,688 |
 | 🎛️ Weight dtype | BF16 |
-| 📏 Context window | 32,768 positions |
-| 🎯 Alignment method | Direct Preference Optimization (DPO) |
-| 📚 Training data | AetherSearch_DPO |
+| 📏 Context window | 32,768 positions; training sequences capped at 4,096 |
+| 🎯 Alignment method | Direct Preference Optimization |
 
-## 🚀 Quick start
+## 🧬 Checkpoint identity
+
+This checkpoint was trained in one DPO stage from the pinned AetherSearch SFT
+checkpoint over all 2,126 records in the canonical `train.jsonl`, using the
+public code under the GitHub repository's `dpo/` directory. Training was
+performed on a separate server; this repository contains the final model
+artifacts rather than server-local run logs or optimizer state.
+
+**Dataset SHA-256**
+
+```text
+c42adcb0f194cff3126134b37afd85e4b89aa9917e5c98dda4b09904509f61e9
+```
+
+## 🎯 Preference contract
+
+Every training row supplies one shared `prompt_text`, one `chosen`
+continuation, and one `rejected` continuation.
+
+- Shared prompt tokens are masked on both sides.
+- Environment-provided `<information>...</information>` spans inside either
+  continuation are masked while remaining visible as context.
+- All remaining continuation tokens contribute to summed sequence log
+  probabilities.
+- Answer-terminal continuations supervise a final `<|im_end|>` token.
+- Search-terminal continuations do not append that token because the runtime
+  must provide retrieved information before generation resumes.
+- The frozen reference and initial policy use the same pinned SFT checkpoint.
+
+The strict public trainer rejects checksum drift, row-count drift, duplicate
+questions, malformed trajectories, identical pairs, all-masked continuations,
+and sequence truncation before allocating model weights.
+
+## 🛠️ Training recipe
+
+| Setting | Value | Setting | Value |
+|---|---:|---|---:|
+| Epochs | 1 | Learning rate | `5e-7` |
+| DPO beta | `0.1` | Scheduler | Cosine |
+| Global batch size | 12 pairs | Per-device batch | 1 pair |
+| Precision | BF16 | Max sequence length | 4,096 |
+| Warmup ratio | `0.03` | Weight decay | `0.0` |
+| Distributed optimizer | DeepSpeed ZeRO-3 | Seed | 42 |
+
+The launcher discovers devices already made visible by the surrounding
+runtime and derives gradient accumulation to preserve global batch 12. It does
+not embed physical GPU IDs, node addresses, communication-fabric settings,
+allocator tuning, or server-local paths. The policy and frozen reference are
+both sharded under ZeRO-3.
+
+The complete implementation and reproduction commands are in the
+[AetherSearch DPO directory](https://github.com/Muradil-mamat-211/AetherSearch/tree/main/dpo).
 
-Load the checkpoint with:
+## 🚀 Quick start
 
 ```python
 import torch
@@ -109,18 +170,26 @@ model.config.use_cache = True
 For end-to-end use, wrap generation in the retrieval loop shown above and
 preserve the XML protocol exactly.
 
+## 📦 Repository contents
+
+The repository contains two BF16 SafeTensors shards, the shard index, model
+and generation configuration, tokenizer assets, this model card, and the
+AetherSearch logo. Intermediate checkpoints, optimizer state, report bundles,
+and server-local logs are not included.
+
 ## ⚠️ Limitations
 
 Generated searches and answers can be incorrect, unsupported, or unsafe.
-Retrieval quality, evidence validation, and answer verification remain the
-caller's responsibility.
+Retrieval quality, evidence validation, answer verification, and deployment
+safeguards remain the caller's responsibility.
 
 ## 📜 Terms
 
 No additional blanket license is asserted here. Review the
 [Qwen2.5-3B-Instruct license](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE),
-the AetherSearch SFT terms, and the AetherSearch_DPO dataset terms before
-redistribution or downstream use.
+the [AetherSearch SFT terms](https://huggingface.co/muradil211/AetherSearch_SFT),
+and the [AetherSearch DPO data attribution](https://huggingface.co/datasets/muradil211/AetherSearch_DPO/blob/main/ATTRIBUTION.md)
+before redistribution or downstream use.
 
 ---