Update README.md

README.md

@@ -23,419 +23,482 @@ tags:
   - parameter-efficient-fine-tuning
 ---
 
-# Qwen2.5-Coder-7B-Instruct — DPO Fine-Tuned for Python
+# Qwen2.5-Coder-7B-Instruct — DPO Fine-Tuned for Python**
 
-## Model at a Glance
+**## Model at a Glance**
 
-This model is a **Python-focused, DPO fine-tuned version of Qwen2.5-Coder-7B-Instruct**, designed to improve the model's ability to generate **functionally correct Python code**.
+This model is a ****Python-focused, DPO fine-tuned version of Qwen2.5-Coder-7B-Instruct****, designed to improve the model's ability to generate ****functionally correct Python code****.
 
-The model was trained using **Direct Preference Optimization (DPO)** with automatically constructed preference pairs. Candidate Python solutions were generated for programming problems and evaluated by executing them against unit tests. Solutions that passed the required tests were used as preferred examples, while solutions that failed tests were used as rejected examples.
+The model was trained using ****Direct Preference Optimization (DPO)**** with automatically constructed preference pairs. Candidate Python solutions were generated for programming problems and evaluated by executing them against unit tests. Solutions that passed the required tests were used as preferred examples, while solutions that failed tests were used as rejected examples.
 
 The resulting model was evaluated against the original Qwen2.5-Coder-7B-Instruct model on held-out programming problems.
 
-### What changed?
+**### What changed?**
 
 The base model:
 
-**Qwen2.5-Coder-7B-Instruct**
+****Qwen2.5-Coder-7B-Instruct****
 
 was fine-tuned using:
 
-**DPO + LoRA + execution-based preference data**
+****DPO + LoRA + execution-based preference data****
 
 The resulting model demonstrates improved coding performance on the evaluated held-out benchmark:
 
-| Metric                  | Qwen2.5-Coder-7B-Instruct |  DPO Model |  Improvement |
-| ----------------------- | ------------------------: | ---------: | -----------: |
-| Pass@1                  |                    41.87% | **45.22%** | **+3.35 pp** |
-| Pass@5                  |                    48.87% | **51.10%** | **+2.23 pp** |
-| Pass@10                 |                    50.99% | **53.16%** | **+2.17 pp** |
-| Tests passed per answer |                    66.20% | **69.41%** | **+3.21 pp** |
+\| Metric                  | Qwen2.5-Coder-7B-Instruct |  DPO Model |  Improvement |
 
-The primary result is the improvement in **Pass@1**, meaning that a single generated solution was more likely to produce a correct solution on the evaluated programming problems.
+\| ----------------------- | ------------------------: | ---------: | -----------: |
+
+\| Pass@1                  |                    41.87% | ****45.22%**** | ****+3.35 pp**** |
+
+\| Pass@5                  |                    48.87% | ****51.10%**** | ****+2.23 pp**** |
+
+\| Pass@10                 |                    50.99% | ****53.16%**** | ****+2.17 pp**** |
+
+\| Tests passed per answer |                    66.20% | ****69.41%**** | ****+3.21 pp**** |
+
+The primary result is the improvement in ****Pass@1****, meaning that a single generated solution was more likely to produce a correct solution on the evaluated programming problems.
 
 The model also showed improved Pass@5, Pass@10, and partial test coverage.
 
----
+**---**
+
+****Base model:**** \`Qwen/Qwen2.5-Coder-7B-Instruct\` at snapshot
 
-**Base model:** `Qwen/Qwen2.5-Coder-7B-Instruct` at snapshot
-`c03e6d358207e414f1eca0bb1891e29f1db0e242`. The merge was performed against exactly these weights.
+\`c03e6d358207e414f1eca0bb1891e29f1db0e242\`. The adapter was trained against exactly these weights.
 
-## Downloading and using
+**## Downloading and using**
 
-### The merged model
+**### The LoRA adapter (recommended release)**
+
+\`\`\`python
 
-```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
 model = AutoModelForCausalLM.from_pretrained(
+
     "ranjanrajib/DPO-trained-Qwen2.5-Coder-7B", dtype="bfloat16", device_map="auto")
+
 tok = AutoTokenizer.from_pretrained("ranjanrajib/DPO-trained-Qwen2.5-Coder-7B")
 
 msgs = [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}]
+
 text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+
 out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=512)
+
 print(tok.decode(out[0], skip_special_tokens=True))
-```
+
+\`\`\`
 
 Or fetch the files without loading them (~14 GB, four shards):
 
-```bash
+\`\`\`bash
+
 hf download ranjanrajib/DPO-trained-Qwen2.5-Coder-7B --exclude "adapter/*" --local-dir ./d4
-```
 
-### The LoRA adapter only (308 MB)
+\`\`\`
+
+**### The LoRA adapter only (308 MB)**
+
+The adapter lives in \`adapter/\`. Applying it to the base model gives the same result as the
 
-The adapter lives in `adapter/`. Applying it to the base model gives the same result as the
 merged weights, at a fraction of the download.
 
-```bash
+\`\`\`bash
+
 hf download ranjanrajib/DPO-trained-Qwen2.5-Coder-7B --include "adapter/*" --local-dir ./d4-lora
-```
 
-```python
+\`\`\`
+
+\`\`\`python
+
 import torch
+
 from peft import PeftModel
+
 from transformers import AutoModelForCausalLM
 
 BASE = "Qwen/Qwen2.5-Coder-7B-Instruct"
+
 REVISION = "c03e6d358207e414f1eca0bb1891e29f1db0e242"   # the snapshot this was trained against
 
 base = AutoModelForCausalLM.from_pretrained(
+
     BASE, revision=REVISION, dtype=torch.bfloat16, device_map="auto")
+
 model = PeftModel.from_pretrained(
+
     base, "ranjanrajib/DPO-trained-Qwen2.5-Coder-7B", subfolder="adapter")
-```
----
 
-# Why Use This Model?
+\`\`\`
+
+**---**
+
+**# Why Use This Model?**
 
-If your primary use case is **Python code generation**, this model is intended to be a stronger alternative to the original Qwen2.5-Coder-7B-Instruct model based on the evaluation performed in this project.
+If your primary use case is ****Python code generation****, this model is intended as an experimental alternative to the original Qwen2.5-Coder-7B-Instruct model, with higher measured performance on the held-out benchmark used in this project.
 
 It is particularly relevant for users who want to:
 
 * generate Python functions and implementations
+
 * solve programming problems
+
 * generate code that needs to satisfy functional requirements
+
 * improve first-attempt coding correctness
+
 * experiment with DPO-trained code-generation models
+
 * study execution-based preference optimization
+
 * build research or evaluation pipelines for Python code generation
 
 The main motivation for the fine-tuning was not simply to make the model generate more code.
 
-It was to encourage the model toward **solutions that work**.
+It was to encourage the model toward ****solutions that work****.
 
-The preference data therefore used actual program execution and unit-test results rather than subjective judgments about whether generated code *looked* correct.
+The preference data therefore used actual program execution and unit-test results rather than subjective judgments about whether generated code **looked** correct.
 
----
+**---**
 
-# How Is It Different From Qwen2.5-Coder-7B-Instruct?
+**# How Is It Different From Qwen2.5-Coder-7B-Instruct?**
 
-The underlying model architecture and capabilities come from **Qwen2.5-Coder-7B-Instruct**.
+The underlying model architecture and capabilities come from ****Qwen2.5-Coder-7B-Instruct****.
 
 This model adds a preference-optimization stage on top of that model.
 
 Conceptually:
 
-```text
+\`\`\`text
+
 Qwen2.5-Coder-7B-Instruct
+
              │
+
              ▼
+
    Generate multiple Python
+
         candidate solutions
+
              │
+
              ▼
+
      Execute candidates
+
        against tests
+
              │
+
        ┌─────┴─────┐
+
        ▼           ▼
+
     Pass          Fail
+
        │           │
+
        ▼           ▼
+
     Chosen       Rejected
+
        │           │
+
        └─────┬─────┘
+
              ▼
+
             DPO
+
              │
+
              ▼
+
        LoRA Adapter
+
              │
+
              ▼
+
    Python-focused model
-```
+
+\`\`\`
 
 The goal is to shift the model toward coding behaviors associated with solutions that successfully execute against the available tests.
 
-### Important distinction
+**### Important distinction**
 
-This is **not a new foundation model**.
+This is ****not a new foundation model****.
 
 It is a parameter-efficient fine-tuned adapter for Qwen2.5-Coder-7B-Instruct.
 
 The base model remains unchanged.
 
----
+**---**
 
-# What Kind of Model Is This?
+**# What Kind of Model Is This?**
 
-### Primary task
+**### Primary task**
 
-**Python code generation**
+****Python code generation****
 
-### Model family
+**### Model family**
 
 Qwen2.5-Coder
 
-### Base model
+**### Base model**
 
-`Qwen/Qwen2.5-Coder-7B-Instruct`
+\`Qwen/Qwen2.5-Coder-7B-Instruct\`
 
-### Fine-tuning
+**### Fine-tuning**
 
 Direct Preference Optimization (DPO)
 
-### Parameter-efficient method
+**### Parameter-efficient method**
 
 LoRA
 
-### Precision
+**### Precision**
 
 bfloat16
 
-### Quantization
+**### Quantization**
 
 None
 
-### Model size
+**### Model size**
 
 7.7B-parameter base model
 
-### Adapter
+**### Adapter**
 
 Approximately 80.7M trainable parameters
 
----
+**---**
 
-# Performance
+**# Performance**
 
 The model was evaluated against the original Qwen2.5-Coder-7B-Instruct using a controlled benchmark consisting of 508 held-out BigCodeBench programming problems.
 
 The same prompts, generation settings, random seeds, number of samples, and grading procedure were used for both models.
 
-## Pass@k
+**## Pass@k**
+
+\| Metric  | Base Model |  DPO Model |
+
+\| ------- | ---------: | ---------: |
+
+\| Pass@1  |     41.87% | ****45.22%**** |
+
+\| Pass@5  |     48.87% | ****51.10%**** |
 
-| Metric  | Base Model |  DPO Model |
-| ------- | ---------: | ---------: |
-| Pass@1  |     41.87% | **45.22%** |
-| Pass@5  |     48.87% | **51.10%** |
-| Pass@10 |     50.99% | **53.16%** |
+\| Pass@10 |     50.99% | ****53.16%**** |
 
-The improvement in Pass@1 is approximately **3.35 percentage points**.
+The improvement in Pass@1 is approximately ****3.35 percentage points****.
 
-## Partial Correctness
+**## Partial Correctness**
 
 The average proportion of tests passed by each generated answer increased from:
 
-**66.20% → 69.41%**
+****66.20% → 69.41%****
 
 This is a secondary metric intended to provide additional information about functional correctness beyond binary problem-level pass/fail.
 
----
+**---**
 
-# Paired Problem-Level Analysis
+**# Paired Problem-Level Analysis**
 
 The evaluation used the same 508 problems for both models, allowing direct comparison of model behavior on individual problems.
 
 For the first generated answer:
 
-| Base → DPO            | Problems |
-| --------------------- | -------: |
-| Correct → Correct     |      186 |
-| Correct → Incorrect   |       17 |
-| Incorrect → Correct   |       41 |
-| Incorrect → Incorrect |      264 |
+\| Base → DPO            | Problems |
+
+\| --------------------- | -------: |
+
+\| Correct → Correct     |      186 |
+
+\| Correct → Incorrect   |       17 |
+
+\| Incorrect → Correct   |       41 |
+
+\| Incorrect → Incorrect |      264 |
 
 The DPO model converted 41 problems from incorrect to correct while 17 previously correct problems became incorrect.
 
 This provides additional evidence that the aggregate improvement is associated with meaningful changes in problem-level behavior rather than simply differences in the evaluated problem sets.
 
----
+**---**
 
-# Who Should Use This Model?
+**# Who Should Use This Model?**
 
 This model is most appropriate for:
 
-### Python Developers
+**### Python Developers**
 
 Developers interested in experimenting with a locally hosted code-generation model that has been specifically fine-tuned toward execution-validated solutions.
 
-### AI/ML Researchers
+**### AI/ML Researchers**
 
 Researchers studying:
 
 * DPO
+
 * preference learning
+
 * LLM alignment
+
 * code-generation models
+
 * automated preference construction
+
 * execution-based evaluation
+
 * parameter-efficient fine-tuning
 
-### LLM Evaluation Researchers
+**### LLM Evaluation Researchers**
 
 The model can also be useful as an experimental model for comparing:
 
 * base versus preference-tuned models
+
 * coding correctness
+
 * Pass@k
+
 * partial test coverage
+
 * preference-learning behavior
 
----
+**---**
 
-# When Should You Prefer the Original Qwen Model?
+**# When Should You Prefer the Original Qwen Model?**
 
 This model should not automatically be considered better for every possible task.
 
-The experiment specifically focused on **Python code generation**.
+The experiment specifically focused on ****Python code generation****.
 
 If your application requires broad general-purpose instruction following, other programming languages, or capabilities that were not evaluated here, you should benchmark both models for your specific workload.
 
 The reported improvement should therefore be interpreted as:
 
-> **Improved performance on the evaluated Python coding benchmark, not a guarantee of improvement across every coding or general-purpose task.**
+\> ****Improved performance on the evaluated Python coding benchmark, not a guarantee of improvement across every coding or general-purpose task.****
 
----
-
-# Quick Start
-
-This repository contains the **LoRA adapter**, not a merged copy of the full model.
+**---**
 
-You need the original base model:
+The base model and the trained model use the same VRAM.
 
-```text
-Qwen/Qwen2.5-Coder-7B-Instruct
-```
+The merge  changed weight values, not shapes — same 7.6B parameters, same
 
-and this adapter.
-
-Example with Transformers and PEFT:
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-from peft import PeftModel
+architecture — so they are byte-for-byte the same size.
 
-base_model = "Qwen/Qwen2.5-Coder-7B-Instruct"
-adapter = "ranjanrajib/DPO-trained-Qwen2.5-Coder-7B"
+  ┌───────────────────────────────┬────────────┐
 
-tokenizer = AutoTokenizer.from_pretrained(base_model)
+  │                               │ bf16 VRAM  │
 
-model = AutoModelForCausalLM.from_pretrained(
-    base_model,
-    torch_dtype="auto",
-    device_map="auto"
-)
+  ├───────────────────────────────┼────────────┤
 
-model = PeftModel.from_pretrained(
-    model,
-    adapter
-)
+  │ Weights (base or merged D4)   │ 14.19 GiB  │
 
-prompt = """Write a Python function that returns the
-factorial of a number."""
+  ├───────────────────────────────┼────────────┤
 
-messages = [
-    {"role": "user", "content": prompt}
-]
+  │ KV cache, per token           │ 56 KiB     │
 
-text = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
+  ├───────────────────────────────┼────────────┤
 
-inputs = tokenizer(text, return_tensors="pt").to(model.device)
+  │ Activations / workspace       │ ~0.5–1 GiB │
 
-outputs = model.generate(
-    **inputs,
-    max_new_tokens=256,
-    temperature=0.2,
-    top_p=0.95
-)
+  ├───────────────────────────────┼────────────┤
 
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
+  │ Practical single-stream total │ ~15–16 GiB │
 
-For production serving, the adapter can also be served using infrastructure that supports LoRA adapters.
+  └───────────────────────────────┴────────────┘
 
----
+**---**
 
-# Model Architecture and Training
+**# Model Architecture and Training**
 
 The model starts from:
 
-`Qwen/Qwen2.5-Coder-7B-Instruct`
+\`Qwen/Qwen2.5-Coder-7B-Instruct\`
 
 The base model contains approximately 7.7 billion parameters.
 
-Rather than updating all parameters, the experiment uses **LoRA-based parameter-efficient fine-tuning**.
+Rather than updating all parameters, the experiment uses ****LoRA-based parameter-efficient fine-tuning****.
 
 The resulting adapter contains approximately 80.7 million trainable parameters.
 
 The adapter is approximately 308 MB.
 
----
+**---**
 
-# DPO Training
+**# DPO Training**
 
-## Why DPO?
+**## Why DPO?**
 
 Direct Preference Optimization provides a method for optimizing a language model directly from preference pairs without requiring a separate reward-model training stage.
 
 For this experiment, a preference pair has the form:
 
-```text
+\`\`\`text
+
 Problem
+
    │
+
    ├── Candidate A → passes tests → chosen
+
    │
+
    └── Candidate B → fails tests  → rejected
-```
+
+\`\`\`
 
 DPO then learns to increase the relative likelihood of the chosen response compared with the rejected response.
 
----
+**---**
 
-# Preference Data Construction
+**# Preference Data Construction**
 
 The preference dataset was generated specifically for this experiment.
 
 It was constructed from programming problems drawn from:
 
 * BigCodeBench
+
 * MBPP
+
 * HumanEval
+
 * handwritten programming problems
 
-### Training Data
+**### Training Data**
+
+\| Dataset      | License    |
+
+\| ------------ | ---------- |
+
+\| BigCodeBench | Apache 2.0 |
+
+\| MBPP         | CC BY 4.0  |
 
-| Dataset      | License    |
-| ------------ | ---------- |
-| BigCodeBench | Apache 2.0 |
-| MBPP         | CC BY 4.0  |
-| HumanEval    | MIT        |
+\| HumanEval    | MIT        |
 
 Candidate solutions were generated automatically.
 
 Five generation strategies were used:
 
 * normal
+
 * straightforward
+
 * edge-case-focused
+
 * alternative
+
 * optimized
 
 Multiple candidates were generated for each problem.
@@ -444,18 +507,21 @@ Every candidate was then executed in a Docker sandbox against the associated uni
 
 The resulting execution outcome was used to construct chosen/rejected preference pairs.
 
----
+**---**
 
-# Preference Dataset
+**# Preference Dataset**
 
 After filtering and deduplication:
 
 * 1,326 preference pairs
+
 * 221 programming problems
+
 * 933 training pairs
+
 * 114 validation pairs
 
-The dataset was split by **problem**, rather than by individual preference pair.
+The dataset was split by ****problem****, rather than by individual preference pair.
 
 This prevents different candidate solutions from the same underlying programming problem from being distributed across training and validation.
 
@@ -465,91 +531,121 @@ No external LLM was used as a preference judge.
 
 The preference signal was generated from objective unit-test execution.
 
----
+**---**
 
-# DPO Configuration
+**# DPO Configuration**
 
 The primary DPO configuration was:
 
-| Parameter               | Value   |
-| ----------------------- | ------- |
-| DPO beta                | 0.1     |
-| Learning rate           | 1e-5    |
-| Scheduler               | Cosine  |
-| Warmup                  | 5%      |
-| Epochs                  | 2       |
-| Maximum sequence length | 1,024   |
-| Batch size              | 4       |
-| Gradient accumulation   | 4       |
-| Effective batch size    | 16      |
-| Optimizer               | AdamW   |
-| Gradient clipping       | 1.0     |
-| Gradient checkpointing  | Enabled |
-
-## LoRA Configuration
-
-| Parameter      | Value                         |
-| -------------- | ----------------------------- |
-| Rank           | 32                            |
-| Alpha          | 64                            |
-| Dropout        | 0.05                          |
-| Target modules | Attention and MLP projections |
+\| Parameter               | Value   |
 
----
+\| ----------------------- | ------- |
+
+\| DPO beta                | 0.1     |
+
+\| Learning rate           | 1e-5    |
 
-# Training Hardware
+\| Scheduler               | Cosine  |
+
+\| Warmup                  | 5%      |
+
+\| Epochs                  | 2       |
+
+\| Maximum sequence length | 1,024   |
+
+\| Batch size              | 4       |
+
+\| Gradient accumulation   | 4       |
+
+\| Effective batch size    | 16      |
+
+\| Optimizer               | AdamW   |
+
+\| Gradient clipping       | 1.0     |
+
+\| Gradient checkpointing  | Enabled |
+
+**## LoRA Configuration**
+
+\| Parameter      | Value                         |
+
+\| -------------- | ----------------------------- |
+
+\| Rank           | 32                            |
+
+\| Alpha          | 64                            |
+
+\| Dropout        | 0.05                          |
+
+\| Target modules | Attention and MLP projections |
+
+**---**
+
+**# Training Hardware**
 
 DPO training was performed on:
 
-**NVIDIA RTX A6000 — 48 GB VRAM**
+****NVIDIA RTX A6000 — 48 GB VRAM****
 
 Training environment:
 
 * CUDA 12.8
+
 * PyTorch 2.11
+
 * bfloat16
+
 * gradient checkpointing
 
 Peak GPU memory consumption was approximately 33.4 GiB.
 
----
+**---**
 
-# Evaluation Methodology
+**# Evaluation Methodology**
 
 The primary evaluation benchmark consisted of:
 
-**508 BigCodeBench programming problems**
+****508 BigCodeBench programming problems****
 
 The evaluation problems were reserved separately from the preference-training pool.
 
 The same generation and grading procedure was applied to both models.
 
-### Generation
+**### Generation**
 
 * 10 answers per problem
+
 * temperature: 0.2
+
 * top-p: 0.95
+
 * maximum new tokens: 512
+
 * repetition penalty: 1.0
+
 * sampling enabled
+
 * identical random seeds for base and DPO models
 
-### Pass@k
+**### Pass@k**
 
 Pass@k was calculated using the standard unbiased estimator based on the number of correct samples among the generated candidates.
 
----
+**---**
 
-# Preference-Learning Diagnostics
+**# Preference-Learning Diagnostics**
 
 The DPO training objective produced a strong preference signal on the training examples.
 
 However, the validation results were substantially weaker.
 
-| Metric              | Training |            Validation |
-| ------------------- | -------: | --------------------: |
-| Preference accuracy |   ~84.4% |                ~45.8% |
-| Reward margin       | Positive | Approximately neutral |
+\| Metric              | Training |            Validation |
+
+\| ------------------- | -------: | --------------------: |
+
+\| Preference accuracy |   ~84.4% |                ~45.8% |
+
+\| Reward margin       | Positive | Approximately neutral |
 
 This indicates that the explicit preference signal learned during training did not generalize strongly to the validation preference examples.
 
@@ -557,28 +653,31 @@ At the same time, downstream coding performance improved on the held-out benchma
 
 This is an important observation from the experiment.
 
----
+**---**
 
-# What Does This Mean?
+**# What Does This Mean?**
 
 The results should not be interpreted as proof that the model learned a broadly generalized notion of code quality.
 
-Instead, they suggest that **optimizing an automatically constructed preference objective can improve downstream coding performance even when explicit preference metrics show weak validation generalization.**
+Instead, they suggest that ****optimizing an automatically constructed preference objective can improve downstream coding performance even when explicit preference metrics show weak validation generalization.****
 
 Several explanations are possible, including:
 
 * differences between training and validation preference distributions
+
 * imperfect test-derived preference labels
+
 * overfitting to characteristics of the training preference pairs
+
 * changes in generation behavior that are not fully captured by preference accuracy
 
 These are hypotheses for further investigation rather than established conclusions.
 
----
+**---**
 
-# Important Limitations
+**# Important Limitations**
 
-## No Human Preference Evaluation
+**## No Human Preference Evaluation**
 
 No human preference study was performed.
 
@@ -586,99 +685,127 @@ The preference signal was derived from automated unit-test execution.
 
 Therefore, this model should not be described as being aligned to human preferences based on this experiment.
 
-## Unit Tests Are Imperfect
+**## Unit Tests Are Imperfect**
 
 Passing the available tests does not guarantee that a solution is universally correct.
 
 A test suite may fail to cover some edge cases or undesirable implementation behavior.
 
-## Benchmark Source Overlap
+**## Benchmark Source Overlap**
 
 The evaluation problems were held out at the problem level.
 
 However, the preference-training data includes other BigCodeBench problems.
 
-Therefore, the evaluation is **problem-held-out**, but not completely source-family-disjoint.
+Therefore, the evaluation is ****problem-held-out****, but not completely source-family-disjoint.
 
-## Python Focus
+**## Python Focus**
 
 The primary objective and evaluation were focused on Python programming.
 
 Performance should not be assumed to generalize to other programming languages without additional evaluation.
 
-## Experimental Model
+**## Experimental Model**
 
 This is an experimental research model.
 
 Users should evaluate it against their own workloads before using it in production.
 
----
+**---**
 
-# Reproducibility
+**# Reproducibility**
 
 The experiment documents:
 
 * base model
+
 * preference-data construction
+
 * data sources
+
 * dataset split methodology
+
 * DPO configuration
+
 * LoRA configuration
+
 * training hardware
+
 * generation configuration
+
 * evaluation methodology
+
 * random seed strategy
+
 * benchmark methodology
+
 * limitations
 
 The objective is to make the experiment sufficiently transparent for others to reproduce, extend, or critique.
 
----
+**---**
 
-# Independent Research Project
+**# Independent Research Project**
 
-This model was developed as an **independent, out-of-office AI/ML research project**.
+This model was developed as an ****independent, out-of-office AI/ML research project****.
 
 The project was conceived, designed, implemented, trained, evaluated, analyzed, and released independently.
 
 The work covers the complete experimental lifecycle:
 
-```text
+\`\`\`text
+
 Research Question
+
        ↓
+
 Experimental Design
+
        ↓
+
 Preference Data Generation
+
        ↓
+
 Automated Evaluation
+
        ↓
+
 DPO + LoRA Training
+
        ↓
+
 Controlled Benchmarking
+
        ↓
+
 Statistical / Paired Analysis
+
        ↓
+
 Model Release
-```
 
----
+\`\`\`
+
+**---**
 
-# Model Release
+**# Model Release**
 
 The trained LoRA adapter and technical documentation are available here:
 
-**Hugging Face:**
+****Hugging Face:****
+
 https://huggingface.co/ranjanrajib/DPO-trained-Qwen2.5-Coder-7B
 
-The repository contains the model adapter and documentation required to understand the training methodology, evaluation, results, intended use, and limitations.
+The repository contains the LoRA adapter and documentation required to understand the training methodology, evaluation, results, intended use, and limitations. The base Qwen weights are not redistributed in this repository.
 
----
+**---**
 
-# Licensing and Attribution
+**# Licensing and Attribution**
 
 The model is derived from:
 
-`Qwen/Qwen2.5-Coder-7B-Instruct`
+\`Qwen/Qwen2.5-Coder-7B-Instruct\`
 
 Users should review and comply with the licensing and usage terms of the base model.
 
@@ -686,47 +813,50 @@ The preference-generation pipeline used programming problems from BigCodeBench,
 
 The respective dataset licenses and attribution requirements should be observed when using or redistributing material derived from those datasets.
 
----
+**---**
 
-# Citation
+**# Citation**
 
 If you use this model or the experimental methodology in your work, please cite the original Qwen2.5-Coder model and the relevant datasets used in the experiment.
 
----
+**---**
 
-# Summary
+**# Summary**
 
-**What is this?**
+****What is this?****
 
 A Python-focused DPO fine-tuned version of Qwen2.5-Coder-7B-Instruct.
 
-**What is different?**
+****What is different?****
 
 The model was trained using LoRA-based Direct Preference Optimization with automatically constructed preferences derived from unit-test execution.
 
-**Does it improve coding performance?**
+****Does it improve coding performance?****
 
-On the evaluated held-out benchmark, yes. Pass@1 improved from **41.87% to 45.22%**, with improvements also observed in Pass@5, Pass@10, and partial test coverage.
+On the evaluated held-out benchmark, yes. Pass@1 improved from ****41.87% to 45.22%****, with improvements also observed in Pass@5, Pass@10, and partial test coverage.
 
-**Is it human-aligned?**
+****Is it human-aligned?****
 
 No human preference evaluation was performed. The preference signal was generated automatically from unit-test outcomes.
 
-**Who should use it?**
+****Who should use it?****
 
 Users primarily interested in Python code generation, coding experiments, DPO research, preference learning, and execution-based LLM evaluation.
 
-**What is being released?**
+****What is being released?****
 
-A LoRA adapter that can be loaded on top of `Qwen/Qwen2.5-Coder-7B-Instruct`.
+A LoRA adapter that can be loaded on top of \`Qwen/Qwen2.5-Coder-7B-Instruct\`.
 
-**Why is the experiment interesting?**
+****Why is the experiment interesting?****
 
 The model improved downstream coding performance even though explicit preference metrics showed weak validation generalization, highlighting the distinction between optimizing a preference objective and improving downstream model capability.
 
 
-## Licence and attribution
 
-Apache-2.0, inherited from `Qwen/Qwen2.5-Coder-7B-Instruct`. This is a modified derivative of that
+**## Licence and attribution**
+
+Apache-2.0, inherited from \`Qwen/Qwen2.5-Coder-7B-Instruct\`. This is a modified derivative of that
+
 model; the original is by the Qwen team at Alibaba Cloud. The preference data was generated from the
+
 base model's own outputs on BigCodeBench problems.