Initial release

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,62 @@
+---
+base_model: Qwen/Qwen3.5-9B
+library_name: peft
+license: apache-2.0
+pipeline_tag: image-text-to-text
+tags:
+- lora
+- peft
+- presentation
+- slide-editing
+- visual-qa
+---
+
+# EditPPT Vision Validator (LoRA on Qwen3.5-9B)
+
+A LoRA adapter that checks whether an edit to a PowerPoint slide **newly introduced a severe visual defect**,
+and if so, returns a repair as tool calls — in one generation. It serves as the vision validator of EditPPT.
+
+Given the slide rendered **before** and **after** an edit, plus the edit instruction, the tool calls that made the
+edit and the slide's shape JSON, the model returns:
+
+```json
+{"HasCriticalIssues": "Yes",
+ "Issues": [{"IssueType": "ELEMENT_OVERLAP", "AffectedShapeIDs": [2],
+             "TechnicalDiagnosis": "textbox 2 overlaps the caption below it by about 20 pt"}],
+ "ToolCalls": [{"name": "adjust_layout", "arguments": {"slide_number": 3, "shape_id": 2, "top": 470}}]}
+```
+
+* **Defect types:** `CONTENT_OFF_SLIDE`, `ELEMENT_OVERLAP`, `LOW_CONTRAST`.
+* **Repair tools:** `set_shape_font_size`, `adjust_layout`, `set_text_style`. Values are absolute targets in slide points.
+* Defects already present in the BEFORE image are treated as the deck's own design and are not reported.
+
+## Usage
+
+See `inference_example.py` and `prompt_template.txt`.
+
+* Message order: `[image BEFORE, image AFTER, prompt text]`.
+* Images: 960x540 (`min_pixels = max_pixels = 960*540`). One slide point = one pixel for a 16:9 deck.
+* Greedy decoding, default chat template.
+
+## Training
+
+| | |
+|---|---|
+| Base | Qwen3.5-9B, bf16 |
+| Adapter | LoRA, r 16, alpha 32, on the language model and the vision encoder |
+| Data | 7,260 training samples (plus 374 for validation) with human-confirmed labels, from publicly licensed decks (CC0 / CC-BY) |
+
+## Evaluation
+
+Held-out set of 750 samples from decks not seen in training:
+
+| Metric | Value |
+|---|---|
+| Verdict accuracy, edits without a repair (n=497) | 0.932 (base model: 0.60) |
+| Verdict accuracy, all 750 samples | 0.955 |
+| Defect precision / recall / F1, all 750 samples | 0.963 / 0.954 / 0.959 |
+| Repair targets the right shapes with a permitted tool, all 750 samples | 0.948 |
+
+## License
+
+Apache-2.0, following the base model.

adapter_config.json

@@ -0,0 +1,43 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3.5-9B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.2",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*language_model.*\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj))|(?:.*language_model.*\\.linear_attn\\.(in_proj_qkv|in_proj_z|out_proj))|(?:.*visual\\.(blocks\\.\\d+\\.(attn\\.qkv|attn\\.proj|mlp\\.linear_fc1|mlp\\.linear_fc2)|merger\\.(linear_fc1|linear_fc2)))",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2146dc09d1a49801a4bcb6058250b4fee4b9586a13e7b9d834d27ec8f9c72c0c
+size 192469744

inference_example.py

@@ -0,0 +1,34 @@
+"""Minimal inference with the EditPPT vision-validator LoRA on Qwen3.5-9B.
+
+Input order matters: [image BEFORE the edit, image AFTER the edit, prompt]. Render slides at 960x540
+(1 slide pt = 1 px for a standard 16:9 deck). Fill prompt_template.txt with str.replace, not str.format
+(the template contains literal JSON braces).
+"""
+import json
+
+import torch
+from peft import PeftModel
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
+
+BASE, ADAPTER = "Qwen/Qwen3.5-9B", "EditPPT/editppt-vision-validator-lora"
+PIXELS = 960 * 540
+
+processor = AutoProcessor.from_pretrained(BASE, min_pixels=PIXELS, max_pixels=PIXELS)
+model = AutoModelForImageTextToText.from_pretrained(BASE, dtype=torch.bfloat16, device_map="cuda:0")
+model = PeftModel.from_pretrained(model, ADAPTER).eval()
+
+
+def validate(before_png, after_png, instruction, tools_used, slide_json):
+    prompt = (open("prompt_template.txt").read()
+              .replace("{instruction}", instruction)
+              .replace("{tools_used}", str(tools_used))   # python-repr list of {'name', 'arguments'}
+              .replace("{slide_json}", str(slide_json)))  # python-repr dict: slide_w, slide_h, current_slide, objects
+    msgs = [{"role": "user", "content": [{"type": "image"}, {"type": "image"}, {"type": "text", "text": prompt}]}]
+    text = processor.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False)
+    imgs = [Image.open(p).convert("RGB") for p in (before_png, after_png)]
+    enc = processor(text=[text], images=imgs, return_tensors="pt").to(model.device)
+    with torch.no_grad():
+        out = model.generate(**enc, max_new_tokens=1024, do_sample=False)
+    raw = processor.tokenizer.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True)
+    return json.loads(raw[raw.index("{"): raw.rindex("}") + 1])

prompt_template.txt

@@ -0,0 +1,68 @@
+You are a PPT QA specialist. Evaluate whether THIS modification newly introduced a SEVERE visual defect.
+
+## Images
+Two renders of the same slide are attached: the FIRST is the slide BEFORE this modification, the
+SECOND is AFTER it. Judge only what changed between them — anything already visible in the first
+image is the deck's own design and is NOT a defect of this modification.
+
+## What was modified
+{instruction}
+
+## Tools used
+{tools_used}
+
+## Defect categories
+CONTENT_OFF_SLIDE   content crosses the slide edge and is cut off
+ELEMENT_OVERLAP     content collides with other content, is painted over by it, or spills across a visible boundary (another shape, or its own filled box), so it cannot be read
+LOW_CONTRAST        text cannot be read against the background behind it (e.g. dark text on a dark band)
+
+## Reporting rule
+Report a defect ONLY if it (a) was newly caused by this modification on the modified shapes or their spatial neighbors, (b) is immediately obvious at normal presentation scale, and (c) clearly degrades readability/credibility. If uncertain, do NOT report.
+
+## Repair tools
+A repair is a tool call on the AFTER slide. Only these three tools exist. `slide_number` is the JSON
+`current_slide`, `shape_id` is the JSON `id`, and every value is an absolute target in the slide JSON
+units (the canvas is `slide_w` × `slide_h` from the JSON, 960x540 for a standard 16:9 deck; no unit
+conversion) — read the current value from the slide JSON below (each
+object has `id`, `left`, `top`, `width`, `height`, `max_font_size`, and, when text exists, `text`).
+- set_shape_font_size(slide_number, shape_id, font_size): set the font size (pt) of every run in the
+  shape; the current size is `max_font_size`.
+- adjust_layout(slide_number, shape_id, left?, top?, width?, height?): set the box's position and/or
+  size; pass only the fields that change, as target values, never as deltas.
+- set_text_style(slide_number, shape_id, scope="all", color_hex): recolour all text in the shape,
+  e.g. "#FFFFFF" when the text sits on a dark background (LOW_CONTRAST).
+
+## Repair rule
+Fix the shape THIS modification changed, and leave the rest of the slide as the author designed it.
+The defect came from that edit, so the repair belongs there: move it, resize its box, or reduce its
+font size.
+
+Choose by the space around the shape, not by habit:
+- Free space beside, above or below the shape -> MOVE it into that space, along one axis where that
+  is enough, so the slide's existing alignment survives.
+- No free space -> make the shape fit instead: reduce its font size, or change its box.
+- A box's width is not a fixed property of the shape. Widen it when the text is cramped and there is
+  room next to it; narrow it when the shape runs past the slide edge and the text can wrap instead.
+
+Combine actions when one alone leaves the defect — moving a box up AND reducing its font size is one
+repair, not two. Prefer the smallest change that removes the defect: one action when one is enough,
+two when one is not.
+
+Name a neighbouring shape only when nothing the edited shape may legally change can resolve the
+defect (e.g. its font size was what the user asked for and is therefore fixed) — moving a shape the
+user never mentioned is itself a change to their slide, so it is a last resort, and then move it by
+the smallest amount that clears the overlap.
+
+## Output format
+Return one JSON object and nothing else:
+{"HasCriticalIssues": "Yes" | "No",
+ "Issues": [{"IssueType": "<category>", "AffectedShapeIDs": [<id>, ...],
+             "TechnicalDiagnosis": "<what is wrong, on which shape, by how much>"}],
+ "ToolCalls": [{"name": "<tool>", "arguments": {...}}],
+ "Unfixable": "<why no permitted change removes the defect>"}
+"No" means empty `Issues` and empty `ToolCalls`. Every "Yes" carries at least one tool call, except
+when the request itself fixed the only property that could change — then `ToolCalls` is empty and
+`Unfixable` says so. Omit `Unfixable` otherwise.
+
+## Slide JSON
+{slide_json}