Add usage example, interleaved input spec, and download guidance

README.md

@@ -17,6 +17,8 @@ datasets:
 
 PRIMO R1 (Process Reasoning Induced Monitoring) is a 7B video multimodal large language model (MLLM) framework designed for accurate process supervision in long-horizon robotic manipulation. It was introduced in the paper [From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation](https://huggingface.co/papers/2603.15600).
 
+This is the **final checkpoint**: a Qwen2.5-VL-7B-Instruct base taken through an SFT cold start and then GRPO reinforcement learning. Use this model unless you specifically want the ablation.
+
 ## Model Description
 
 Current video MLLMs often function as passive "Observers" that recognize ongoing events rather than evaluating the current state relative to the final task goal. PRIMO R1 transforms these models into active "Critics" by:
@@ -25,12 +27,264 @@ Current video MLLMs often function as passive "Observers" that recognize ongoing
 - **Temporal Anchoring**: Constructing a structured temporal input that explicitly anchors the video sequence between initial and current state images.
 - **Process Reasoning**: Focusing on evaluating the current state against the intended task goal to detect failures and track progress.
 
+## Resources
+
+| | |
+| --- | --- |
+| Code | [10-OASIS-01/PRIMO-R1](https://github.com/10-OASIS-01/PRIMO-R1) |
+| Collection | [PRIMO R1](https://huggingface.co/collections/LeonOverload/primo-r1) |
+| Paper | [arXiv 2603.15600](https://arxiv.org/abs/2603.15600) · [project page](https://10-oasis-01.github.io/primo-r1-website/) |
+| Stage-1 ablation | [PRIMO-COT-SFT-7B](https://huggingface.co/LeonOverload/PRIMO-COT-SFT-7B) |
+| Benchmark | [primo-bench-json](https://huggingface.co/datasets/LeonOverload/primo-bench-json) |
+| Training data | [primo-sft-json](https://huggingface.co/datasets/LeonOverload/primo-sft-json) · [primo-rl-json](https://huggingface.co/datasets/LeonOverload/primo-rl-json) |
+| Videos | [primo-video-media](https://huggingface.co/datasets/LeonOverload/primo-video-media) |
+
+## Download
+
+The `main` branch holds inference files only (**16.6 GB**). DeepSpeed resume state from RL step 2500 lives on a separate branch, so you do not pay ~100 GB for it by accident.
+
+```python
+from huggingface_hub import snapshot_download
+
+# Inference: 16.6 GB
+snapshot_download("LeonOverload/PRIMO-R1-7B", local_dir="models/PRIMO-R1-7B")
+
+# Resume RL training from step 2500: ~116 GB
+snapshot_download("LeonOverload/PRIMO-R1-7B", revision="training-state")
+```
+
+Or from the CLI:
+
+```bash
+hf download LeonOverload/PRIMO-R1-7B --local-dir models/PRIMO-R1-7B
+```
+
+If you are on a revision where the optimizer state is still on `main`, exclude it explicitly. Note that `--exclude` is ignored if you also pass filenames positionally:
+
+```bash
+hf download LeonOverload/PRIMO-R1-7B \
+    --exclude "global_step*" --exclude "rng_state*" \
+    --exclude "scheduler.pt" --exclude "latest" \
+    --local-dir models/PRIMO-R1-7B
+```
+
+## Setup
+
+The environment matters more than usual here. Qwen2.5-VL support shifted between transformers releases, and installing a PyPI `transformers` over the pinned tree is the most common cause of shape and processor errors:
+
+```bash
+git clone https://github.com/10-OASIS-01/PRIMO-R1 && cd PRIMO-R1
+conda create -n primo-r1 python=3.11 && conda activate primo-r1
+bash setup.sh
+```
+
+`setup.sh` pins `vllm==0.7.2`, `trl==0.16.0`, and installs the vendored `transformers-main/` tree last so nothing replaces it.
+
+## Input format — read this before running anything
+
+PRIMO R1 is trained on an **interleaved** input that anchors the clip between two still frames. The content list must be in exactly this order:
+
+```
+image (initial frame)  →  video (the clip)  →  image (current frame)  →  text (question)
+```
+
+Feeding a bare video clip will still produce output, but it degrades quality silently — the paper's Table 4 ablation measures this directly, and a current-state image alone raises average MAE from 15.52 to 59.50. Keep the order and keep both anchor frames.
+
+The prompt template is equally load-bearing: the answer extractor is a regex over `<answer>...</answer>`, and the `<think>` block is required to contain `<planning>`, `<observation>`, and `<reasoning>` subsections in that order. Changing the template without changing the extractor collapses scores rather than raising an error.
+
+## Usage
+
+This example is transcribed from `src/eval/eval_interleave.py`, the harness that produced the published numbers.
+
+```python
+import cv2
+import torch
+from PIL import Image
+from transformers import AutoProcessor, AutoTokenizer
+from vllm import LLM, SamplingParams
+from qwen_vl_utils import process_vision_info
+
+MODEL_PATH = "models/PRIMO-R1-7B"   # or "LeonOverload/PRIMO-R1-7B"
+video_path = "path/to/your/episode.mp4"
+question = "What is the completion percentage of the task in the video?"
+problem_type = "regression"
+
+SYSTEM_PROMPT = (
+    "A conversation between User and Assistant. The Assistant is an expert AI specializing in embodied procedure and event reasoning based on visual input (video or images). "
+    "The assistant must strictly follow a specific thought process and output format. "
+    "The reasoning process is enclosed within <think> </think> tags, and the final answer is within <answer> </answer> tags. "
+    "The <think> block must contain three ordered subsections: <planning>, <observation>, and <reasoning>. "
+    "The <answer> block must contain only the final output required by the question type and no other commentary."
+)
+
+QUESTION_TEMPLATE = (
+    "QUESTION:\n{Question}\n\n"
+    "QUESTION TYPE:\n{question_type}\n\n"
+    "Analyze the provided visual data and reason about the ongoing task.\n\n"
+    "Please think about this question as if you were a human pondering deeply. "
+    "Provide your detailed reasoning between the <think> and </think> tags, following the subsections <planning>, <observation>, and <reasoning>. "
+    "Then give your final answer between the <answer> and </answer> tags.\n\n"
+    "Below is the required template:\n\n"
+    "<think>\n"
+    "<planning>\n"
+    "Identify the high-level goal of the agent, what is the initial state? What does successful completion look like?\n"
+    "Break down the high-level goal into a logical sequence of canonical steps. This serves as your mental plan for interpreting the task.\n"
+    "Use this plan to interpret actions, map observed behaviors to steps, assess progress, detect anomalies, and predict what happens next.\n"
+    "</planning>\n"
+    "<observation>\n"
+    "View the video as a temporal sequence of actions contributing to the procedure.\n"
+    "Objectively describe what is occurring in the current moment, noting evidence of progress or state changes.\n"
+    "Identify fine-grained actions and explain how they move the task forward.\n"
+    "List relevant objects, tools, and environmental context, emphasizing functional states and transformations.\n"
+    "Note cues—repetition, transitions, or completion indicators—that situate the action in the procedural script.\n"
+    "</observation>\n"
+    "<reasoning>\n"
+    "Think through the question as a human would, Engage in an internal dialogue using expressions such as 'let me think', 'wait', 'hmm', 'oh, I see', 'let's break it down', etc.\n"
+    "Connect observations to the procedural plan to determine which step is being executed, progress, correctness, or anomalies.\n"
+    "Reflect on assumptions, verify interpretations, and, if appropriate, predict the agent's next likely action.\n"
+    "Synthesize understanding of what the agent is doing, how it fits into the broader task, and whether the process seems successful.\n"
+    "You are encouraged to include self-reflection or verification in your reasoning process.\n"
+    "</reasoning>\n"
+    "</think>\n"
+    "<answer>\n"
+    "[Final answer here — strictly follow the `{question_type}` output format and include no extra commentary.]\n"
+    "</answer>"
+)
+
+TYPE_TEMPLATE = {
+    "multiple choice": " Please provide only the single option letter (e.g., A, B, C, D, etc.) within the <answer> </answer> tags.",
+    "numerical": " Please provide the numerical value (e.g., 42 or 3.14) within the <answer> </answer> tags.",
+    "OCR": " Please transcribe text from the image/video clearly and provide your text answer within the <answer> </answer> tags.",
+    "free-form": " Please provide your text answer within the <answer> </answer> tags.",
+    "regression": " Please provide the numerical value (e.g., 42 or 3.14) within the <answer> </answer> tags.",
+    "boolean": " Please provide only 'Yes' or 'No' as your answer within the <answer> </answer> tags.",
+}
+
+
+def extract_first_and_last_frame(path):
+    """Initial state = first frame, current state = last decodable frame."""
+    cap = cv2.VideoCapture(path)
+    ok, first = cap.read()
+    if not ok:
+        raise RuntimeError(f"cannot read {path}")
+    total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
+    cap.set(cv2.CAP_PROP_POS_FRAMES, max(0, total - 1))
+    ok, last = cap.read()
+    if not ok:                                    # some containers mis-report the count
+        cap.set(cv2.CAP_PROP_POS_FRAMES, max(0, total - 2))
+        ok, last = cap.read()
+    cap.release()
+    to_pil = lambda f: Image.fromarray(cv2.cvtColor(f, cv2.COLOR_BGR2RGB))
+    return to_pil(first), to_pil(last if ok else first)
+
+
+init_img, current_img = extract_first_and_last_frame(video_path)
+
+messages = [
+    {"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "image": init_img},                      # 1. initial state
+            {"type": "video", "video": video_path, "nframes": 22},     # 2. the clip
+            {"type": "image", "image": current_img},                   # 3. current state
+            {
+                "type": "text",
+                "text": QUESTION_TEMPLATE.format(
+                    Question=question, question_type=problem_type
+                ) + TYPE_TEMPLATE[problem_type],
+            },
+        ],
+    },
+]
+
+llm = LLM(
+    model=MODEL_PATH,
+    tensor_parallel_size=torch.cuda.device_count(),
+    max_model_len=16384,
+    gpu_memory_utilization=0.8,
+    limit_mm_per_prompt={"image": 3, "video": 1},   # 2 anchor frames + 1 video
+)
+
+# top_p must stay this low. Larger values produce garbled output on this model.
+sampling_params = SamplingParams(temperature=0.1, top_p=0.001, max_tokens=4096)
+
+processor = AutoProcessor.from_pretrained(MODEL_PATH)
+tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
+tokenizer.padding_side = "left"
+processor.tokenizer = tokenizer
+
+prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True)
+
+mm_data = {"video": video_inputs[0]}
+if image_inputs:
+    mm_data["image"] = image_inputs
+
+outputs = llm.generate(
+    [{
+        "prompt": prompt,
+        "multi_modal_data": mm_data,
+        "mm_processor_kwargs": {k: v[0] for k, v in video_kwargs.items()},
+    }],
+    sampling_params=sampling_params,
+)
+print(outputs[0].outputs[0].text)
+```
+
+### Parsing the output
+
+```python
+import re
+
+def extract_answer(text):
+    m = re.search(r"<answer>\s*(.*?)\s*</answer>", text, re.DOTALL)
+    return m.group(1).strip() if m else ""
+
+def extract_think(text):
+    m = re.search(r"<think>\s*(.*?)\s*</think>", text, re.DOTALL)
+    return m.group(1).strip() if m else ""
+```
+
+For `regression` and `numerical` questions the answer is a **progress percentage on a 0–100 scale** (`42.5`, not `0.425`). A trailing `%` is tolerated by the reference parser, which divides by 100 when it sees one.
+
+### Question types
+
+`regression` and `numerical` (progress estimation), `multiple choice`, `boolean` (failure detection), `free-form`, `OCR`. Pass the type both in `QUESTION_TEMPLATE` and via `TYPE_TEMPLATE`; the model was trained with both present.
+
+### A note on frame counts
+
+The published results were produced with the per-video frame cap at 22, so the launcher's `--nframes 32` effectively sampled 22. `nframes=22` above reproduces that. Training capped videos at 16 frames; eval samples more at higher resolution.
+
 ## Performance
 
-PRIMO R1 achieves state-of-the-art performance across several benchmarks:
-- **RoboFail Benchmark**: Achieves 67.0% accuracy, surpassing closed-source models like OpenAI o1 by 6.0%.
-- **Error Reduction**: Demonstrates a 50% reduction in the mean absolute error compared to specialized reasoning baselines.
-- **Generalization**: Shows strong zero-shot generalization on difficult failure detection tasks in out-of-domain real-world humanoid scenarios.
+Progress estimation averaged over four environments (paper Table 1):
+
+| Model | Avg MRA ↑ | Avg MAE ↓ |
+| --- | --- | --- |
+| GPT-4o | 79.33 | 20.67 |
+| GPT-5 mini | 75.38 | 23.96 |
+| Qwen2.5-VL-72B | 73.80 | 23.80 |
+| Qwen2.5-VL-7B (base) | 67.79 | 29.99 |
+| InternVL 3.5 8B | 71.74 | 28.09 |
+| ProgressLM | 78.32 | 20.87 |
+| VLAC | 74.90 | 25.10 |
+| **PRIMO R1 (7B)** | **82.90** | **15.52** |
+
+Both training stages are needed (paper Table 2, MRA↑):
+
+| Model | ID avg | OOD avg | Overall |
+| --- | --- | --- | --- |
+| Qwen2.5-VL-7B (base) | 70.38 | 65.26 | 67.46 |
+| SFT only | 81.46 | 77.77 | 79.35 |
+| RL only | 81.71 | 72.97 | 76.72 |
+| **PRIMO R1 (SFT+RL)** | **88.47** | **82.90** | **85.28** |
+
+RL without SFT underperforms because the model struggles to discover the output format from scratch; SFT alone generalizes poorly out of domain.
+
+Zero-shot failure detection on RoboFail (paper Table 3): **67.0%**, matching Gemini 2.0 Flash and above GPT-4o at 63.0.
+
+To reproduce these numbers, see [primo-bench-json](https://huggingface.co/datasets/LeonOverload/primo-bench-json), which documents the full evaluation path. One caveat when comparing against other work: the repo's harnesses score `regression` with three different formulas, and each output file records which one it used in its `regression_metric` field. The published numbers above come from `linear_relative_accuracy`.
 
 ## Citations