Sen-illion/Evofig-Judge-9B

EvoFig Judge 9B

This repository contains the trained EvoFig Judge LoRA adapter for scientific framework-figure evaluation. It is loaded on top of the vision-language base model Qwen/Qwen3.5-9B.

The adapter is one checkpoint used by both Judge seats in EvoFig:

• the advisory seat proposes bounded repairs;
• the acceptance seat decides whether a repaired figure is accepted or rolled back.

Both seats use the same weights, matching the configuration used for the reported runs.

Checkpoint

Field	Value
Adapter	choice9b-sft-aesthetic
Training step	3174
Epoch	0
LoRA rank	16
LoRA alpha	32
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training examples	873 multiple-choice comparisons (749 positive, 124 negative)
Reference key	sft-no-reference
Base model	Qwen/Qwen3.5-9B

An offline key check maps all 256 adapter tensors to parameters in the pinned Qwen3.5-9B index; see adapter_key_check.json.

Files

• adapter_model.safetensors - LoRA weights (about 111 MB).
• adapter_config.json - PEFT configuration.
• tokenizer.json, tokenizer_config.json, processor_config.json, chat_template.jinja - processor and tokenizer files copied from the base model at training time.
• training_state.json - training step and reference key.
• adapter_key_check.json - offline check that every adapter tensor maps to a Qwen3.5-9B base parameter.
• SHA256SUMS - checksums for the released files.
• LICENSE - Apache-2.0, following the Qwen3.5-9B base model license.

Usage

Install the inference stack:

pip install -U torch torchvision transformers peft accelerate bitsandbytes safetensors pillow

Load the base model and this adapter:

from pathlib import Path

import torch
from PIL import Image
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor, BitsAndBytesConfig

base_id = "Qwen/Qwen3.5-9B"
adapter_id = "your-namespace/Evofig-Judge-9B"  # the repository this adapter was loaded from

processor = AutoProcessor.from_pretrained(
    adapter_id,
    trust_remote_code=True,
)
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
base = AutoModelForImageTextToText.from_pretrained(
    base_id,
    quantization_config=quantization_config,
    device_map="auto",
    trust_remote_code=True,
)
judge = PeftModel.from_pretrained(base, adapter_id)
judge.eval()

figure = Image.open(Path("figure.png")).convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": figure},
            {
                "type": "text",
                "text": (
                    "Evaluate the scientific framework figure against the paper. "
                    "Return only the JSON judgement requested by the EvoFig prompt."
                ),
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = processor(text=[text], images=[figure], return_tensors="pt").to(judge.device)
with torch.inference_mode():
    generated = judge.generate(**inputs, max_new_tokens=256, do_sample=False)
answer = processor.batch_decode(
    generated[:, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)[0]
print(answer)

For the full EvoFig chain and the exact Judge prompts, use the EvoFig code release (link withheld for anonymous review). The code names the two seats and passes the same adapter to both.

Full-precision merge

This repository distributes the adapter rather than a 9B merged checkpoint. To produce full weights, load Qwen/Qwen3.5-9B and run PEFT's merge_and_unload() on a machine with enough host memory for the full model, then save the merged model with save_pretrained. The adapter alone is the released artifact and the reproducible starting point.

Intended use

The Judge scores scientific framework figures within the EvoFig pipeline. It checks whether a figure carries the paper's modules, stages, relations, terminology and evidence, and whether the figure is suitable for bounded repair.

Limitations

• This is not a general-purpose reward model or a general image-quality model.
• It was trained and evaluated on scientific framework figures, not on arbitrary photographs, charts or natural images.
• It should not be used as the sole authority for scientific correctness. Paper evidence, deterministic grounding checks and the EvoFig acceptance gates remain authoritative.
• The base model is not redistributed here. Users must obtain Qwen/Qwen3.5-9B separately and comply with its Apache-2.0 license.

Reproduction notes

The adapter was trained with PEFT 0.21.0, rank 16, alpha 32, on a Qwen3.5-9B vision-language backbone. The local reproduction used:

base:  Qwen/Qwen3.5-9B @ c202236235762e1c871ad0ccb60c8ee5ba337b9a
torch: 2.11.0+cu128
transformers / peft / bitsandbytes: see the environment lock written by the reproduction run

The original training environment is documented in the EvoFig release notes. If a future base revision changes the processor or architecture, pin the revision above instead of following main.

License

The adapter is released under Apache-2.0, following the license of Qwen/Qwen3.5-9B. EvoFig code is released separately under MIT.