Joe0.1-27B

TL;DR

  • What it is: a 27B vision-language model that answers multiple-choice questions about images. It returns a probability for every option from one forward pass (no text generation). Built for the vision board of the Jev Decision Index.
  • Base model: Qwen/Qwen3.8-27B (Apache-2.0), revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, read in non-thinking mode. 27B parameters, bf16, 12 safetensors shards (about 54 GB).
  • Score: public vision index 76.64 on the Decision Index kit (all 12,210 public rows, our own run on 1 x H200).
  • Licence: research and noncommercial use only (see "Licence").

How to run it

The inference code ships in this repo: jev_vision.py holds the model, prompt and readout, and jev_vision_engine.py is the Decision Index kit engine.

pip install -r requirements.txt
from PIL import Image
from jev_vision import JevVision

m = JevVision("vantu-fit/Joe0.1-27B")   # or a local dir; about 54 GB of bf16 weights on one GPU
probs = m.decide([Image.open("photo.jpg")], "Which object is closer to the camera?", ["the chair", "the lamp"])

Decision Index kit, vision suite. This is the command of the submitted run, one request at a time:

git clone https://github.com/apolinario/decision-index && git -C decision-index checkout f5a6e4f
pip install --no-deps -e decision-index && pip install httpx pyarrow
decision-index suite rebuild --vision --work work --font "<Arial Bold .ttf>" && decision-index vision verify --dir work/vision
PYTHONPATH=<this repo dir> decision-index vision pipeline --dir work/vision \
    --engine jev_vision_engine:JevVisionEngine --option model=<this repo dir or id> --workers 1 --out runs/Joe0.1-27B

The run's status.json / environment.json record the engine as jev_base_lora_engine:JevVisionEngine, a thin module in our research repository that only re-exports jev_vision_engine:JevVisionEngine; the SHA-256 of jev_vision.py and jev_vision_engine.py in environment.json equal the files in this repo, so a run with jev_vision_engine:JevVisionEngine executes the same code.

What one request does

  1. Prompt. One fixed rendering for every question: the model's chat template with thinking turned off, the images, the question, the options as A. ..., B. ... lines, and a fixed start of the answer:
    <|im_start|>user\n{<|vision_start|><|image_pad|><|vision_end|> per image}{question}\nA. {option}\nB. {option}\n...
    Answer with the option's letter from the given choices directly.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nThe answer is (
    
  2. Images. Converted to RGB. An image over 1,638,400 pixels (1.6 MP) is downscaled to fit, keeping the aspect ratio (LANCZOS). Smaller images are used at their native size.
  3. Attention ("noncausal reading"). The softmax-attention layers see the whole prompt in both directions (a padding mask only); the Gated DeltaNet layers are unchanged. base_lora.json switches this on.
  4. Readout. The last hidden state is scored against the single-token letters A..Z (in fp32), and a softmax over the first n letters (the shown options) gives the probabilities. No sampling, no temperature.

Declared limits

  • Task: choice only (multiple choice with lettered options). Free-form answers are not supported.
  • Options: 2 to 26 per question. No truncation.
  • Images: each image is capped at 1.6 MP (1,638,400 px). The kit suite uses up to 2 images per row; more images work as long as they fit in GPU memory.

Evaluation

All numbers are from our own machines. The board's maintainers run the private half and measure latency themselves.

Decision Index kit, public vision suite. Kit commit f5a6e4fd, edition 0.3, engine jev_vision_engine:JevVisionEngine, one request at a time, 1 x H200. The run is complete: 12,210 of 12,210 rows ok, vision verify passed, 0 missing rows. Public vision index: 76.64. Calibration on the public suite (ECE over 10 confidence bins): 0.025.

Benchmark Rows Accuracy Skill Weight
CV-Bench 2,038 88.37 80.82 1
BLINK 961 83.45 74.39 1
RealWorldQA 589 82.51 71.42 1
CharXiv 532 86.65 82.21 1
InfographicVQA 244 99.18 98.91 1
Mind2Web 973 93.53 91.37 1
Winoground 800 93.13 86.25 1
KIE (CORD+FUNSD) 2,465 94.77 91.38 1
Moderation (Hateful Memes) 1,213 71.39 42.79 1
R-Bench-M 665 43.76 32.50 0.5
MMMU-Pro vision 1,730 52.66 45.70 0.25
Public vision index 12,210 76.64

Skill = (accuracy โˆ’ chance) / (1 โˆ’ chance) x 100: 0 is guessing, 100 is perfect. The index is the board's weighted mean of the skills, clipped to [0, 100]. Results files: vantu-fit/Joe0.1-27B-results.

Latency (kit total_wall_ms, one request at a time, 1 x H200, 8 CPU cores, 12,210 rows): median 168 ms, mean 217 ms, p80 337 ms, p95 449 ms, max 9.9 s. Two-image rows (BLINK): median 467 ms, p80 548 ms.

base_lora.json and the results' environment.json keep absolute paths from our training cluster; they are informational and not read at inference (the engine loads weights from this repo).

Limitations

  • Multiple choice only. The model reads one letter after The answer is (. Free-form generation was not trained or evaluated, and noncausal reading is defined over a complete prompt, not for token-by-token decoding.
  • Weak benchmarks: Moderation (Hateful Memes) skill 42.8; R-Bench-M 32.5; MMMU-Pro vision 45.7.
  • Numerical noise. Changing the weights by one bf16 unit in the last place changed about 100 of the 12,210 kit answers and the index by about 0.1 in our tests, so expect this much variation across GPUs and kernels.
  • Unmeasured: the board's private half; latency on the board's RTX PRO 6000; GPU peak memory.
  • Image cap. Images above 1.6 MP are downscaled.
  • Not for safety-critical decisions. Moderation and other judgements are benchmark-level only.

Disclosures

  • Benchmark images in training data. 138 training items use images that match BLINK (val / test) and CV-Bench images; 17 of these images are in the kit's public vision suite (14 CV-Bench, 3 BLINK val). The training questions on them are our own, not the benchmarks'. Some of the BLINK test images may also be in the board's private BLINK set.
  • Benchmark training split. The training data includes rows from the Mind2Web train split; the board uses the test_website split, which has different websites.

Licence

  • Base weights: Qwen3.8-27B is Apache-2.0; its licence text is included as LICENSE.base-Qwen3.8-27B.
  • Fine-tuned weights: released for research and noncommercial use only, as is. Some training data carries noncommercial or share-alike terms, and LLM-generated data whose providers' terms may restrict training competing models was used. See LICENSE_NOTICE.md. This is not legal advice.
Downloads last month
77
Safetensors
Model size
27B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for vantu-fit/Joe0.1-27B

Base model

Qwen/Qwen3.8-27B
Finetuned
(927)
this model
Quantizations
1 model