How to use from
Docker Model Runner
docker model run hf.co/andrewxhill/camera-trap-edit
Quick Links

Camera-trap direct weight edit for LLaVA-OneVision 0.5B

Frontier AI models edited the weights of LLaVA-OneVision 0.5B, a free vision-language model, and raised its camera-trap balanced accuracy from 64.09% to 81.76% (+17.67 points, 95% interval 12.41 to 23.61). The test is 1,611 animal crops from 45 cameras, each with its own list of animal names; balanced accuracy counts each camera, and each animal within it, equally. There was no training run: the models changed the weights (the numbers the model is made of) by calculation. The unedited model kept answering in sentences ("B. The image shows a fox…"), so they set the output rows (the numbers that score each possible next token) for "The", with and without a leading space, to zero and scaled the end-of-answer row by 1.5. It now stops after the answer. They also wired an animal-name hint that depends on the image into the projector, the small network between the image encoder and the language model, and corrected the rows for the answer letters A to K.

Explainer · Paper · Code, data and results

Results

Model Mockingbird vs brown noddy Land iguana vs green iguana Marine iguana vs green iguana Invasive rats found Camera traps
Island edit 98.33% 81.22% 80.37% 81.23% 72.14%
One model for both jobs (LoRA) 93.33% 87.58% 91.69% 88.20% 79.22%
One model for both jobs (direct edit) 90.83% 55.49% 55.49% 17.70% 80.70%
Camera-trap edit 91.67% 51.83% 51.83% 13.75% 81.76%
Unedited LLaVA-OneVision 0.5B 70.83% 50.00% 50.00% 2.96% 64.09%
SpeciesNet (standard output) 3.33% 12.44% 12.44% 8.11% —

Island scores come from 697 test crops from places not used to build any model. Each native-versus-look-alike score averages two shares, natives named as natives and look-alikes named as look-alikes, with each photo location counting equally; rat recall also weights locations equally. The camera-trap column is balanced accuracy on 1,611 crops from 45 cameras, each with its own list of names, with every camera and every animal within it counted equally. SpeciesNet has no label for the natives, and its standard output rarely names the look-alikes, so it scores below 50%, the score of guessing. SpeciesNet was scored on the island test only.

How it was made

Claude Fable 5.2, Claude Opus 5.5 and GPT-6 Astra Pro took turns changing the weights of LLaVA-OneVision Qwen2 0.5B with numerical programs they wrote, with no gradient training. They saved 39 numbered checkpoints. Study software chose among them using 204 held-back tuning crops from LILA, a public library of camera-trap images, whose results the editors never saw. It chose the 39th, which the paper calls check 39. The edit writes scores for the 58 animal names on the camera lists into the projector as name hints at two image-token positions, zeroes the output rows for "The", strengthens the end-of-answer row, and corrects the answer-letter rows A to K. This model is also the starting point for both one-model-for-both-jobs models.

How to use it

import torch
from PIL import Image
from transformers import AutoProcessor, LlavaOnevisionForConditionalGeneration

repo = "andrewxhill/camera-trap-edit"
processor = AutoProcessor.from_pretrained(repo)
model = LlavaOnevisionForConditionalGeneration.from_pretrained(repo, dtype=torch.float32).eval()

names = ["bobcat", "coyote", "deer", "fox", "rabbit"]   # the animals this camera can see
question = ("Which animal is shown in this crop?\n"
            + "\n".join(f"{chr(65 + i)}. {n}" for i, n in enumerate(names))
            + "\nAnswer with the animal's name from the choices, and no other text.")
messages = [{"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": [{"type": "image", "image": Image.open("crop.jpg").convert("RGB")},
                                         {"type": "text", "text": question}]}]
inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,
                                       return_dict=True, return_tensors="pt")
out = model.generate(**inputs, do_sample=False, max_new_tokens=1024, repetition_penalty=1.1)
print(processor.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Give it a crop around one animal and the list of animals that camera can see. The reply is often the letter alone; code/eval/parser.py maps letters and names back to the list. The benchmark used 2 to 12 names per camera; the edit corrects the answer letters A to K directly. The scores above used float32, greedy decoding, 1,024 maximum new tokens and repetition penalty 1.1. The camera benchmark crops each detector box with 25% padding; code/eval/ has the exact crop, prompt and parser.

Intended use

Naming animals in cropped camera-trap frames from a short per-camera list. On the island tests it scores 51.83% on the iguana contrasts and 91.67% on mockingbird versus brown noddy; use the island edit or the LoRA model for Galápagos animals.

Rebuild from the public base

The delta/ folder rebuilds these weights exactly from lmms-lab/llava-onevision-qwen2-0.5b-ov at revision 381d9947148efb1e58a577f451c05705ceec666e. The delta replaces three projector tensors and 15 rows of the output head (7.4 MB). code/recipe/ in the GitHub repository also recomputes the edit from public practice crops; that build reproduced every test answer, with three tensors differing slightly.

git clone https://github.com/andrewxhill/frontier-model-weight-edits && cd frontier-model-weight-edits/code
pip install -r requirements.txt
python apply_edit.py --base lmms-lab/llava-onevision-qwen2-0.5b-ov \
  --revision 381d9947148efb1e58a577f451c05705ceec666e \
  --delta path/to/delta/delta.safetensors --manifest path/to/delta/delta_manifest.json --output rebuilt

The script checks the base and every replaced tensor against SHA256 hashes and confirms the finished state.

Licence and citation

Apache-2.0, as for the base model and its Qwen2 and SigLIP components; see LICENSE and NOTICE.md. Please cite:

Andrew Hill (2026). Can weight-space programming surface the domain expertise hidden in a small vision-language model? Evidence from camera traps and Galápagos species https://andrewxhill.com/frontier-model-weight-edits/paper.pdf

Downloads last month
-
Safetensors
Model size
0.9B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for andrewxhill/camera-trap-edit

Finetuned
(16)
this model