CaptchaKraken SunlightV2-0.8B-LoRA (LoRA adapter + heads)

The unmerged training output of CaptchaKraken/SunlightV2-0.8B: the rank-16 LoRA adapter for Qwen/Qwen3.5-0.8B and the pointer heads. Most people want the merged CaptchaKraken/SunlightV2-0.8B (ready for vLLM) or CaptchaKraken/SunlightV2-0.8B-GGUF (CPU, llama.cpp). This repo is for continuing training or merging onto a different base build.

The Sunlight V2 family (0.8B)

Sunlight V2 is the 0.8B CaptchaKraken v2 pointer model. Every repo below holds the same trained model:

Repo	What
CaptchaKraken/SunlightV2-0.8B	merged bf16 weights, for vLLM (GPU)
CaptchaKraken/SunlightV2-0.8B-q4	4-bit GPTQ (W4A16) weights, for vLLM (GPU)
CaptchaKraken/SunlightV2-0.8B-LoRA	unmerged LoRA adapter + pointer heads, for further training — this repo
CaptchaKraken/SunlightV2-0.8B-GGUF	llama.cpp builds (Q4_K_M, BF16, mmproj), for CPU

The other v2 pointer families: Twilight V2 (2B, CaptchaKraken/TwilightV2-2B), Midnight V2 (4B, CaptchaKraken/MidnightV2-4B). The numbered element marks come from the v2 CNN element detector, CaptchaKraken/CaptchaKrakenV2-CNN-large (CPU-lite build: CaptchaKraken/CaptchaKrakenV2-CNN-lite).

What this is

The generation-3 pointer model: Qwen/Qwen3.5-0.8B with LoRA (rank 16, alpha 32, on the language model's attention, linear-attention and MLP projections and the vision tower's qkv/proj/linear_fc*) and small heads that read the whole answer (action, points, drags, text, video moment) from one prefill of the prompt. There is no text generation.

• Training run 20261030-100636-qwen-0.8b: 43,963 boards, 1 epoch, lr 1e-4 (heads 1e-3), accum 8, half the boards drawn with numbered element marks. CaptchaKraken/SunlightV2-0.8B was exported from this run (its config.json names it under pointer.run).
• Prompt: the fixed single prompt ("every captcha is asked the same prompt").

Files

• adapter_model.safetensors, adapter_config.json: the PEFT adapter. base_model_name_or_path is set to Qwen/Qwen3.5-0.8B (training used a local snapshot of it, revision 2fc0636).
• pointer_heads.safetensors: the heads, keys pointer_heads.<name>, float32. Identical to the run's heads.pt["heads"] and to the file in CaptchaKraken/SunlightV2-0.8B.
• training_config.json: the training run's own config.json (renamed so loaders don't mistake it for a model config).
• client-fixes.patch: the same patch as in CaptchaKraken/SunlightV2-0.8B; see below.

To rebuild the merged model, merge the adapter onto Qwen/Qwen3.5-0.8B (PeftModel.from_pretrained(...).merge_and_unload()), save it, copy pointer_heads.safetensors beside it, and add the pointer block and the CaptchaPointerForConditionalGeneration architecture to config.json exactly as CaptchaKraken/SunlightV2-0.8B's config.json has them.

Loading it (read this first)

These heads use an older head layout than the 2B and 4B (PACKED = 165, against 225 for TwilightV2-2B and MidnightV2-4B): there is no element-number head. They match the client the CaptchaKraken/SunlightV2-0.8B card names, commit ef2ec28 of CaptchaKraken, exactly. A later client's heads (the offset/element head changes of 2026-10-30) will refuse them. Apply client-fixes.patch (in this repo) on top of ef2ec28 before serving with vLLM 0.25.1; it fixes the two serving bugs described on that card (full video markers; prefix caching off).

Gate results

See CaptchaKraken/SunlightV2-0.8B (Tier 2 gate set, 1,067 hand-labelled real boards, served through vLLM 0.25.1): bf16 53.5% plain / 51.3% marked; the GPTQ build 53.9% / 50.8%. Per-board results are in that repo under eval/.

Results across the family

This adapter is the same trained model as the bf16 0.8B column. The bold heading marks this size.

Every v2 pointer size, on the same 1,067 hand-labelled real boards, served through vLLM 0.25.1 on an RTX 5080. Percent of boards solved; higher is better. q4 is 4-bit GPTQ (W4A16); the others are bf16.

Plain (the bare board):

	0.8B	0.8B q4	2B	4B	4B q4
All (1,067)	53.5%	53.9%	72.6%	75.4%	75.1%
Click (674)	60.4	61.6	76.3	78.3	77.9
Grid (223)	31.8	29.6	75.8	78.9	78.9
Video (158)	56.3	57.0	54.4	59.5	59.5
Type (12)	33.3	33.3	50.0	50.0	50.0

Marked (the CNN detector's numbered marks drawn on the board), stills only, on the boards all five runs marked:

	0.8B	0.8B q4	2B	4B	4B q4
All (904)	50.2%	49.8%	70.5%	71.6%	71.0%
Click (673)	58.5	58.8	73.3	73.7	73.6
Grid (223)	26.0	22.9	62.3	65.5	63.7
Type (8)	25.0	37.5	62.5	62.5	62.5

• 2B solves about 19 points more than 0.8B on the plain boards, split about evenly between clicks and grids. Grids are the sharpest jump (31.8 to 75.8). 4B adds about 3 more.
• Video barely follows size: 0.8B is ahead of 2B there, and 4B leads by about 3 points.
• 4-bit moves the totals by less than a point. Its largest cost is on marked grids (2 to 3 points). A 2B q4 has not been measured or published.

How far to trust the gaps:

• The 0.8B was trained on an earlier build of the data (2026-10-30), so not all of its gap is model size.
• Compare sizes on the plain table. Marked scores also depend on the detector, and the 4B card names a different detector checkpoint from the 0.8B and 2B cards.
• The 4B run did not score marked video, so marked video is left out for every size. That is why these marked totals differ from the ones quoted elsewhere on the cards, which cover different board sets.
• Type has only 12 boards plain and 8 marked, so treat those rows as rough.
• MidnightV2-4B's own gate results were run through transformers, so use this table, not those, to compare it with the other sizes.

Per-board results: 0.8B, 0.8B q4, 2B, 4B, 4B q4.

License: LGPL-3.0.