Card: columns named by model, plain boards only (no element detector), Sunlight 11-13 and Abyss added
ab9dd78 verified |
Download README.md from CaptchaKraken/AbyssV2: direct link, hf CLI and curl.
- Browser
- Download file 7.89 kB
-
https://huggingface.co/CaptchaKraken/AbyssV2/resolve/main/README.md
- Command line
-
hf download hf://CaptchaKraken/AbyssV2/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/CaptchaKraken/AbyssV2/resolve/main/README.md
7.89 kB
| base_model: Qwen/Qwen3.5-4B | |
| license: lgpl-3.0 | |
| tags: [captcha, captchakraken, pointer, vllm] | |
| # CaptchaKraken v1.2 Abyss: AbyssV2-4B (merged bf16) | |
| **Private.** Abyss is the model the hosted CaptchaKraken API answers with. These repos hold | |
| exactly the checkpoint it serves (in production since 2026-11-12). | |
| | Repo | What | | |
| |---|---| | |
| | [`CaptchaKraken/AbyssV2-4B`](https://huggingface.co/CaptchaKraken/AbyssV2-4B) | merged bf16 weights, for vLLM (GPU) — **this repo** | | |
| | [`CaptchaKraken/AbyssV2-4B-q4`](https://huggingface.co/CaptchaKraken/AbyssV2-4B-q4) | 4-bit GPTQ (W4A16) weights, for vLLM (GPU): the build the hosted API serves | | |
| | [`CaptchaKraken/AbyssV2-4B-LoRA`](https://huggingface.co/CaptchaKraken/AbyssV2-4B-LoRA) | unmerged LoRA adapter + pointer heads, for further training | | |
| The public v2 pointer families are Sunlight V2 (0.8B), Twilight V2 (2B) and Midnight V2 (4B). | |
| ## What this is | |
| The generation-3 **pointer** model: Qwen/Qwen3.5-4B, LoRA-trained (rank 16) with small heads | |
| (`pointer_heads.safetensors`) that read the whole answer (action, points, drags, text, video | |
| moment) from **one prefill** of the prompt. There is no text generation. vLLM runs it as a | |
| pooling model through the `CaptchaPointerForConditionalGeneration` plugin. | |
| - Training run `20261111-qwen-4b-fc-focus`: continued from the 2026-11-09 4B run | |
| (`20261109-qwen-4b-data1108-plain`) on 27,131 rows for one epoch, lr 1e-4 (heads 1e-3), | |
| accum 8, plain boards only. The data was half every other captcha kind (replayed from the | |
| previous run's data) and half FunCaptcha's pager games, split evenly between arrow screens | |
| and Submit screens. Trained on one Radeon RX 7900 XTX. | |
| - The q4 build is GPTQ W4A16 (block 128, static act-order); the vision tower, embeddings, | |
| `lm_head` and pointer heads stay in bf16 (`recipe.yaml`). | |
| ## Loading it | |
| These heads match client **4.2.0** (`pip install captchakraken`), 513-wide packed output. | |
| On vLLM 0.25.1 set `VLLM_ENABLE_V1_MULTIPROCESSING=0`, or the engine process will report | |
| the architecture as unsupported. | |
| ## End-to-end solve rate (Tier 3) | |
| Tier 3 is the whole product working: the CaptchaKraken client opens a real browser, finds the captcha on the page, asks this model, then clicks, drags, types or holds like a person, and the page's own answer check decides whether it was solved. It covers **149 captcha kinds from 18 vendors**, each rebuilt as a local page from real captures, driven through both the Python and the JavaScript client (298 kind × client pairs). A pair counts as solved when **2 attempts succeed, out of at most 3**, each on a freshly dealt board. Mobile repeats 14 kinds on an emulated Pixel 7 with touch input. The gauntlet deals boards that cannot be solved mixed with ones that can, and passes only if the client works through the impossible ones without stalling and solves the rest. | |
| | Pairs solved | Sunlight | Twilight | Midnight | **Abyss (hosted API)** | | |
| |---|---|---|---|---| | |
| | **All kinds** | **73% (219/298)** | **79% (236/298)** | **84% (251/298)** | **88% (263/298)** | | |
| | Mobile (Pixel 7) | 54% (15/28) | 86% (24/28) | 79% (22/28) | 86% (24/28) | | |
| | Gauntlet | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| **By vendor** (pairs; each kind counts twice, once per client): | |
| | Vendor | Sunlight | Twilight | Midnight | **Abyss (hosted API)** | | |
| |---|---|---|---|---| | |
| | hCaptcha | 81% (158/194) | 86% (166/194) | 92% (178/194) | 93% (180/194) | | |
| | Dingxiang | 60% (12/20) | 70% (14/20) | 70% (14/20) | 85% (17/20) | | |
| | GeeTest | 62% (10/16) | 75% (12/16) | 75% (12/16) | 88% (14/16) | | |
| | FunCaptcha (Arkose) | 14% (2/14) | 14% (2/14) | 14% (2/14) | 36% (5/14) | | |
| | NetEase Yidun | 29% (4/14) | 57% (8/14) | 64% (9/14) | 79% (11/14) | | |
| | reCAPTCHA | 50% (4/8) | 75% (6/8) | 75% (6/8) | 75% (6/8) | | |
| | BotDetect | 100% (6/6) | 100% (6/6) | 100% (6/6) | 100% (6/6) | | |
| | Tencent | 100% (6/6) | 100% (6/6) | 100% (6/6) | 100% (6/6) | | |
| | AWS WAF | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | CaptchaFox | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | HUMAN Security | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | KCAPTCHA | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | Lemin | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | MTCaptcha | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | Press-and-hold bars | 100% (2/2) | 0% (0/2) | 100% (2/2) | 100% (2/2) | | |
| | Prosopo | 50% (1/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | Proton | 100% (2/2) | 100% (2/2) | 100% (2/2) | 100% (2/2) | | |
| | Yandex SmartCaptcha | 0% (0/2) | 0% (0/2) | 0% (0/2) | 0% (0/2) | | |
| **By what the captcha asks for:** | |
| | Kind | Sunlight | Twilight | Midnight | **Abyss (hosted API)** | | |
| |---|---|---|---|---| | |
| | Click a still board | 66% (101/154) | 81% (125/154) | 81% (125/154) | 88% (136/154) | | |
| | Click an animated board | 79% (22/28) | 68% (19/28) | 86% (24/28) | 79% (22/28) | | |
| | Drag | 83% (68/82) | 80% (66/82) | 90% (74/82) | 93% (76/82) | | |
| | Slider | 75% (12/16) | 75% (12/16) | 75% (12/16) | 81% (13/16) | | |
| | Type the text | 83% (10/12) | 83% (10/12) | 83% (10/12) | 83% (10/12) | | |
| | Press and hold | 100% (6/6) | 67% (4/6) | 100% (6/6) | 100% (6/6) | | |
| Measured with the client that became release 4.2.0: Twilight, Midnight and Abyss on 2026-11-12, Sunlight on 2026-11-13 in bf16 through vLLM 0.25.1 on an RTX 3060. Twilight was served in bf16 through transformers, Midnight and Abyss as 4-bit GPTQ through vLLM 0.25.1 (Abyss's is the build the hosted API runs). Abyss is the hosted model (CaptchaKraken v1.2 Abyss), whose weights are not public. With 2–3 attempts per kind, a single kind can flip between runs, so read small vendors as rough. Known weak spots for every model: FunCaptcha's matching games, and Yandex, whose two words are bent around a curve (the larger models read the first word and lose the second). | |
| ## Static image annotation solving (Tier 2) | |
| Static image annotation solving is scored against **real human solves**. Every board is a real captcha captured from a vendor's live widget and solved by hand, and the points, drag ends, grid cells and text that person chose are the answer key. Each answer is scored by **how close it lands to the human solve points**: a click counts when every target is pressed once, each within 6% of the image diagonal of the human's point; a drag when it is dropped within 5%; a grid when it picks the same cells; typed text when it matches; and on an animated board the moment must be right too. No browser or vendor is involved: this measures how well the model reads one board, not the end-to-end solve (see Tier 3 above). | |
| Solved rate on the Tier 2 gate set as of 2026-11-12 (1,074 hand-labelled real boards), merged | |
| bf16 through transformers, on the bare board (what the hosted API sends): | |
| | | solved | | |
| |---|---| | |
| | **All (1,074)** | **79%** | | |
| | Click (681) | 84 | | |
| | Grid (223) | 81 | | |
| | Video (158) | 58 | | |
| | Type (12) | 42 | | |
| The public cards' family table uses 1,067 boards served through vLLM, so these numbers are | |
| close to it but not directly comparable (MidnightV2-4B: 75.4% plain there). | |
| Per-board results are in `eval/eval_gate.jsonl`, the run's validation split in `eval/val.json`. | |
| ## FunCaptcha pager games (held out) | |
| FunCaptcha shows one screen at a time: the model must press the arrow until the right | |
| picture is up, then Submit. Scored per screen on held-out real game screens, served as the | |
| q4 build through vLLM: | |
| | Game | Submit right | Arrow right | | |
| |---|---|---| | |
| | claw prizes | 30/38 | 54/62 | | |
| | dice icons | 28/33 | 43/47 | | |
| | goal icons | 41/41 | 58/59 | | |
| | maze objects | 31/41 | 27/59 | | |
| | orbit icons | 35/41 | 52/59 | | |
| | seat numbers | 28/33 | 59/67 | | |
| | standing icons | 32/41 | 37/59 | | |
| | **All** | **225/268 (84%)** | **330/412 (80%)** | | |
| A game is only solved when every screen is right, which is why FunCaptcha still scores low | |
| end to end (Tier 3 above) despite these per-screen numbers. | |
| License: LGPL-3.0. | |