asermnasr commited on
Commit
66c872e
·
verified ·
1 Parent(s): 5aa3d92

docs: update training datasets in model card

Browse files
Files changed (1) hide show
  1. README.md +16 -3
README.md CHANGED
@@ -6,6 +6,8 @@ base_model:
6
  - microsoft/Florence-2-base-ft
7
  datasets:
8
  - inclusionAI/VenusBench-CAPTCHA
 
 
9
  pipeline_tag: image-text-to-text
10
  library_name: onnxruntime
11
  model-index:
@@ -95,12 +97,23 @@ Data used for model training and how the data was processed.
95
 
96
  ### Training Dataset
97
 
98
- Fiptcha was fine-tuned on CAPTCHA challenge pairs derived from [`inclusionAI/VenusBench-CAPTCHA`](https://huggingface.co/datasets/inclusionAI/VenusBench-CAPTCHA), covering `<OCR>` and `<CAPTION_TO_PHRASE_GROUNDING>` across `drag_end`, `slider_puzzle`, `icon_click`, `text_click`, `image_rotation`, and `visual_reasoning` tasks.
 
 
 
 
 
 
 
 
 
 
 
99
 
100
  ### Data Preprocessing
101
 
102
- * **Image Preprocessing:** Input images are resized to `768x768` (`resample=3`) and normalized with standard ImageNet/CLIP mean (`[0.485, 0.456, 0.406]`) and std (`[0.229, 0.224, 0.225]`), producing `577` visual tokens.
103
- * **Prompt Formatting:** Each example is formatted as `(image, prefix, suffix)` pairs in JSONL (`train.jsonl` / `val.jsonl`).
104
 
105
  ---
106
 
 
6
  - microsoft/Florence-2-base-ft
7
  datasets:
8
  - inclusionAI/VenusBench-CAPTCHA
9
+ - OpenCaptchaWorld/Open_CaptchaWorld
10
+ - hammer888/captcha-data
11
  pipeline_tag: image-text-to-text
12
  library_name: onnxruntime
13
  model-index:
 
97
 
98
  ### Training Dataset
99
 
100
+ Fiptcha was fine-tuned on a combined mixture of three public CAPTCHA datasets (augmented with repetition factors and split into `train.jsonl` and `val.jsonl`), plus an optional local click-target set:
101
+
102
+ * **[`inclusionAI/VenusBench-CAPTCHA`](https://huggingface.co/datasets/inclusionAI/VenusBench-CAPTCHA)** (`test` split, upsampled `15x`):
103
+ * **OCR CAPTCHAs:** Text-based challenges parsed from `action_raw` and annotated with `<OCR>` prompts.
104
+ * **Visual / Grounding CAPTCHAs:** Click and interaction target boxes annotated with `<CAPTION_TO_PHRASE_GROUNDING>{prompt}` to predict localized target boxes (`<loc_x1><loc_y1><loc_x2><loc_y2>`).
105
+ * **[`OpenCaptchaWorld/Open_CaptchaWorld`](https://huggingface.co/datasets/OpenCaptchaWorld/Open_CaptchaWorld)** (`train` split, upsampled `10x`):
106
+ * **Point-Click / Multi-Point CAPTCHAs:** Point coordinates `(x, y)` mapped into padded bounding boxes for visual localization prompts.
107
+ * **VQA / Question-Answering CAPTCHAs:** General alphanumeric text and label verification questions mapped to a `<VQA>` prefix and target answers.
108
+ * **[`hammer888/captcha-data`](https://huggingface.co/datasets/hammer888/captcha-data)** (`40,000` samples):
109
+ * Distorted text-based images representing classic visual OCR CAPTCHAs, mapped to the `<OCR>` task prefix.
110
+ * **Optional CAPTCHA-X Manifest** (upsampled `15x`, when present):
111
+ * A specialized local click-target dataset (`raw_captchax/manifest.jsonl`).
112
 
113
  ### Data Preprocessing
114
 
115
+ * **Image Preprocessing:** Collected images were cached and normalized into `train.jsonl` / `val.jsonl` catalogs, resized to `768x768` (`resample=3`), and normalized with standard ImageNet/CLIP mean (`[0.485, 0.456, 0.406]`) and std (`[0.229, 0.224, 0.225]`), producing `577` visual tokens.
116
+ * **Prompt Formatting:** Each example is formatted as `(image, prefix, suffix)` pairs across `<OCR>`, `<VQA>`, and `<CAPTION_TO_PHRASE_GROUNDING>` task prefixes.
117
 
118
  ---
119