File size: 4,215 Bytes
db208b4 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | ---
license: mit
library_name: onnx
datasets:
- bxding/recaptcha-390k
tags:
- computer-vision
- image-classification
- onnx
---
# RC1
RC1 is a pair of small, fast image recognition models that select the matching
cells in reCAPTCHA v2 image grids. The 3×3 model classifies each of the nine
cells separately, while the 4×4 model processes the whole grid at once. Both
run locally on a CPU with ONNX Runtime. On my $2 VPS, the 3×3 model processes
about **9 grids per second** using two CPU threads.
*Inference in this demo ran on my $2 VPS. The left window shows a 4×4 grid and
the right window shows a 3×3 grid.*
## Models
| Model | Backbone | Input | Supported targets |
| --- | --- | --- | --- |
| `grid9.onnx` | MobileNetV4 (`mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k`) | Nine cells from a 3×3 grid | bicycle, bus, car, crosswalk, fire hydrant, motorcycle, parking meter, stairs, traffic light |
| `grid16.onnx` | ConvNeXt Tiny (`convnext_tiny.dinov3_lvd1689m`) | One 4×4 grid image | bicycle, bus, chimney, crosswalk, fire hydrant, motorcycle, parking meter, stairs, taxi, tractor, traffic light |
Each model outputs a probability for every cell. The inference script then
selects cells using the thresholds stored in the model file.
Both models were fine-tuned on the
[recaptcha-390k dataset](https://huggingface.co/datasets/bxding/recaptcha-390k).
## Running inference
Requires Python 3.11 or newer. From this directory, install the dependencies
and run the prediction script:
```sh
python -m pip install onnxruntime numpy pillow
python predict_onnx.py --model weights/grid16.onnx --image challenge.jpg --target buses
```
Use `weights/grid9.onnx` for 3×3 grids and `weights/grid16.onnx` for 4×4 grids.
The script prints JSON with a probability for each cell and a `selected_cells`
list. Cells are numbered from 0, left to right and then top to bottom.
To limit the number of CPU threads, pass `--threads`. For all options, run
`python predict_onnx.py --help`.
## Benchmarks
Approximate CPU throughput once the model is loaded. Times include image
loading and preprocessing.
| Machine | CPU threads | 3×3 grids/sec | 4×4 grids/sec |
| --- | ---: | ---: | ---: |
| Apple M2 Pro | 4 | ~10 | ~7 |
| $2 VPS | 2 | ~9 | ~4 |
In a separate 3×3 test on the M2 Pro, RC1 ran about **10× faster** than a
[ViT fine-tuned on reCAPTCHA](https://huggingface.co/nobodyPerfecZ/vit-finetuned-patch16-224-recaptchav2-v1).
Both models were run as float32 ONNX models using four CPU threads.
## Training and export
The models were trained on an AMD Ryzen Threadripper 9960X with an NVIDIA RTX
PRO 6000.
Training requires Python 3.11, [uv](https://docs.astral.sh/uv/), and a local
copy of the recaptcha-390k dataset. From this directory:
```sh
# Install dependencies
uv sync --locked
# Prepare the dataset
uv run python train.py prepare --data /path/to/recaptcha-390k --output runs/rc
# Train the 3×3 and 4×4 models
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 9
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 16 --backbone convnext_tiny.dinov3_lvd1689m
# Export to ONNX
uv run --group export python train.py export --checkpoint runs/rc/grid9/best.pt
# Run tests
uv run --group export python -m unittest test_training
```
Notes:
- Training uses a GPU if one is available. On CUDA, add `--amp` to reduce
memory usage.
- The `prepare` step keeps the dataset's existing splits and leaves out the
`excluded` split.
- All training outputs are written locally to `runs/`.
- For dataset and training options, run `python train.py --help` or add
`--help` to any subcommand.
## Roadmap
- Improve chimney and parking meter recognition with more training data.
- Add training data and model support for bridges.
## Disclaimer
RC1 was created for educational and research purposes. It is not affiliated
with or endorsed by Google or reCAPTCHA. Users are responsible for complying
with all applicable laws and terms of service.
## License
RC1 is released under the [MIT License](LICENSE). This license does not cover
the source images in the training dataset.
|