RC1

RC1 is a pair of small, fast image recognition models that select the matching cells in reCAPTCHA v2 image grids. The 3×3 model classifies each of the nine cells separately, while the 4×4 model processes the whole grid at once. Both run locally on a CPU with ONNX Runtime. On my $2 VPS, the 3×3 model processes about 9 grids per second using two CPU threads.

Inference in this demo ran on my $2 VPS. The left window shows a 4×4 grid and the right window shows a 3×3 grid.

Models

Model	Backbone	Input	Supported targets
grid9.onnx	MobileNetV4 (mobilenetv4_conv_medium.e250_r384_in12k_ft_in1k)	Nine cells from a 3×3 grid	bicycle, bus, car, crosswalk, fire hydrant, motorcycle, parking meter, stairs, traffic light
grid16.onnx	ConvNeXt Tiny (convnext_tiny.dinov3_lvd1689m)	One 4×4 grid image	bicycle, bus, chimney, crosswalk, fire hydrant, motorcycle, parking meter, stairs, taxi, tractor, traffic light

Each model outputs a probability for every cell. The inference script then selects cells using the thresholds stored in the model file.

Both models were fine-tuned on the recaptcha-390k dataset.

Running inference

Requires Python 3.11 or newer. From this directory, install the dependencies and run the prediction script:

python -m pip install onnxruntime numpy pillow
python predict_onnx.py --model weights/grid16.onnx --image challenge.jpg --target buses

Use weights/grid9.onnx for 3×3 grids and weights/grid16.onnx for 4×4 grids.

The script prints JSON with a probability for each cell and a selected_cells list. Cells are numbered from 0, left to right and then top to bottom.

To limit the number of CPU threads, pass --threads. For all options, run python predict_onnx.py --help.

Benchmarks

Approximate CPU throughput once the model is loaded. Times include image loading and preprocessing.

Machine	CPU threads	3×3 grids/sec	4×4 grids/sec
Apple M2 Pro	4	~10	~7
$2 VPS	2	~9	~4

In a separate 3×3 test on the M2 Pro, RC1 ran about 10× faster than a ViT fine-tuned on reCAPTCHA. Both models were run as float32 ONNX models using four CPU threads.

Training and export

The models were trained on an AMD Ryzen Threadripper 9960X with an NVIDIA RTX PRO 6000.

Training requires Python 3.11, uv, and a local copy of the recaptcha-390k dataset. From this directory:

# Install dependencies
uv sync --locked

# Prepare the dataset
uv run python train.py prepare --data /path/to/recaptcha-390k --output runs/rc

# Train the 3×3 and 4×4 models
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 9
uv run python train.py train --manifest runs/rc/split.json --output runs/rc --tiles 16 --backbone convnext_tiny.dinov3_lvd1689m

# Export to ONNX
uv run --group export python train.py export --checkpoint runs/rc/grid9/best.pt

# Run tests
uv run --group export python -m unittest test_training

Notes:

• Training uses a GPU if one is available. On CUDA, add --amp to reduce memory usage.
• The prepare step keeps the dataset's existing splits and leaves out the excluded split.
• All training outputs are written locally to runs/.
• For dataset and training options, run python train.py --help or add --help to any subcommand.

Roadmap

• Improve chimney and parking meter recognition with more training data.
• Add training data and model support for bridges.

Disclaimer

RC1 was created for educational and research purposes. It is not affiliated with or endorsed by Google or reCAPTCHA. Users are responsible for complying with all applicable laws and terms of service.

License

RC1 is released under the MIT License. This license does not cover the source images in the training dataset.