Handwritten Panjabi Digit Recognition

Recognises one handwritten Gurmukhi digit (੦ ੧ ੨ ੩ ੪ ੫ ੬ ੭ ੮ ੯) in an image. It is the model behind the digit recognition on AI Panjab.

Try it without code: the Space: draw a digit and get the top three guesses. The Space also has a Use via API page (Python, JavaScript, curl).

Files

File	What it is
`model.onnx`	The model for ONNX Runtime (CPU is enough). Use this one.
`labels.json`	The ten labels, in the order of the model's outputs.
`inference.py`	The code to use it: picture in, top guesses out (below).
`requirements.txt`	What `inference.py` needs: `onnxruntime`, `numpy`, `pillow`.
`model.keras`	The same model for training, in Keras 3 (PyTorch backend): to fine-tune or continue training.
`training.py`	The code to train it: images in, a new model folder (`model.keras` + `model.onnx`) out.
`requirements-train.txt`	What training needs: adds `torch`, `keras`, `onnx`, `datasets`.
`config.json`	The model's description, and its training dataset.

Input and output

Input image: float32, shape (batch, 28, 28, 1), grayscale with values 0 to 255 (the model rescales them itself). Dark ink on a light background, one digit filling most of the picture.
Output: ten probabilities, one per label in labels.json.

To turn a picture into the input exactly as the model was trained:

a transparent background (RGBA) is put on white; any other image is converted to RGB;
grayscale with the ITU-R 601 weights: 0.2989 R + 0.5870 G + 0.1140 B, kept in floating point;
resized to 28 x 28 with bilinear (antialiased) resizing.

Use it

In two lines, with aipanjab/model-loader:

pip install git+https://huggingface.co/aipanjab/model-loader

from aipanjab import load

load("digit-recognition").predict("my-digit.png")    # {'੨': 0.99, '੩': 0.01, '੭': 0.0}

Or with this repo alone: it has everything, the model, its labels and inference.py, which only needs ONNX Runtime, NumPy and Pillow.

From the command line

pip install -U huggingface_hub
hf download aipanjab/digit-recognition --local-dir digit-recognition
pip install -r digit-recognition/requirements.txt
python digit-recognition/inference.py my-digit.png

my-digit.png: ੨ 0.99, ੩ 0.01, ੭ 0.00

From Python, inside the downloaded folder:

from inference import DigitRecognizer

recognizer = DigitRecognizer()                 # device="auto": an NVIDIA GPU if onnxruntime-gpu finds one, else the CPU
print(recognizer.predict("my-digit.png"))      # {'੨': 0.99, '੩': 0.01, '੭': 0.0}

predict takes a file path, an open file or a PIL image, and top_k (default 3). A picture with nothing drawn on it raises BlankImage. inference.py is short: read it to see the preprocessing, or copy the folder into your project.

Train it

Continue training this model on your own handwriting, or on this model's dataset:

pip install "aipanjab-loader[train] @ git+https://huggingface.co/aipanjab/model-loader"

from aipanjab import load

model = load("digit-recognition").train(data="my-digits", epochs=10)   # my-digits/੨/1.png, my-digits/੨/2.png, ...
model.predict("my-digit.png")
model.save("my-digit-model")                                          # later: load("my-digit-model")

data is a folder with one folder per digit (named ੦ … ੯), a Hugging Face dataset id with image and label columns, or a list of (image, label) pairs; without data, the model's own dataset. Other options: batch_size (32), lr (0.0001), test_size (0.1, the share of images kept to check the model when the data has no test split). The new model comes with both model.keras and model.onnx.

Without the loader, training.py does the same: pip install -r requirements-train.txt, then from training import train; train(".", data="my-digits", out="my-digit-model", epochs=10) inside the repo folder.

Training data

aipanjab/handwritten-digits: digits drawn by people on the AI Panjab platform (3,533 for training, 884 for testing), drawn with a black pen on a canvas of about 350 x 350. The model works best on pictures like these: one digit, dark on light, not too small in the picture.

Training procedure

Two convolutions (128 filters, 3 x 3, GELU then ReLU, each followed by 3 x 3 max pooling), dense layers of 128 and 64 (tanh) and a softmax over the 10 digits; Adam (learning rate 0.001), sparse categorical cross-entropy, 50 epochs, keeping the best epoch on the test split. Accuracy on the 884 test images: 98.6%. model.onnx and model.keras hold the same weights and give the same answers (outputs differ by less than 0.000002).

aipanjab
/

digit-recognition