README.md

---
license: cc-by-4.0
language:
  - pa
tags:
  - gurmukhi
  - panjabi
  - punjabi
  - aipanjab
---

# model-loader: AI Panjab models in one line

Load any [AI Panjab](https://huggingface.co/aipanjab) model and run it on your own machine, in two lines of Python.

```bash
pip install git+https://huggingface.co/aipanjab/model-loader
```

```python
from aipanjab import load

load("digit-recognition").predict("digit.png")                       # {'੨': 0.99, '੩': 0.01, '੭': 0.0}
load("alphabet-recognition").predict("letter.png")                   # {'ਕ': 0.98, ...}
load("spell-checker-base").check("ਮੈਂ ਸਕੂਲ ਜਾਂਦਾ ਹਾ").corrected_text   # 'ਮੈਂ ਸਕੂਲ ਜਾਂਦਾ ਹਾਂ'
```

`load()` downloads only the model you ask for, the first time (it is cached after that, and works offline), and runs
it on your CPU. This package holds no model: every model and its code live in their own repo, so `load()` always
gets the current one. Several models work side by side in one program.

## Models

| `load(...)` | Use | Returns |
| :-- | :-- | :-- |
| `"digit-recognition"` | handwritten Gurmukhi digits ੦–੯ | `.predict(image, top_k=3)` → `{label: probability}` |
| `"alphabet-recognition"` | handwritten Gurmukhi letters and signs (57) | `.predict(image, top_k=3)` → `{label: probability}` |
| `"spell-checker-base"` | Panjabi spelling correction | `.check(text)` → `.corrected_text`, `.corrections`, `.to_dict()` |

Every model also has `.train(...)` and `.save(path)` (below).

`image` is a file path, an open file or a PIL image. Each model's card on Hugging Face has the details.

- A version of a model: `load("digit-recognition", revision="v1")` (a tag, branch or commit).
- A private model: log in first with `hf auth login`.
- A folder: `load("path/to/folder")` loads a model you saved, or a downloaded repo.
- On an NVIDIA GPU: install `onnxruntime-gpu` (instead of `onnxruntime`) and models use the GPU by themselves;
  `load(..., device="cpu")` or `device="cuda"` chooses. Without a GPU everything runs on the CPU.

## Train

Every model can be trained further on your own data, on your machine. Install the training extra once:

```bash
pip install "aipanjab-loader[train] @ git+https://huggingface.co/aipanjab/model-loader"
```

```python
from aipanjab import load

# handwriting: a folder with one folder per label (my-digits/੨/1.png, ...), a Hugging Face dataset id,
# or (image, label) pairs; without data, the model's own dataset
digits = load("digit-recognition").train(data="my-digits", epochs=10)
digits.predict("digit.png")

# spelling: a word list (.txt, one correctly spelled word per line, most common first) or a list of words
checker = load("spell-checker-base").train(data="my_words.txt", epochs=2)
checker.check("ਮੈਂ ਸਕੂਲ ਜਾਂਦਾ ਹਾ").corrected_text

checker.save("my-spell-checker")          # later: load("my-spell-checker")
```

`train()` continues from the model you loaded and returns the new one, ready to use. Options are keywords
(`epochs`, `batch_size`, `lr`, ...; the spell checker also `steps`, `gold`), `workdir="runs"` keeps the new model
there (default: a temporary folder). A saved folder is laid out like the model's repo, so it can be pushed to Hugging
Face as it is. Training uses a GPU when there is one (NVIDIA, or Apple silicon); `device="cpu"`, `"cuda"` or `"mps"`
chooses. The spell checker takes a word list of any
size: with 1,000 words or more, 1% are held out to test the model as it trains; with fewer, all of them train.

The first `load()` in a program also fetches this repo's small `config.json`, which is how Hugging Face counts the
loader's downloads; offline it is skipped.

## Pin a version

To keep a program on one version of this package: `pip install git+https://huggingface.co/aipanjab/model-loader@v0.1.0`.

By [AI Panjab](https://aipanjab.com). Licence: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).