Instructions to use mdevst/trocr-large-handwritten with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mdevst/trocr-large-handwritten with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("image-to-text", model="mdevst/trocr-large-handwritten")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("mdevst/trocr-large-handwritten") model = AutoModelForMultimodalLM.from_pretrained("mdevst/trocr-large-handwritten", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Create README.md
Browse files# TrOCR Large Handwritten - ONNX
This is an ONNX conversion of Microsoft's [trocr-large-handwritten](https://huggingface.co/microsoft/trocr-large-handwritten) model, optimized for web deployment and edge inference.
## π Model Description
**TrOCR (Transformer-based Optical Character Recognition)** is a state-of-the-art model for handwritten text recognition. This ONNX version enables:
- β‘ **Web deployment** with [Transformers.js](https://huggingface.co/docs/transformers.js)
- π **Edge inference** with ONNX Runtime
- π± **Cross-platform** compatibility (Web, Node.js, Mobile)
## π§ Usage
### Web (Transformers.js)
```javascript
import { pipeline } from '@huggingface/transformers';
const captioner = await pipeline('image-to-text', 'mdevst/trocr-large-handwritten');
// Use with image
const result = await captioner('path/to/handwritten/image.jpg');
console.log(result[0].generated_text);
```
### Node.js (ONNX Runtime)
```javascript
const ort = require('onnxruntime-node');
// Load the model
const session = await ort.InferenceSession.create('./model.onnx');
// Run inference
const results = await session.run(feeds);
```
### Python (ONNX Runtime)
```python
from optimum.onnxruntime import ORTModelForVision2Seq
from transformers import TrOCRProcessor
from PIL import Image
# Load model and processor
model = ORTModelForVision2Seq.from_pretrained('mdevst/trocr-large-handwritten')
processor = TrOCRProcessor.from_pretrained('microsoft/trocr-large-handwritten')
# Process image
image = Image.open('handwritten_text.jpg')
pixel_values = processor(image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
```
## π Model Performance
- **Original Model Size**: ~2.23GB
- **ONNX Model Size**: ~4.7GB
## π― Use Cases
- βοΈ Handwritten document digitization
- π Note transcription
- π OCR for historical documents
- π± Mobile text recognition apps
- π Web-based OCR services
## β‘ Performance Tips
- Enable **WebGPU** for faster inference in browsers
- **Batch processing** for multiple images
- **Image preprocessing** for better accuracy
## π Credits
- **Original Model**: Microsoft Research
- **Paper**: [TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models](https://arxiv.org/abs/2109.10282)
- **ONNX Conversion**: Converted using π€ Optimum
- **Framework**: Built with π€ Transformers
## π Issues & Support
For issues related to:
- **Original model**: [Microsoft TrOCR Issues](https://huggingface.co/microsoft/trocr-large-handwritten/discussions)
- **ONNX conversion**: Open an issue in this repository
- **Transformers.js**: [Transformers.js Issues](https://github.com/huggingface/transformers.js/issues)