Instructions to use much1na/gemma-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use much1na/gemma-tokenizer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="much1na/gemma-tokenizer")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("much1na/gemma-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
5 days ago
-
Fix title typo
Verified signature9134e8d -
Fill in usage section
Verified signaturee293685 -
Prune config.json (not needed by AutoTokenizer)
Verified signatureb0a7f65 -
Update README metadata: pipeline_tag feature-extraction, tags
Verified signature2245267
23 days ago
-
Upload folder using huggingface_hub
Verified signature39653c3 -
initial commit
Verified signature6903717