Token Classification
Transformers
Safetensors
Malagasy
xlm-roberta
punctuation-restoration
truecasing
malagasy
asr-postprocessing
Instructions to use Flo976/malagasy-punctuation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Flo976/malagasy-punctuation with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="Flo976/malagasy-punctuation")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("Flo976/malagasy-punctuation") model = AutoModelForTokenClassification.from_pretrained("Flo976/malagasy-punctuation", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 809 Bytes
7df18eb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | {
"schema": "etiquette combinee = indice_casse * 7 + indice_ponct, lue sur le premier sous-jeton de chaque mot normalise",
"cases": {
"L": "minuscule",
"C": "Capitale initiale",
"U": "TOUT EN MAJUSCULES",
"M": "mixte (forme lue dans tables.json['mixed'], sinon capitale apres la derniere apostrophe/trait d'union)"
},
"puncts": [
"O",
",",
".",
"?",
"!",
":",
";"
],
"normalisation_entree": "evaluate_asr.normalize (minuscules, [a-z0-9'-], apostrophes/tirets unifies)",
"tables": "tables.json : 'mixed' = formes a casse mixte ; 'lex' = n-gramme normalise -> forme accentuee (ex. 'tan na' -> 'tanàna'), construite sur train + Wikipedia",
"base": "Davlan/xlm-roberta-base-finetuned-malagasy",
"entrainement": "work/punct/train.py --noise 0.5 --epochs 3 (checkpoint epoque 2.5)"
} |