Token Classification
Transformers
ONNX
Safetensors
English
Japanese
Chinese
bert
anime
filename-parsing
Eval Results (legacy)
Instructions to use ModerRAS/AniFileBERT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ModerRAS/AniFileBERT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="ModerRAS/AniFileBERT")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("ModerRAS/AniFileBERT") model = AutoModelForTokenClassification.from_pretrained("ModerRAS/AniFileBERT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
重建词表:从632K dmhy_weak.jsonl统计频率取top8000,覆盖96.2%
- 词表从3000扩展到8000,新增'['、']'、常见字幕组名(Snow/LoliHouse/KTXP等) - OOV率从25%降到3.8%,修复训练/推理token不一致问题 - 更新config.py默认vocab_size,修复build_vocab_from_data传递max_size - 添加colab_train.py自动训练脚本 - 更新README训练说明和CUDA 12.6配置410e000 -
完成整个数据集的整理
f4f4e0e
-
Add AniFileBERT model and training project
Publish the fine-tuned filename parser checkpoint, training/export scripts, documentation, and a nested AnimeName dataset submodule.be5f706