Token Classification
Transformers
ONNX
Safetensors
English
Japanese
Chinese
bert
anime
filename-parsing
Eval Results (legacy)
Instructions to use ModerRAS/AniFileBERT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ModerRAS/AniFileBERT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="ModerRAS/AniFileBERT")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("ModerRAS/AniFileBERT") model = AutoModelForTokenClassification.from_pretrained("ModerRAS/AniFileBERT", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Add robust LLM relabel pipeline and enforce contiguous title
fed9d99 -
set gpt-5.4-mini default token pricing
8165cc3 -
feat: add concurrency and token/cost telemetry for llm relabel runs
7c5b41e -
feat: add tool-call based llm relabel pipeline and update dataset pointer
e964ae5 -
chore: update AnimeName submodule checkpoint
1804249 -
chore: checkpoint current training and manual relabel progress
efb213a
-
Add tests for combo generator
1bef818 -
Add balanced focus dataset generator
651ad49 -
Add path-aware focus dataset support
1e1bc1f
-
Train virtual-shard anime parser
359ff82
-
Train augmented anime filename parser
be6a29a
-
Organize parser modules and tools
8c50d16 -
Clean stale local scripts
f95ce71 -
Remove structural parser rule assists
116c87c -
Document training lineage
f2ec095 -
Train thin anime filename parser
f712f4b
-
Remove stale diagnostics reports
76e084f -
Add inference performance benchmark
ce3a60d -
Polish Hugging Face repository docs
376db19
-
Clean special code parsing
e8412e3 -
Fix GM-Team bilingual title parsing
adf92db -
Improve anime filename parser model
e63569d
-
Add parser diagnostics and inference debugging
- add diagnose_pipeline.py for BIO validation, tokenizer alignment, entity stats, truncation, UNK, and confusion analysis - add diagnostics reports for char and regex DMHY datasets - add inference debug output, constrained BIO decoding, checkpoint max-length handling, and rule-assisted parsingb57780c -
Add Codex Colab training workflow
e458112 -
Add agent repository guidelines
beb8c7e -
Mirror character vocab in model repo
f408729
-
Add character-token DMHY training path
0779202 -
colab: onnx导出改为非阻塞+补全onnxscript
e34dc04 -
colab: 加上onnx/onnxruntime依赖
c5ba3cb -
修复colab_train.py shell引号嵌套导致的Syntax error
4df510c -
重建词表:从632K dmhy_weak.jsonl统计频率取top8000,覆盖96.2%
- 词表从3000扩展到8000,新增'['、']'、常见字幕组名(Snow/LoliHouse/KTXP等) - OOV率从25%降到3.8%,修复训练/推理token不一致问题 - 更新config.py默认vocab_size,修复build_vocab_from_data传递max_size - 添加colab_train.py自动训练脚本 - 更新README训练说明和CUDA 12.6配置410e000 -
重新训练模型,补充训练结果
24a2cb6 -
提高训练速度并使用GPU进行训练
3a13ca3 -
完成整个数据集的整理
f4f4e0e
-
Document AniFileBERT maintenance workflow
3197202 -
Add AniFileBERT model and training project
Publish the fine-tuned filename parser checkpoint, training/export scripts, documentation, and a nested AnimeName dataset submodule.be5f706 -
initial commit
Verified signaturea94e250