Add library_name, pipeline_tag, and links to paper and code
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,10 +1,14 @@
|
|
| 1 |
---
|
| 2 |
language:
|
| 3 |
- vi
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
# GoiBERT-M: a half-size Vietnamese encoder that outperforms BamiBERT
|
| 7 |
|
|
|
|
|
|
|
| 8 |
GoiBERT-M is a **49.4M-parameter** Vietnamese RoBERTa encoder, about **½ the size of BamiBERT**. It is trained by general distillation from BamiBERT on a cleaned Vietnamese corpus of 329M tokens. It uses BamiBERT's tokenizer (20,481 tokens) and takes raw text, with no word segmentation needed.
|
| 9 |
|
| 10 |
After task fine-tuning (see *Evaluation*), GoiBERT-M **outperforms BamiBERT on 11 of 13 metrics** across 7 Vietnamese benchmarks, **+0.73 points on average**. It has about **2× the batched GPU throughput** and uses about **½ the GPU memory**.
|
|
@@ -118,4 +122,4 @@ GoiBERT-M was fine-tuned on 7 Vietnamese benchmarks, covering 13 metrics.
|
|
| 118 |
| Peak GPU memory, batch 32 × 256 | 801 MB | **410 MB** |
|
| 119 |
| Weights (bf16) | 206 MB | 99 MB |
|
| 120 |
|
| 121 |
-
GoiBERT-M has the same depth as BamiBERT (12 layers). For single sequences on GPU, latency is bound by per-layer overhead, so it matches BamiBERT. The speed-up appears with batched inference and on CPU.
|
|
|
|
| 1 |
---
|
| 2 |
language:
|
| 3 |
- vi
|
| 4 |
+
pipeline_tag: feature-extraction
|
| 5 |
+
library_name: transformers
|
| 6 |
---
|
| 7 |
|
| 8 |
# GoiBERT-M: a half-size Vietnamese encoder that outperforms BamiBERT
|
| 9 |
|
| 10 |
+
**Paper**: [Hugging Face paper page](https://huggingface.co/papers/2610.30067) · **Code**: [GitHub](https://github.com/Nhantran65/GoiBERT.git)
|
| 11 |
+
|
| 12 |
GoiBERT-M is a **49.4M-parameter** Vietnamese RoBERTa encoder, about **½ the size of BamiBERT**. It is trained by general distillation from BamiBERT on a cleaned Vietnamese corpus of 329M tokens. It uses BamiBERT's tokenizer (20,481 tokens) and takes raw text, with no word segmentation needed.
|
| 13 |
|
| 14 |
After task fine-tuning (see *Evaluation*), GoiBERT-M **outperforms BamiBERT on 11 of 13 metrics** across 7 Vietnamese benchmarks, **+0.73 points on average**. It has about **2× the batched GPU throughput** and uses about **½ the GPU memory**.
|
|
|
|
| 122 |
| Peak GPU memory, batch 32 × 256 | 801 MB | **410 MB** |
|
| 123 |
| Weights (bf16) | 206 MB | 99 MB |
|
| 124 |
|
| 125 |
+
GoiBERT-M has the same depth as BamiBERT (12 layers). For single sequences on GPU, latency is bound by per-layer overhead, so it matches BamiBERT. The speed-up appears with batched inference and on CPU.
|