Add metadata and links
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,10 +1,14 @@
|
|
| 1 |
---
|
| 2 |
language:
|
| 3 |
- vi
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
# GoiBERT-S: a small, fast Vietnamese encoder distilled from BamiBERT
|
| 7 |
|
|
|
|
|
|
|
| 8 |
GoiBERT-S is a **26.4M-parameter** Vietnamese RoBERTa encoder, about **¼ the size of BamiBERT**. It is trained by general distillation from BamiBERT on a cleaned 329M-token Vietnamese corpus. It uses BamiBERT's tokenizer (20,481 tokens) and works on raw text; no word segmentation is needed.
|
| 9 |
|
| 10 |
After task fine-tuning (see *Evaluation*), GoiBERT-S scores within **0.07 points** of BamiBERT on average across 13 metrics on 7 Vietnamese benchmarks. It runs **2–3.5× faster** and uses about **⅓ of the GPU memory**.
|
|
@@ -116,4 +120,4 @@ Encoder only. GPU: NVIDIA L40S, bf16. CPU: fp32, 8 threads.
|
|
| 116 |
| GPU throughput, batch 32 × 256 | 2,397 seq/s | **8,290 seq/s (3.5×)** |
|
| 117 |
| CPU latency, batch 1 × 128 | 31.0 ms | **14.2 ms (2.2×)** |
|
| 118 |
| Peak GPU memory, batch 32 × 256 | 801 MB | **248 MB** |
|
| 119 |
-
| Weights (bf16) | 206 MB | 53 MB |
|
|
|
|
| 1 |
---
|
| 2 |
language:
|
| 3 |
- vi
|
| 4 |
+
library_name: transformers
|
| 5 |
+
pipeline_tag: feature-extraction
|
| 6 |
---
|
| 7 |
|
| 8 |
# GoiBERT-S: a small, fast Vietnamese encoder distilled from BamiBERT
|
| 9 |
|
| 10 |
+
> **Paper:** [GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation](https://huggingface.co/papers/2610.30067) · **Code:** [GitHub](https://github.com/Nhantran65/GoiBERT)
|
| 11 |
+
|
| 12 |
GoiBERT-S is a **26.4M-parameter** Vietnamese RoBERTa encoder, about **¼ the size of BamiBERT**. It is trained by general distillation from BamiBERT on a cleaned 329M-token Vietnamese corpus. It uses BamiBERT's tokenizer (20,481 tokens) and works on raw text; no word segmentation is needed.
|
| 13 |
|
| 14 |
After task fine-tuning (see *Evaluation*), GoiBERT-S scores within **0.07 points** of BamiBERT on average across 13 metrics on 7 Vietnamese benchmarks. It runs **2–3.5× faster** and uses about **⅓ of the GPU memory**.
|
|
|
|
| 120 |
| GPU throughput, batch 32 × 256 | 2,397 seq/s | **8,290 seq/s (3.5×)** |
|
| 121 |
| CPU latency, batch 1 × 128 | 31.0 ms | **14.2 ms (2.2×)** |
|
| 122 |
| Peak GPU memory, batch 32 × 256 | 801 MB | **248 MB** |
|
| 123 |
+
| Weights (bf16) | 206 MB | 53 MB |
|