mahwizzzz commited on
Commit
7eed242
·
verified ·
1 Parent(s): 882d630

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -18
README.md CHANGED
@@ -45,9 +45,7 @@ widget:
45
 
46
  # Avey-B Urdu 25M
47
 
48
- **A compact, fast Urdu encoder trained from scratch for masked-language modeling.**
49
-
50
- **24.87M parameters** · **408K tokens/s on RTX 4060** · **32K Urdu WordPiece vocabulary**
51
 
52
  ![Quality and efficiency overview](assets/benchmark_overview.png)
53
 
@@ -57,7 +55,7 @@ widget:
57
  - **81.27 WikiANN Urdu entity F1** with only 24.87M parameters.
58
  - **1.71 ms** encoder latency at batch 1 / length 128 and **148 MiB** peak VRAM at batch 16 / length 512.
59
  - **14.29 held-out MLM perplexity**, **50.46% top-1**, and **70.39% top-5** masked-token accuracy.
60
- - Compact Urdu tokenization: **4.19 characters/token** on held-out data.
61
 
62
  ## Quick start
63
 
@@ -101,18 +99,6 @@ print(hidden.shape) # [batch, tokens, 384]
101
  | HPLT Urdu BERT | 124.36M | 83.93% ± 1.27 | 90.92 |
102
  | XLM-R base | 278.04M | 78.80% ± 0.52 | 84.89 |
103
 
104
- Sentiment uses 4K train / 1K validation / 1K test examples for two epochs; the test result is mean ± sample standard deviation over three seeds after validation learning-rate selection. The source is a machine-translated Urdu IMDb dataset. NER uses 4K training examples and the full 1K validation / 1K test Urdu WikiANN splits for three epochs; it reports validation-selected entity F1 for one seed. See [`benchmark_results.json`](benchmark_results.json) for machine-readable results.
105
-
106
- ### RTX 4060 BF16 inference
107
-
108
- | Model | B1/L128 latency | B16/L512 throughput | B16/L512 peak VRAM |
109
- |---|---:|---:|---:|
110
- | **Avey-B Urdu** | **1.71 ms** | **408,329 tok/s** | **148 MiB** |
111
- | UrNova | 4.98 ms | 84,695 tok/s | 345 MiB |
112
- | HPLT Urdu BERT | 4.95 ms | 40,400 tok/s | 1,821 MiB |
113
- | XLM-R base | 2.18 ms | 136,274 tok/s | 678 MiB |
114
-
115
- Eager encoder inference with synthetic full-length token tensors, 10 warmup and 30 measured iterations. Hardware and software choices can materially change these numbers.
116
 
117
  ## Training
118
 
@@ -129,7 +115,6 @@ Eager encoder inference with synthetic full-length token tensors, 10 warmup and
129
  | Hardware/runtime | NVIDIA RTX 4060 8GB, approximately 20 minutes |
130
  | Final train loss | 2.52 |
131
 
132
- The architecture has 12 alternating static/dynamic Avey layers, 384-dimensional embeddings, 128-token chunks, and `k=3` retrieval/compression context. The checkpoint was trained at sequence length 512; longer contexts were not evaluated here.
133
 
134
  ## Intended use and limitations
135
 
@@ -137,7 +122,7 @@ Use this checkpoint for Urdu classification, token classification, representatio
137
 
138
  HPLT packages its dataset under CC0 but does not own the underlying web text. The Apache-2.0 model license does not grant rights to third-party source documents that may be reproduced by the model.
139
 
140
- ## 📄 Architecture and citation
141
 
142
  Based on [Avey-B](https://arxiv.org/abs/2602.15814), an attention-free bidirectional encoder architecture.
143
 
 
45
 
46
  # Avey-B Urdu 25M
47
 
48
+ A compact, fast Urdu encoder trained from scratch for masked-language modeling.
 
 
49
 
50
  ![Quality and efficiency overview](assets/benchmark_overview.png)
51
 
 
55
  - **81.27 WikiANN Urdu entity F1** with only 24.87M parameters.
56
  - **1.71 ms** encoder latency at batch 1 / length 128 and **148 MiB** peak VRAM at batch 16 / length 512.
57
  - **14.29 held-out MLM perplexity**, **50.46% top-1**, and **70.39% top-5** masked-token accuracy.
58
+ - Compact Urdu tokenization: **4.19 characters/token** on held out data.
59
 
60
  ## Quick start
61
 
 
99
  | HPLT Urdu BERT | 124.36M | 83.93% ± 1.27 | 90.92 |
100
  | XLM-R base | 278.04M | 78.80% ± 0.52 | 84.89 |
101
 
 
 
 
 
 
 
 
 
 
 
 
 
102
 
103
  ## Training
104
 
 
115
  | Hardware/runtime | NVIDIA RTX 4060 8GB, approximately 20 minutes |
116
  | Final train loss | 2.52 |
117
 
 
118
 
119
  ## Intended use and limitations
120
 
 
122
 
123
  HPLT packages its dataset under CC0 but does not own the underlying web text. The Apache-2.0 model license does not grant rights to third-party source documents that may be reproduced by the model.
124
 
125
+ ## Architecture and citation
126
 
127
  Based on [Avey-B](https://arxiv.org/abs/2602.15814), an attention-free bidirectional encoder architecture.
128