Update README.md
Browse files
README.md
CHANGED
|
@@ -14,4 +14,6 @@ tags:
|
|
| 14 |
|
| 15 |
ill clean this page up at some point.
|
| 16 |
|
| 17 |
-
109 token IDS for vocab.txt/.bin, line number is implied token id, token value e.g. line 99 == id 99, "token", coverted into binary(.bin).
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
ill clean this page up at some point.
|
| 16 |
|
| 17 |
+
109 token IDS for vocab.txt/.bin, line number is implied token id, token value e.g. line 99 == id 99, "token", coverted into binary(.bin).
|
| 18 |
+
|
| 19 |
+
datasets included; cleaned to fit the training and token format, along with %95/%5 train/val split.
|