ContextReq commited on
Commit
0897af6
·
verified ·
1 Parent(s): 07cf3bf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -14,4 +14,6 @@ tags:
14
 
15
  ill clean this page up at some point.
16
 
17
- 109 token IDS for vocab.txt/.bin, line number is implied token id, token value e.g. line 99 == id 99, "token", coverted into binary(.bin).
 
 
 
14
 
15
  ill clean this page up at some point.
16
 
17
+ 109 token IDS for vocab.txt/.bin, line number is implied token id, token value e.g. line 99 == id 99, "token", coverted into binary(.bin).
18
+
19
+ datasets included; cleaned to fit the training and token format, along with %95/%5 train/val split.