sutharsan-311 commited on
Commit
8fb071f
·
verified ·
1 Parent(s): c501181

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +17 -12
README.md CHANGED
@@ -1,3 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # tiny-dllm
2
 
3
  A Masked Diffusion Language Model built from scratch in PyTorch — for learning and robotics research.
@@ -130,17 +146,6 @@ Logits [B, T, vocab_size]
130
 
131
  Both trained from scratch on TinyShakespeare with identical model size and steps.
132
 
133
- ## Trained Checkpoint
134
-
135
- A checkpoint trained to 50,000 steps on TinyShakespeare is available on HuggingFace:
136
-
137
- ```bash
138
- from huggingface_hub import hf_hub_download
139
- path = hf_hub_download(repo_id="sutharsan311/tiny-dllm", filename="dllm_step50000.pt")
140
- ```
141
-
142
- Hardware: NVIDIA RTX 3050 (4GB VRAM) — ~2 hours training time.
143
-
144
  ## Roadmap
145
 
146
  - [x] Character-level tokenizer
@@ -150,7 +155,7 @@ Hardware: NVIDIA RTX 3050 (4GB VRAM) — ~2 hours training time.
150
  - [x] Tamil Unicode tokenizer
151
  - [x] Train on Thirukkural + Sangam poetry
152
  - [x] Tamil Wikipedia downloader (API + full dump)
153
- - [x] Train to 50k steps
154
  - [ ] GPT baseline comparison (dLLM vs GPT on same data)
155
  - [ ] Blog post: dLLM vs GPT on TinyShakespeare
156
  - [ ] Fill-in-the-blanks (conditional generation)
 
1
+ ---
2
+ language:
3
+ - en
4
+ - ta
5
+ license: mit
6
+ tags:
7
+ - diffusion
8
+ - language-model
9
+ - pytorch
10
+ - shakespeare
11
+ - tamil
12
+ - nlp
13
+ - from-scratch
14
+ library_name: pytorch
15
+ ---
16
+
17
  # tiny-dllm
18
 
19
  A Masked Diffusion Language Model built from scratch in PyTorch — for learning and robotics research.
 
146
 
147
  Both trained from scratch on TinyShakespeare with identical model size and steps.
148
 
 
 
 
 
 
 
 
 
 
 
 
149
  ## Roadmap
150
 
151
  - [x] Character-level tokenizer
 
155
  - [x] Tamil Unicode tokenizer
156
  - [x] Train on Thirukkural + Sangam poetry
157
  - [x] Tamil Wikipedia downloader (API + full dump)
158
+ - [ ] Train to 50k steps
159
  - [ ] GPT baseline comparison (dLLM vs GPT on same data)
160
  - [ ] Blog post: dLLM vs GPT on TinyShakespeare
161
  - [ ] Fill-in-the-blanks (conditional generation)