Plans11 commited on
Commit
84c34bf
·
verified ·
1 Parent(s): c3c1935

FrontD.11m completed session 2

Browse files
README.md CHANGED
@@ -16,8 +16,8 @@ tags:
16
 
17
  # FrontD.11m
18
 
19
- FrontD.11m is an approximately 11-million-parameter
20
- decoder-only language model trained completely from scratch.
21
 
22
  ## Architecture
23
 
@@ -33,19 +33,15 @@ decoder-only language model trained completely from scratch.
33
  - SwiGLU
34
  - Tied input/output embeddings
35
 
36
- ## Pretraining
37
-
38
- Dataset:
39
 
40
  `Plans11/Organized_PreTrain_1k_Context`
41
 
42
- Training is performed in 20,000-example sessions.
43
-
44
- A persistent SHA-256 ledger prevents completed examples
45
- from being selected again.
46
 
47
- Interrupted sessions resume from their latest checkpoint.
48
 
49
- ## Training from scratch
 
50
 
51
- No pretrained model weights are imported.
 
16
 
17
  # FrontD.11m
18
 
19
+ FrontD.11m is an approximately 11-million-parameter decoder-only
20
+ language model trained completely from scratch.
21
 
22
  ## Architecture
23
 
 
33
  - SwiGLU
34
  - Tied input/output embeddings
35
 
36
+ ## Pretraining dataset
 
 
37
 
38
  `Plans11/Organized_PreTrain_1k_Context`
39
 
40
+ ## Training
 
 
 
41
 
42
+ Training is performed in 20,000-example sessions.
43
 
44
+ A persistent SHA-256 example ledger records only examples whose
45
+ training session has successfully completed.
46
 
47
+ The model is initialized randomly and pretrained from scratch.
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dc1dbc61d3691a3b5d97446b5f400c0f638c7f36b2089981751c122927a25e32
3
  size 44000846
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa800cfca16fa420a2b064733b5a804c46f9391d22c0976cbbd71f79ec76148a
3
  size 44000846
session_000002.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "session": 2,
3
+ "parameters": 10996608,
4
+ "context_length": 1096,
5
+ "vocab_size": 16000,
6
+ "dataset": "Plans11/Organized_PreTrain_1k_Context",
7
+ "examples_per_session": 20000,
8
+ "uploaded_at": "2026-08-19 06:35:12 UTC"
9
+ }
training_state/latest.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:439f3806541090f30e1ed5b09aed57833bae14132ba9cc1806153f33cc8dd66f
3
- size 164223243
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:195b6e5a1159ddcd3fec9e9f03558a7d5f6fc9b7396c65d659a55469d910197a
3
+ size 132030728
training_state/run_state.json CHANGED
@@ -1,15 +1,13 @@
1
  {
2
- "checkpoint_version": 4,
3
  "model_repo": "ITLL/FrontD.11m",
4
  "dataset_repo": "Plans11/Organized_PreTrain_1k_Context",
5
- "session_number": 1,
6
- "global_step": 313,
7
- "examples_seen_total": 20000,
8
- "session_examples_current": 20000,
9
- "next_batch_index": 2500,
10
- "session_complete": true,
11
  "parameter_count": 10996608,
12
  "context_length": 1096,
13
  "vocab_size": 16000,
14
- "updated_at": "2026-08-19 06:22:29 UTC"
15
  }
 
1
  {
 
2
  "model_repo": "ITLL/FrontD.11m",
3
  "dataset_repo": "Plans11/Organized_PreTrain_1k_Context",
4
+ "session_number": 2,
5
+ "global_step": 626,
6
+ "examples_seen_total": 40000,
7
+ "current_session_examples": 0,
8
+ "next_batch_index": 0,
 
9
  "parameter_count": 10996608,
10
  "context_length": 1096,
11
  "vocab_size": 16000,
12
+ "updated_at": "2026-08-19 06:35:11 UTC"
13
  }
training_state/seen_examples.jsonl CHANGED
The diff for this file is too large to render. See raw diff