judotens commited on
Commit
9eb8b76
·
verified ·
1 Parent(s): 60e527e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -16
README.md CHANGED
@@ -19,11 +19,15 @@ pipeline_tag: text-generation
19
 
20
  # Flatbot-micro-4M
21
 
 
 
22
  Flatbot-micro-4M is the flagship demonstration model for **Flatseek**.
23
 
24
- It is a compact conversational language model trained entirely from scratch using the Flatseek framework. The goal is not to compete with large language models, but to demonstrate the complete workflow of building a modern transformer—from dataset generation and tokenizer training to model training, checkpoint export, and inference.
 
 
25
 
26
- Everything in this repository is produced by Flatbuild without relying on pretrained weights.
27
 
28
  ## Architecture
29
 
@@ -54,7 +58,7 @@ rope_theta = 10000
54
 
55
  ## Training
56
 
57
- The model was trained completely from random initialization using Flatbuild.
58
 
59
  ### Dataset
60
 
@@ -93,11 +97,11 @@ The dataset contains general-purpose conversations including greetings, introduc
93
  - Weight-tied embeddings
94
  - SafeTensors export
95
  - GGUF export
96
- - Native Flatrun compatibility
97
 
98
  ## Usage
99
 
100
- ### Flatbuild Training
101
 
102
  ```bash
103
  pip install flatbuild
@@ -105,28 +109,27 @@ flatbuild train config/demo_chat.yml
105
  flatbuild export output/demo_chat/*/checkpoint/final --format gguf --output flatbot-micro-4M
106
  ```
107
 
108
- ### LMStudio Import
109
 
110
  ```bash
111
  lms import flatbot-micro-4M/model.gguf
112
  ```
113
 
114
-
115
- ### Flatrun Inference
116
 
117
  ```bash
118
  pip install flatrun
119
  flatrun chat --model flatbot-micro-4M.gguf --temp 0.2
120
  ```
121
 
122
- ```bash
123
  Detected format: gguf
124
  Building tokenizer from GGUF metadata (flatbot-micro-4M.gguf) ...
125
  Tokenizer vocab: 516
126
  Chat template: {% if messages and messages[0]['role'] != 'system' %}{{ 'You...
127
  Loaded model in 0.01 s; layers=6
128
 
129
- Chat mode (max_new=128/turn, history=True). Type your message; Ctrl-D (EOF) or 'exit' to quit.
130
 
131
  You: Who are you?
132
  Assistant:
@@ -136,9 +139,7 @@ Sure — I'm Flatbot — I'm Flatbot — an assistant trained on a sm a sm chat
136
 
137
  ## Purpose
138
 
139
- Flatbot-4M exists as an end-to-end demonstration of the Flatseek ecosystem.
140
-
141
- It showcases every stage of training a conversational transformer:
142
 
143
  1. Build a dataset
144
  2. Train a tokenizer
@@ -146,10 +147,10 @@ It showcases every stage of training a conversational transformer:
146
  4. Train from scratch
147
  5. Export checkpoints
148
  6. Convert to GGUF
149
- 7. Run inference with Flatrun
150
 
151
- The complete pipeline is reproducible on consumer hardware and is intended for learning, experimentation, and research.
152
 
153
  ## Limitations
154
 
155
- Flatbot-4M is a small conversational language model trained on a relatively small dataset. It has limited factual knowledge and reasoning ability compared to larger models, and is intended as a demonstration of the complete Flatseek workflow rather than a production-grade assistant.
 
19
 
20
  # Flatbot-micro-4M
21
 
22
+ **Try it online:** https://chat.flatseek.io
23
+
24
  Flatbot-micro-4M is the flagship demonstration model for **Flatseek**.
25
 
26
+ It is a compact conversational language model trained entirely from scratch using the Flatseek ecosystem. Rather than competing with large language models, Flatbot demonstrates the complete lifecycle of building an AI model—from dataset creation and tokenizer training to transformer training, checkpoint export, and inference.
27
+
28
+ Powered by a **4.1M-parameter** language model in just **20 MB**, Flatbot was trained from scratch on approximately **2,500 conversational examples**, built with **FlatBuild**, and served by **FlatRun**.
29
 
30
+ > **Experimental model:** Expect occasional hallucinations, factual errors, and inconsistent responses. Flatbot is intended for learning, experimentation, and showcasing the Flatseek ecosystem rather than production use.
31
 
32
  ## Architecture
33
 
 
58
 
59
  ## Training
60
 
61
+ The model was trained completely from random initialization using FlatBuild.
62
 
63
  ### Dataset
64
 
 
97
  - Weight-tied embeddings
98
  - SafeTensors export
99
  - GGUF export
100
+ - Native FlatRun compatibility
101
 
102
  ## Usage
103
 
104
+ ### FlatBuild Training
105
 
106
  ```bash
107
  pip install flatbuild
 
109
  flatbuild export output/demo_chat/*/checkpoint/final --format gguf --output flatbot-micro-4M
110
  ```
111
 
112
+ ### LM Studio
113
 
114
  ```bash
115
  lms import flatbot-micro-4M/model.gguf
116
  ```
117
 
118
+ ### FlatRun Inference
 
119
 
120
  ```bash
121
  pip install flatrun
122
  flatrun chat --model flatbot-micro-4M.gguf --temp 0.2
123
  ```
124
 
125
+ ```text
126
  Detected format: gguf
127
  Building tokenizer from GGUF metadata (flatbot-micro-4M.gguf) ...
128
  Tokenizer vocab: 516
129
  Chat template: {% if messages and messages[0]['role'] != 'system' %}{{ 'You...
130
  Loaded model in 0.01 s; layers=6
131
 
132
+ Chat mode (max_new=128/turn, history=True). Type your message; Ctrl-D (EOF) or 'exit' to quit.
133
 
134
  You: Who are you?
135
  Assistant:
 
139
 
140
  ## Purpose
141
 
142
+ Flatbot-micro-4M demonstrates the complete Flatseek pipeline:
 
 
143
 
144
  1. Build a dataset
145
  2. Train a tokenizer
 
147
  4. Train from scratch
148
  5. Export checkpoints
149
  6. Convert to GGUF
150
+ 7. Run inference with FlatRun
151
 
152
+ The entire workflow is reproducible on consumer hardware and is designed for education, experimentation, and research.
153
 
154
  ## Limitations
155
 
156
+ Flatbot-micro-4M is a compact language model trained on a relatively small dataset. Compared with modern large language models, it has limited factual knowledge, reasoning ability, and context capacity. Its primary purpose is to demonstrate an end-to-end open-source training and inference pipeline, not to serve as a production-grade AI assistant.