Text Generation
GGUF
English
flatbuild
language-model
conversational
flatseek
flatrun
trained-from-scratch
rope
rmsnorm
swiglu
gqa
Instructions to use flatseek/flatbot-micro-4M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use flatseek/flatbot-micro-4M with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf flatseek/flatbot-micro-4M # Run inference directly in the terminal: llama cli -hf flatseek/flatbot-micro-4M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf flatseek/flatbot-micro-4M # Run inference directly in the terminal: llama cli -hf flatseek/flatbot-micro-4M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf flatseek/flatbot-micro-4M # Run inference directly in the terminal: ./build/bin/llama cli -hf flatseek/flatbot-micro-4M
Use Docker
docker model run hf.co/flatseek/flatbot-micro-4M
- Llama new
- LM Studio
- Jan
- vLLM
How to use flatseek/flatbot-micro-4M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "flatseek/flatbot-micro-4M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flatseek/flatbot-micro-4M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/flatseek/flatbot-micro-4M
- Ollama
How to use flatseek/flatbot-micro-4M with Ollama:
ollama run hf.co/flatseek/flatbot-micro-4M
- Unsloth Desktop
- Docker Model Runner
How to use flatseek/flatbot-micro-4M with Docker Model Runner:
docker model run hf.co/flatseek/flatbot-micro-4M
- Lemonade
How to use flatseek/flatbot-micro-4M with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull flatseek/flatbot-micro-4M
Run and chat with the model
lemonade run user.flatbot-micro-4M-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,11 +19,15 @@ pipeline_tag: text-generation
|
|
| 19 |
|
| 20 |
# Flatbot-micro-4M
|
| 21 |
|
|
|
|
|
|
|
| 22 |
Flatbot-micro-4M is the flagship demonstration model for **Flatseek**.
|
| 23 |
|
| 24 |
-
It is a compact conversational language model trained entirely from scratch using the Flatseek
|
|
|
|
|
|
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
## Architecture
|
| 29 |
|
|
@@ -54,7 +58,7 @@ rope_theta = 10000
|
|
| 54 |
|
| 55 |
## Training
|
| 56 |
|
| 57 |
-
The model was trained completely from random initialization using
|
| 58 |
|
| 59 |
### Dataset
|
| 60 |
|
|
@@ -93,11 +97,11 @@ The dataset contains general-purpose conversations including greetings, introduc
|
|
| 93 |
- Weight-tied embeddings
|
| 94 |
- SafeTensors export
|
| 95 |
- GGUF export
|
| 96 |
-
- Native
|
| 97 |
|
| 98 |
## Usage
|
| 99 |
|
| 100 |
-
###
|
| 101 |
|
| 102 |
```bash
|
| 103 |
pip install flatbuild
|
|
@@ -105,28 +109,27 @@ flatbuild train config/demo_chat.yml
|
|
| 105 |
flatbuild export output/demo_chat/*/checkpoint/final --format gguf --output flatbot-micro-4M
|
| 106 |
```
|
| 107 |
|
| 108 |
-
###
|
| 109 |
|
| 110 |
```bash
|
| 111 |
lms import flatbot-micro-4M/model.gguf
|
| 112 |
```
|
| 113 |
|
| 114 |
-
|
| 115 |
-
### Flatrun Inference
|
| 116 |
|
| 117 |
```bash
|
| 118 |
pip install flatrun
|
| 119 |
flatrun chat --model flatbot-micro-4M.gguf --temp 0.2
|
| 120 |
```
|
| 121 |
|
| 122 |
-
```
|
| 123 |
Detected format: gguf
|
| 124 |
Building tokenizer from GGUF metadata (flatbot-micro-4M.gguf) ...
|
| 125 |
Tokenizer vocab: 516
|
| 126 |
Chat template: {% if messages and messages[0]['role'] != 'system' %}{{ 'You...
|
| 127 |
Loaded model in 0.01 s; layers=6
|
| 128 |
|
| 129 |
-
Chat mode (max_new=128/turn, history=True).
|
| 130 |
|
| 131 |
You: Who are you?
|
| 132 |
Assistant:
|
|
@@ -136,9 +139,7 @@ Sure — I'm Flatbot — I'm Flatbot — an assistant trained on a sm a sm chat
|
|
| 136 |
|
| 137 |
## Purpose
|
| 138 |
|
| 139 |
-
Flatbot-4M
|
| 140 |
-
|
| 141 |
-
It showcases every stage of training a conversational transformer:
|
| 142 |
|
| 143 |
1. Build a dataset
|
| 144 |
2. Train a tokenizer
|
|
@@ -146,10 +147,10 @@ It showcases every stage of training a conversational transformer:
|
|
| 146 |
4. Train from scratch
|
| 147 |
5. Export checkpoints
|
| 148 |
6. Convert to GGUF
|
| 149 |
-
7. Run inference with
|
| 150 |
|
| 151 |
-
The
|
| 152 |
|
| 153 |
## Limitations
|
| 154 |
|
| 155 |
-
Flatbot-4M is a
|
|
|
|
| 19 |
|
| 20 |
# Flatbot-micro-4M
|
| 21 |
|
| 22 |
+
**Try it online:** https://chat.flatseek.io
|
| 23 |
+
|
| 24 |
Flatbot-micro-4M is the flagship demonstration model for **Flatseek**.
|
| 25 |
|
| 26 |
+
It is a compact conversational language model trained entirely from scratch using the Flatseek ecosystem. Rather than competing with large language models, Flatbot demonstrates the complete lifecycle of building an AI model—from dataset creation and tokenizer training to transformer training, checkpoint export, and inference.
|
| 27 |
+
|
| 28 |
+
Powered by a **4.1M-parameter** language model in just **20 MB**, Flatbot was trained from scratch on approximately **2,500 conversational examples**, built with **FlatBuild**, and served by **FlatRun**.
|
| 29 |
|
| 30 |
+
> **Experimental model:** Expect occasional hallucinations, factual errors, and inconsistent responses. Flatbot is intended for learning, experimentation, and showcasing the Flatseek ecosystem rather than production use.
|
| 31 |
|
| 32 |
## Architecture
|
| 33 |
|
|
|
|
| 58 |
|
| 59 |
## Training
|
| 60 |
|
| 61 |
+
The model was trained completely from random initialization using FlatBuild.
|
| 62 |
|
| 63 |
### Dataset
|
| 64 |
|
|
|
|
| 97 |
- Weight-tied embeddings
|
| 98 |
- SafeTensors export
|
| 99 |
- GGUF export
|
| 100 |
+
- Native FlatRun compatibility
|
| 101 |
|
| 102 |
## Usage
|
| 103 |
|
| 104 |
+
### FlatBuild Training
|
| 105 |
|
| 106 |
```bash
|
| 107 |
pip install flatbuild
|
|
|
|
| 109 |
flatbuild export output/demo_chat/*/checkpoint/final --format gguf --output flatbot-micro-4M
|
| 110 |
```
|
| 111 |
|
| 112 |
+
### LM Studio
|
| 113 |
|
| 114 |
```bash
|
| 115 |
lms import flatbot-micro-4M/model.gguf
|
| 116 |
```
|
| 117 |
|
| 118 |
+
### FlatRun Inference
|
|
|
|
| 119 |
|
| 120 |
```bash
|
| 121 |
pip install flatrun
|
| 122 |
flatrun chat --model flatbot-micro-4M.gguf --temp 0.2
|
| 123 |
```
|
| 124 |
|
| 125 |
+
```text
|
| 126 |
Detected format: gguf
|
| 127 |
Building tokenizer from GGUF metadata (flatbot-micro-4M.gguf) ...
|
| 128 |
Tokenizer vocab: 516
|
| 129 |
Chat template: {% if messages and messages[0]['role'] != 'system' %}{{ 'You...
|
| 130 |
Loaded model in 0.01 s; layers=6
|
| 131 |
|
| 132 |
+
Chat mode (max_new=128/turn, history=True). Type your message; Ctrl-D (EOF) or 'exit' to quit.
|
| 133 |
|
| 134 |
You: Who are you?
|
| 135 |
Assistant:
|
|
|
|
| 139 |
|
| 140 |
## Purpose
|
| 141 |
|
| 142 |
+
Flatbot-micro-4M demonstrates the complete Flatseek pipeline:
|
|
|
|
|
|
|
| 143 |
|
| 144 |
1. Build a dataset
|
| 145 |
2. Train a tokenizer
|
|
|
|
| 147 |
4. Train from scratch
|
| 148 |
5. Export checkpoints
|
| 149 |
6. Convert to GGUF
|
| 150 |
+
7. Run inference with FlatRun
|
| 151 |
|
| 152 |
+
The entire workflow is reproducible on consumer hardware and is designed for education, experimentation, and research.
|
| 153 |
|
| 154 |
## Limitations
|
| 155 |
|
| 156 |
+
Flatbot-micro-4M is a compact language model trained on a relatively small dataset. Compared with modern large language models, it has limited factual knowledge, reasoning ability, and context capacity. Its primary purpose is to demonstrate an end-to-end open-source training and inference pipeline, not to serve as a production-grade AI assistant.
|