Text Generation
GGUF
English
flatbuild
language-model
conversational
flatseek
flatrun
trained-from-scratch
rope
rmsnorm
swiglu
gqa
q4_0
Instructions to use flatseek/flatbot-mini-35M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use flatseek/flatbot-mini-35M with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf flatseek/flatbot-mini-35M # Run inference directly in the terminal: llama cli -hf flatseek/flatbot-mini-35M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf flatseek/flatbot-mini-35M # Run inference directly in the terminal: llama cli -hf flatseek/flatbot-mini-35M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf flatseek/flatbot-mini-35M # Run inference directly in the terminal: ./build/bin/llama cli -hf flatseek/flatbot-mini-35M
Use Docker
docker model run hf.co/flatseek/flatbot-mini-35M
- Llama new
- LM Studio
- Jan
- vLLM
How to use flatseek/flatbot-mini-35M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "flatseek/flatbot-mini-35M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "flatseek/flatbot-mini-35M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/flatseek/flatbot-mini-35M
- Ollama
How to use flatseek/flatbot-mini-35M with Ollama:
ollama run hf.co/flatseek/flatbot-mini-35M
- Unsloth Desktop
- Docker Model Runner
How to use flatseek/flatbot-mini-35M with Docker Model Runner:
docker model run hf.co/flatseek/flatbot-mini-35M
- Lemonade
How to use flatseek/flatbot-mini-35M with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull flatseek/flatbot-mini-35M
Run and chat with the model
lemonade run user.flatbot-mini-35M-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,200 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
library_name: flatbuild
|
| 6 |
+
tags:
|
| 7 |
+
- language-model
|
| 8 |
+
- conversational
|
| 9 |
+
- flatseek
|
| 10 |
+
- flatbuild
|
| 11 |
+
- flatrun
|
| 12 |
+
- trained-from-scratch
|
| 13 |
+
- rope
|
| 14 |
+
- rmsnorm
|
| 15 |
+
- swiglu
|
| 16 |
+
- gqa
|
| 17 |
+
- gguf
|
| 18 |
+
- q4_0
|
| 19 |
+
pipeline_tag: text-generation
|
| 20 |
---
|
| 21 |
+
|
| 22 |
+
# Flatbot-mini-11M-Q4_0
|
| 23 |
+
|
| 24 |
+
**Try it online:** https://chat.flatseek.io
|
| 25 |
+
|
| 26 |
+
Flatbot-mini-11M-Q4_0 is the flagship compact conversational model for the **Flatseek** ecosystem.
|
| 27 |
+
|
| 28 |
+
It is an **11 million parameter** decoder-only language model trained entirely from scratch using **FlatBuild**, exported to **GGUF**, quantized to **Q4_0**, and optimized for efficient CPU inference with **FlatRun**.
|
| 29 |
+
|
| 30 |
+
The original FP32 checkpoint is approximately **135 MB**, while the **Q4_0 GGUF** version reduces the model size significantly, making it practical for lightweight local deployment without requiring a GPU.
|
| 31 |
+
|
| 32 |
+
> **Experimental model:** Flatbot-mini-11M is intended for research, education, and experimentation. Although substantially more capable than Flatbot-micro-4M, it may still produce hallucinations, factual mistakes, or inconsistent reasoning.
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
|
| 36 |
+
# Architecture
|
| 37 |
+
|
| 38 |
+
| Component | Details |
|
| 39 |
+
|---|---|
|
| 40 |
+
| Architecture | Decoder-only Transformer |
|
| 41 |
+
| Position Encoding | RoPE |
|
| 42 |
+
| Normalization | RMSNorm |
|
| 43 |
+
| Feed Forward | SwiGLU |
|
| 44 |
+
| Attention | Grouped Query Attention (16 Query Heads / 4 KV Heads) |
|
| 45 |
+
| Weight Tying | Yes |
|
| 46 |
+
| Context Length | 512 |
|
| 47 |
+
| Parameters | ~11M |
|
| 48 |
+
|
| 49 |
+
---
|
| 50 |
+
|
| 51 |
+
# Model Configuration
|
| 52 |
+
|
| 53 |
+
```text
|
| 54 |
+
vocab_size = 1024
|
| 55 |
+
hidden_size = 512
|
| 56 |
+
num_layers = 12
|
| 57 |
+
num_heads = 16
|
| 58 |
+
num_kv_heads = 4
|
| 59 |
+
head_dim = 32
|
| 60 |
+
ffn_dim = 1408
|
| 61 |
+
context_length = 512
|
| 62 |
+
rope_theta = 10000
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
---
|
| 66 |
+
|
| 67 |
+
# Training
|
| 68 |
+
|
| 69 |
+
The model was trained entirely from random initialization using **FlatBuild**.
|
| 70 |
+
|
| 71 |
+
## Dataset
|
| 72 |
+
|
| 73 |
+
| Property | Value |
|
| 74 |
+
|---|---:|
|
| 75 |
+
| Conversations | ~10,000 |
|
| 76 |
+
| Train Split | 95% |
|
| 77 |
+
| Validation Split | 5% |
|
| 78 |
+
| Context Length | 512 tokens |
|
| 79 |
+
|
| 80 |
+
The training dataset consists of approximately **10,000** conversational examples covering greetings, general knowledge, explanations, recommendations, question answering, follow-up conversations, and common assistant interactions.
|
| 81 |
+
|
| 82 |
+
---
|
| 83 |
+
|
| 84 |
+
# Training Configuration
|
| 85 |
+
|
| 86 |
+
| Hyperparameter | Value |
|
| 87 |
+
|---|---:|
|
| 88 |
+
| Optimizer | AdamW |
|
| 89 |
+
| Learning Rate | 1e-3 |
|
| 90 |
+
| Scheduler | Cosine |
|
| 91 |
+
| Warmup | 50 steps |
|
| 92 |
+
| Epochs | 10 |
|
| 93 |
+
| Batch Size | 16 |
|
| 94 |
+
| Gradient Accumulation | 2 |
|
| 95 |
+
| Precision | FP32 |
|
| 96 |
+
|
| 97 |
+
---
|
| 98 |
+
|
| 99 |
+
# Quantization
|
| 100 |
+
|
| 101 |
+
This release is provided as a **GGUF Q4_0** model optimized for lightweight inference.
|
| 102 |
+
|
| 103 |
+
| Item | Value |
|
| 104 |
+
|---|---:|
|
| 105 |
+
| Original Format | FP32 |
|
| 106 |
+
| Original Size | ~135 MB |
|
| 107 |
+
| Quantization | GGUF Q4_0 |
|
| 108 |
+
| Intended Runtime | FlatRun / llama.cpp compatible runtimes |
|
| 109 |
+
|
| 110 |
+
---
|
| 111 |
+
|
| 112 |
+
# Features
|
| 113 |
+
|
| 114 |
+
- Trained entirely from scratch
|
| 115 |
+
- Custom tokenizer (1024 vocabulary)
|
| 116 |
+
- Native chat template
|
| 117 |
+
- RoPE positional embeddings
|
| 118 |
+
- RMSNorm
|
| 119 |
+
- SwiGLU
|
| 120 |
+
- Grouped Query Attention (GQA)
|
| 121 |
+
- Weight-tied embeddings
|
| 122 |
+
- SafeTensors export
|
| 123 |
+
- GGUF export
|
| 124 |
+
- Q4_0 quantization
|
| 125 |
+
- Native FlatRun compatibility
|
| 126 |
+
|
| 127 |
+
---
|
| 128 |
+
|
| 129 |
+
# Usage
|
| 130 |
+
|
| 131 |
+
## FlatBuild Training
|
| 132 |
+
|
| 133 |
+
```bash
|
| 134 |
+
pip install flatbuild
|
| 135 |
+
|
| 136 |
+
flatbuild train configs/flatbot-mini-11M.yaml
|
| 137 |
+
|
| 138 |
+
flatbuild export \
|
| 139 |
+
outputs/flatbot-mini-11M-v2/*/checkpoint/final \
|
| 140 |
+
--format gguf \
|
| 141 |
+
--output flatbot-mini-11M
|
| 142 |
+
```
|
| 143 |
+
|
| 144 |
+
---
|
| 145 |
+
|
| 146 |
+
## LM Studio
|
| 147 |
+
|
| 148 |
+
```bash
|
| 149 |
+
lms import flatbot-mini-11M-Q4_0.gguf
|
| 150 |
+
```
|
| 151 |
+
|
| 152 |
+
---
|
| 153 |
+
|
| 154 |
+
## FlatRun Inference
|
| 155 |
+
|
| 156 |
+
```bash
|
| 157 |
+
pip install flatrun
|
| 158 |
+
|
| 159 |
+
flatrun chat \
|
| 160 |
+
--model flatbot-mini-11M-Q4_0.gguf \
|
| 161 |
+
--temp 0.2
|
| 162 |
+
```
|
| 163 |
+
|
| 164 |
+
Example:
|
| 165 |
+
|
| 166 |
+
```text
|
| 167 |
+
Detected format: gguf
|
| 168 |
+
Building tokenizer from GGUF metadata...
|
| 169 |
+
Loaded model in 0.03 s
|
| 170 |
+
|
| 171 |
+
You: Who are you?
|
| 172 |
+
|
| 173 |
+
Assistant:
|
| 174 |
+
I'm Flatbot, a conversational AI assistant trained from scratch using the Flatseek ecosystem. I'm designed to answer questions, explain concepts, and assist with everyday tasks.
|
| 175 |
+
```
|
| 176 |
+
|
| 177 |
+
---
|
| 178 |
+
|
| 179 |
+
# Purpose
|
| 180 |
+
|
| 181 |
+
Flatbot-mini-11M demonstrates the complete **Flatseek AI pipeline**:
|
| 182 |
+
|
| 183 |
+
1. Build conversational datasets
|
| 184 |
+
2. Train a tokenizer
|
| 185 |
+
3. Configure a transformer architecture
|
| 186 |
+
4. Train entirely from scratch
|
| 187 |
+
5. Export SafeTensors checkpoints
|
| 188 |
+
6. Convert to GGUF
|
| 189 |
+
7. Quantize to Q4_0
|
| 190 |
+
8. Run efficient local inference with FlatRun
|
| 191 |
+
|
| 192 |
+
The entire workflow is fully reproducible on consumer hardware and showcases how modern language models can be built without relying on pretrained foundation models.
|
| 193 |
+
|
| 194 |
+
---
|
| 195 |
+
|
| 196 |
+
# Limitations
|
| 197 |
+
|
| 198 |
+
Flatbot-mini-11M is still a relatively small language model compared with modern LLMs containing billions of parameters. While it offers noticeably stronger conversational ability than the earlier 4M demonstration model, it remains limited in factual knowledge, reasoning depth, multilingual performance, and long-context understanding.
|
| 199 |
+
|
| 200 |
+
Its primary goal is to demonstrate an end-to-end open-source language model training, export, quantization, and inference pipeline built entirely within the Flatseek ecosystem.
|