devoppro commited on
Commit
8527b54
·
verified ·
1 Parent(s): 4a0aa68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -10
README.md CHANGED
@@ -1,20 +1,42 @@
1
  ---
2
- title: FastLLM 150M Demo
3
  emoji: ⚡
4
- colorFrom: indigo
5
  colorTo: purple
6
  sdk: gradio
7
- sdk_version: 5.31.0
 
8
  app_file: app.py
9
  pinned: false
10
- license: apache-2.0
11
- short_description: Text generation demo for devoppro/FastLLM (150M)
12
  ---
13
 
14
- # FastLLM (150M) Demo
15
 
16
- Interactive text-generation demo for [devoppro/FastLLM](https://huggingface.co/devoppro/FastLLM),
17
- a ~150M parameter decoder-only causal LM (GQA, SwiGLU, RMSNorm, RoPE) trained from scratch.
18
 
19
- Runs on ZeroGPU — the model loads once and `@spaces.GPU` requests a GPU
20
- slice only for the duration of each generation call.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: FastLLM
3
  emoji: ⚡
4
+ colorFrom: blue
5
  colorTo: purple
6
  sdk: gradio
7
+ sdk_version: "5.38.0"
8
+ python_version: "3.10"
9
  app_file: app.py
10
  pinned: false
 
 
11
  ---
12
 
13
+ # ⚡ FastLLM
14
 
15
+ Chat with FastLLM directly in your browser.
 
16
 
17
+ FastLLM is a ~150M parameter decoder-only causal language model created by **devoppro**.
18
+
19
+ ## Model
20
+
21
+ - Parameters: ~150M
22
+ - Architecture: ModernLLMForCausalLM
23
+ - Hidden size: 768
24
+ - Layers: 12
25
+ - Attention: GQA
26
+ - Query heads: 12
27
+ - KV heads: 4
28
+ - MLP: SwiGLU
29
+ - Normalization: RMSNorm
30
+ - Position encoding: RoPE
31
+ - Context length: 2048
32
+ - Format: Safetensors
33
+
34
+ ## Model
35
+
36
+ https://huggingface.co/devoppro/FastLLM
37
+
38
+ ## Runtime
39
+
40
+ This Space uses Hugging Face Transformers with the model's custom architecture.
41
+
42
+ No AirLLM or flash-attn is required.