Instructions to use schizophyllume/BeeLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use schizophyllume/BeeLLM with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf schizophyllume/BeeLLM # Run inference directly in the terminal: llama cli -hf schizophyllume/BeeLLM
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf schizophyllume/BeeLLM # Run inference directly in the terminal: llama cli -hf schizophyllume/BeeLLM
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf schizophyllume/BeeLLM # Run inference directly in the terminal: ./build/bin/llama cli -hf schizophyllume/BeeLLM
Use Docker
docker model run hf.co/schizophyllume/BeeLLM
- Llama new
- LM Studio
- Jan
- vLLM
How to use schizophyllume/BeeLLM with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "schizophyllume/BeeLLM" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "schizophyllume/BeeLLM", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/schizophyllume/BeeLLM
- Ollama
How to use schizophyllume/BeeLLM with Ollama:
ollama run hf.co/schizophyllume/BeeLLM
- Unsloth Desktop
- Docker Model Runner
How to use schizophyllume/BeeLLM with Docker Model Runner:
docker model run hf.co/schizophyllume/BeeLLM
- Lemonade
How to use schizophyllume/BeeLLM with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull schizophyllume/BeeLLM
Run and chat with the model
lemonade run user.BeeLLM-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,52 +6,132 @@ tags:
|
|
| 6 |
- llama.cpp
|
| 7 |
- bee
|
| 8 |
- buzz
|
|
|
|
|
|
|
|
|
|
| 9 |
---
|
| 10 |
-
## Model Details
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
The
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
|
|
|
|
| 27 |
|
| 28 |
-
|
|
|
|
| 29 |
|
| 30 |
-
Coming soon!
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
- **Model type:** bee
|
| 36 |
-
- **
|
|
|
|
| 37 |
- **License:** Apache 2.0
|
| 38 |
|
| 39 |
## Uses
|
| 40 |
|
| 41 |
-
Buzzing and pollinating flowers
|
| 42 |
|
| 43 |
-
## Bias,
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
### Recommendations
|
| 48 |
|
| 49 |
-
|
| 50 |
|
| 51 |
-
|
| 52 |
|
| 53 |
-
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
Flowers, meadows, and nice forests
|
|
|
|
| 6 |
- llama.cpp
|
| 7 |
- bee
|
| 8 |
- buzz
|
| 9 |
+
- tiny
|
| 10 |
+
- toy
|
| 11 |
+
- toy-model
|
| 12 |
---
|
|
|
|
| 13 |
|
| 14 |
+
# BeeLLM is a bee
|
| 15 |
|
| 16 |
+
A family of very small language models that buzz. The tiny ones only buzz. The Mood ones buzz, and also have feelings.
|
| 17 |
|
| 18 |
+
All of them are llama-architecture `.gguf`, so they run anywhere llama.cpp does, and they're small enough to run inside web applications.
|
| 19 |
|
| 20 |
+
## The bees
|
| 21 |
|
| 22 |
+
| file | what she is | weights | layers | vocabulary | context |
|
| 23 |
+
|---|---|---|---|---|---|
|
| 24 |
+
| `BeeLLM-30P.gguf` | The original bee | 10K | 1 | 8 buzzes | 4096 |
|
| 25 |
+
| `BeeLLM-42P.gguf` | More expressive. Can now sting | 11K | 1 | 12 buzzes | 4096 |
|
| 26 |
+
| `BeeLLM-42P-Chaos.gguf` | Chaos mode: she can shout, stings, and be dramatic | 24K | 1 | 21 buzzes | 4096 |
|
| 27 |
+
| `BeeLLM-1M-Mood.gguf` | A bee with feelings | 1.1M | 4 | 1,024 tokens | 4096 |
|
| 28 |
+
| `BeeLLM-1M-Mood-Chaos.gguf` | A bee with *dramatic* feelings | 1.1M | 4 | 1,024 tokens | 4096 |
|
| 29 |
+
| `BeeLLM-12M-Mood.gguf` | The biggest bee, with memory, holds grudges | 12M | 8 | 2,048 tokens | 4096 |
|
| 30 |
|
| 31 |
+
- **BeeLLM-30P**: the original bee. Her whole vocabulary is `bz`, `zz`, `zzz`, `bzzz`, `buzz`, a space, `~` and `!`.
|
| 32 |
+
- **BeeLLM-42P**: more expressive. Adds `b`, `bu`, `z`, and `*stings you*`.
|
| 33 |
+
- **BeeLLM-42P-Chaos**: chaos mode. Adds shouting: `BZ`, `BUZZ`, `BZZZ`… and `*STINGS YOU*`.
|
| 34 |
+
- **BeeLLM-1M-Mood**: she hears what you say and answers with how she feels.
|
| 35 |
+
- **BeeLLM-1M-Mood-Chaos**: the 1M Mood bee, but louder (`BRR`, `BZZT`), with more `*actions*`.
|
| 36 |
+
- **BeeLLM-12M-Mood**: the biggest bee, with the widest range of buzzes and actions. She knows her name and holds grudges against objects and actions.
|
| 37 |
+
## About the tiny models
|
| 38 |
|
| 39 |
+
The tiny bees (30P, 42P, 42P Chaos) are less chatty and reactive. They mostly just buzz. Give them a buzz or two and they keep buzzing.
|
| 40 |
|
| 41 |
+
> **you:** `hello tiny bee`
|
| 42 |
+
> **bee:** `buzz bzzzz buzzzzzzzzzzz buzz buzz!`
|
| 43 |
|
| 44 |
+
> **you:** `buzz buzz`
|
| 45 |
+
> **bee:** `bzzz bzzzzzzz bzzz buzz buzz buzz! bzzz bzzz~ bzzz bzzz buz buzz bzzzzzz bz! buzzzzz `
|
| 46 |
|
|
|
|
| 47 |
|
| 48 |
+
## About the mood models
|
| 49 |
+
|
| 50 |
+
The Mood bees are expressive, have many more parameters, and fun to chat with.
|
| 51 |
+
|
| 52 |
+
Every reply starts with a mood tag, based on how you treat her in the conversation: `[mood:level]`
|
| 53 |
+
|
| 54 |
+
- **mood:** `neutral`, `happy`, `loving`, `sad`, `angry` or `confused`
|
| 55 |
+
- **level:** `low` or `high`
|
| 56 |
+
|
| 57 |
+
### Mood reaction examples
|
| 58 |
+
|
| 59 |
+
| say | she feels |
|
| 60 |
+
|---|---|
|
| 61 |
+
| `hi bee!` | neutral |
|
| 62 |
+
| `*gives you a flower*` | happy |
|
| 63 |
+
| `i love you` | loving |
|
| 64 |
+
| `*sprays you with bug spray*` | sad |
|
| 65 |
+
| `*swats you*` | angry |
|
| 66 |
+
| `asdfghjkl` | confused |
|
| 67 |
+
|
| 68 |
+
### Making up with an angry bee
|
| 69 |
+
|
| 70 |
+
Swatting her makes her angry. Being sweet to her straight after makes her angrier. She was trained to see through lovebombing. Say sorry.
|
| 71 |
+
|
| 72 |
+
| after `*swats you*`, say | she feels |
|
| 73 |
+
|---|---|
|
| 74 |
+
| `sorry` | neutral |
|
| 75 |
+
| `i love you` | angry (high) |
|
| 76 |
+
| `*gives you a flower*` | angry (high) |
|
| 77 |
+
|
| 78 |
+
### Other things she does
|
| 79 |
+
|
| 80 |
+
- **Put her somewhere** (`*puts you in my mouth*`, `*puts you in a jar*`, `*eats you*`) and she goes completely silent. Let her out (`*takes you out*`, `*opens the jar*`) and she comes back furious:
|
| 81 |
+
> `*FILES A FORMAL COMPLAINT AGAINST THE JAR*` - (1M Mood Chaos)
|
| 82 |
+
- **12M Mood knows her name.** It's Scintilla, and she lights up when you use it. `hi bee!` gets a polite `[neutral:low]`, but her name gets `happy:high` or `loving:high` every time:
|
| 83 |
+
> **you:** `are you scintilla`
|
| 84 |
+
> **bee:** `[loving:high] *perks up at her name* *cuddles your ear* mmm~ hmmbzz~`
|
| 85 |
+
>
|
| 86 |
+
> **you:** `what is your name`
|
| 87 |
+
> **bee:** `[happy:high] *writes Scintilla in pollen* bzzzzz~! bzzbzz! brrr~`
|
| 88 |
+
- **12M Mood holds grudges, against objects too.** Hurt her with something and she'll never forgive it:
|
| 89 |
+
> **you:** `*gives you a flower*`
|
| 90 |
+
> **bee:** `[happy:high] buzz buzz! bizz! bzzt! *hugs the flower like an old friend*`
|
| 91 |
+
>
|
| 92 |
+
> **you:** `*gives you a tomato*`
|
| 93 |
+
> **bee:** `[happy:high] zip zip bzzbzz! *takes a tiny bite of the tomato*`
|
| 94 |
+
>
|
| 95 |
+
> **you:** `*throws the tomato at you*`
|
| 96 |
+
> **bee:** `[angry:high] *STINGS YOU* *will never trust the tomato again* *attacks the tomato* BZZZZzzZ! BZZZZZZzZZZZ!!! BUZZZZzZzZz`
|
| 97 |
+
- **Very rarely, she dies** (`*stings you and then dies*`). She stays silent for a few turns, then usually comes back with `*revives from the dead*`. Sometimes she doesn't.
|
| 98 |
+
- **Ask her to sing** and she may not be in the mood:
|
| 99 |
+
> `[sad:low] buzz. buzz...` - (12M Mood)
|
| 100 |
+
|
| 101 |
+
## How to get started
|
| 102 |
+
|
| 103 |
+
With [llama.cpp](https://github.com/ggml-org/llama.cpp):
|
| 104 |
+
|
| 105 |
+
```
|
| 106 |
+
llama-server -m BeeLLM-1M-Mood.gguf -c 4096 --port 8081
|
| 107 |
+
```
|
| 108 |
+
(or the name of whichever BeeLLM you downloaded)
|
| 109 |
+
|
| 110 |
+
Then open `http://localhost:8081` and say hi.
|
| 111 |
+
|
| 112 |
+
## Model details
|
| 113 |
+
|
| 114 |
+
- **Developed by:** schizophyllume (skizy)
|
| 115 |
- **Model type:** bee
|
| 116 |
+
- **Architecture:** llama, F32 weights
|
| 117 |
+
- **Language(s):** buzz
|
| 118 |
- **License:** Apache 2.0
|
| 119 |
|
| 120 |
## Uses
|
| 121 |
|
| 122 |
+
Buzzing and pollinating flowers
|
| 123 |
|
| 124 |
+
## Bias, risks, and limitations
|
| 125 |
|
| 126 |
+
May sting
|
| 127 |
|
|
|
|
| 128 |
|
| 129 |
+
### Recommendations
|
| 130 |
|
| 131 |
+
Give flowers to pollinate and cheerful words
|
| 132 |
|
| 133 |
+
If you swat her, say sorry
|
| 134 |
|
| 135 |
+
## Training data
|
| 136 |
|
| 137 |
+
Flowers, meadows, and nice forests
|