Instructions to use LiquidAI/LFM2.5-Encoder-230M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16 # Run inference directly in the terminal: llama cli -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16 # Run inference directly in the terminal: llama cli -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16 # Run inference directly in the terminal: ./build/bin/llama cli -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Use Docker
docker model run hf.co/LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
- Llama new
- LM Studio
- Jan
- Ollama
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with Ollama:
ollama run hf.co/LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
- Unsloth Desktop
- Pi
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "LiquidAI/LFM2.5-Encoder-230M-GGUF:F16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with Docker Model Runner:
docker model run hf.co/LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
- Lemonade
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Run and chat with the model
lemonade run user.LFM2.5-Encoder-230M-GGUF-F16
List all available models
lemonade list
- Hermes Agent
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use LiquidAI/LFM2.5-Encoder-230M-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf LiquidAI/LFM2.5-Encoder-230M-GGUF:F16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "LiquidAI/LFM2.5-Encoder-230M-GGUF:F16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
GGUF release: F16/Q8_0/Q4_0 + card aligned with the encoder release card
#1
by v4zhong - opened
- .gitattributes +3 -0
- LFM2.5-Encoder-230M-F16.gguf +3 -0
- LFM2.5-Encoder-230M-Q4_0.gguf +3 -0
- LFM2.5-Encoder-230M-Q8_0.gguf +3 -0
- LICENSE +2 -2
- README.md +55 -27
.gitattributes
CHANGED
|
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
LFM2.5-Encoder-230M-F16.gguf filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
LFM2.5-Encoder-230M-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
LFM2.5-Encoder-230M-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
LFM2.5-Encoder-230M-F16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1516abfdb4ac0befcad6aa547204e1d26e8b9c83da46891a5fc289a6280e5c53
|
| 3 |
+
size 461882336
|
LFM2.5-Encoder-230M-Q4_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:476b1347517eee013337ad77779d2ee32c4df308f19bccef83379d040b66c868
|
| 3 |
+
size 149079008
|
LFM2.5-Encoder-230M-Q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0e92be849493abd5c0ff356416fffadf42cb52cacc717181e75f6be6ba1838e4
|
| 3 |
+
size 246596576
|
LICENSE
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
LFM Open License v1.0
|
| 2 |
|
| 3 |
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 4 |
|
|
@@ -68,4 +68,4 @@ You may add Your own copyright statement to Your modifications and may provide a
|
|
| 68 |
|
| 69 |
11. Termination. This License will terminate automatically and immediately if You fail to comply with any of its terms and conditions. Upon termination, You must cease all use of the Work and any Derivative Works and delete all copies in Your possession.
|
| 70 |
|
| 71 |
-
END OF TERMS AND CONDITIONS
|
|
|
|
| 1 |
+
LFM Open License v1.0
|
| 2 |
|
| 3 |
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 4 |
|
|
|
|
| 68 |
|
| 69 |
11. Termination. This License will terminate automatically and immediately if You fail to comply with any of its terms and conditions. Upon termination, You must cease all use of the Work and any Derivative Works and delete all copies in Your possession.
|
| 70 |
|
| 71 |
+
END OF TERMS AND CONDITIONS
|
README.md
CHANGED
|
@@ -1,52 +1,80 @@
|
|
| 1 |
---
|
| 2 |
-
library_name: gguf
|
| 3 |
-
license: other
|
| 4 |
-
license_name: lfm1.0
|
| 5 |
-
license_link: LICENSE
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
-
- ar
|
| 9 |
-
- zh
|
| 10 |
-
- fr
|
| 11 |
- de
|
| 12 |
-
- ja
|
| 13 |
-
- ko
|
| 14 |
- es
|
| 15 |
-
-
|
| 16 |
- it
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
tags:
|
| 20 |
- liquid
|
|
|
|
| 21 |
- lfm2.5
|
| 22 |
-
-
|
|
|
|
|
|
|
| 23 |
- llama.cpp
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
---
|
| 25 |
|
| 26 |
-
<
|
| 27 |
-
|
| 28 |
-
|
|
|
|
| 29 |
alt="Liquid AI"
|
| 30 |
style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
|
| 31 |
/>
|
| 32 |
-
<div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;">
|
| 33 |
-
<a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> •
|
| 34 |
-
<a href="https://docs.liquid.ai/lfm/getting-started/welcome"><strong>Docs</strong></a> •
|
| 35 |
-
<a href="https://leap.liquid.ai/"><strong>LEAP</strong></a> •
|
| 36 |
-
<a href="https://discord.com/invite/liquid-ai"><strong>Discord</strong></a>
|
| 37 |
-
</div>
|
| 38 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
-
# LFM2.5-Encoder-230M
|
| 41 |
|
| 42 |
-
LFM2 is a
|
| 43 |
|
| 44 |
-
|
|
|
|
|
|
|
| 45 |
|
| 46 |
-
|
|
|
|
|
|
|
| 47 |
|
| 48 |
Example usage with [llama.cpp](https://github.com/ggml-org/llama.cpp):
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
```
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
```
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
|
|
|
|
|
|
|
|
|
| 4 |
- de
|
|
|
|
|
|
|
| 5 |
- es
|
| 6 |
+
- fr
|
| 7 |
- it
|
| 8 |
+
- nl
|
| 9 |
+
- pl
|
| 10 |
+
- pt
|
| 11 |
+
- ar
|
| 12 |
+
- hi
|
| 13 |
+
- ja
|
| 14 |
+
- ru
|
| 15 |
+
- tr
|
| 16 |
+
- vi
|
| 17 |
+
- zh
|
| 18 |
tags:
|
| 19 |
- liquid
|
| 20 |
+
- lfm2
|
| 21 |
- lfm2.5
|
| 22 |
+
- bidirectional
|
| 23 |
+
- masked-lm
|
| 24 |
+
- encoder
|
| 25 |
- llama.cpp
|
| 26 |
+
- gguf
|
| 27 |
+
pipeline_tag: fill-mask
|
| 28 |
+
license: other
|
| 29 |
+
license_name: lfm1.0
|
| 30 |
+
license_link: LICENSE
|
| 31 |
+
base_model:
|
| 32 |
+
- LiquidAI/LFM2.5-Encoder-230M
|
| 33 |
---
|
| 34 |
|
| 35 |
+
<center>
|
| 36 |
+
<div style="text-align: center;">
|
| 37 |
+
<img
|
| 38 |
+
src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png"
|
| 39 |
alt="Liquid AI"
|
| 40 |
style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;"
|
| 41 |
/>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
</div>
|
| 43 |
+
<div style="display: flex; justify-content: center; gap: 0.5em;">
|
| 44 |
+
<a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> • <a href="https://docs.liquid.ai/lfm"><strong>Documentation</strong></a> • <a href="https://leap.liquid.ai/"><strong>LEAP</strong></a>
|
| 45 |
+
</div>
|
| 46 |
+
</center>
|
| 47 |
|
| 48 |
+
# LFM2.5-Encoder-230M
|
| 49 |
|
| 50 |
+
LFM2.5-Encoder-230M is a multilingual bidirectional encoder built on the LFM2 architecture — a lightweight encoder for tight latency and memory budgets, punching above its size class. It is a masked language model with full bidirectional attention, designed to be fine-tuned into task-specific models (classification, token classification, retrieval, reranking, and semantic similarity) across 15 languages, and to run efficiently on-device.
|
| 51 |
|
| 52 |
+
- **Highly capable for its size.** On par with the best similarly sized encoders and well ahead of our own retrieval siblings.
|
| 53 |
+
- **General-purpose.** 8k context, strong across NLI, paraphrase, sentiment, and multilingual tasks.
|
| 54 |
+
- **Fast and on-device.** Matches or beats ModernBERT throughput, with a long-context edge on CPU; runs in the browser on WebGPU.
|
| 55 |
|
| 56 |
+
Find more information about LFM2.5-Encoder-230M in our [blog post](https://www.liquid.ai/blog/lfm2-5-encoders).
|
| 57 |
+
|
| 58 |
+
## 🏃 How to run
|
| 59 |
|
| 60 |
Example usage with [llama.cpp](https://github.com/ggml-org/llama.cpp):
|
| 61 |
|
| 62 |
+
Run masked-token prediction — the prompt's literal `[MASK]` is replaced with the model's mask token, one non-causal forward pass is run, and the top-K predictions at the mask position are printed:
|
| 63 |
+
|
| 64 |
+
```bash
|
| 65 |
+
llama-fill-mask -hf LiquidAI/LFM2.5-Encoder-230M-GGUF "The capital of France is [MASK]." 5
|
| 66 |
+
# 1 16.17 ' Paris'
|
| 67 |
+
# 2 13.41 ' Strasbourg'
|
| 68 |
+
# 3 13.35 'Paris'
|
| 69 |
+
# 4 13.18 ' Lyon'
|
| 70 |
+
# 5 11.87 ' Versailles'
|
| 71 |
```
|
| 72 |
+
|
| 73 |
+
For downstream use, the encoder body also serves per-token embeddings:
|
| 74 |
+
|
| 75 |
+
```bash
|
| 76 |
+
llama-server -hf LiquidAI/LFM2.5-Encoder-230M-GGUF --embeddings
|
| 77 |
+
curl -s http://localhost:8080/embedding -d '{"content": "hello world"}'
|
| 78 |
```
|
| 79 |
+
|
| 80 |
+
Find more details in the original model card: https://huggingface.co/LiquidAI/LFM2.5-Encoder-230M
|