Text Generation
MLX
Safetensors
llama
code
minicpm
chinese
conversational
tool-calling
quantization
4-bit precision
Instructions to use whcl412/LycheeAI-coder-2b-II-pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use whcl412/LycheeAI-coder-2b-II-pro with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("whcl412/LycheeAI-coder-2b-II-pro") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use whcl412/LycheeAI-coder-2b-II-pro with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "whcl412/LycheeAI-coder-2b-II-pro"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "whcl412/LycheeAI-coder-2b-II-pro" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use whcl412/LycheeAI-coder-2b-II-pro with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "whcl412/LycheeAI-coder-2b-II-pro"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "whcl412/LycheeAI-coder-2b-II-pro" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8080/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "whcl412/LycheeAI-coder-2b-II-pro", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use whcl412/LycheeAI-coder-2b-II-pro with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "whcl412/LycheeAI-coder-2b-II-pro"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default whcl412/LycheeAI-coder-2b-II-pro
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use whcl412/LycheeAI-coder-2b-II-pro with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "whcl412/LycheeAI-coder-2b-II-pro"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "whcl412/LycheeAI-coder-2b-II-pro" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
README: 新增 MLX 无损版 bf16 入口,明确 4bit 为有损、再加工需用无损版
Browse files
README.md
CHANGED
|
@@ -28,8 +28,19 @@ library_name: mlx
|
|
| 28 |
它最大的用处是:**跑在你自己的机器上,不用联网,能调工具,响应飞快。**
|
| 29 |
|
| 30 |
**本仓库是 MLX 4bit 量化版本**,1.4 GB,Apple Silicon 上开箱即用。
|
| 31 |
-
|
| 32 |
-
**要
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
---
|
| 35 |
|
|
@@ -58,7 +69,7 @@ library_name: mlx
|
|
| 58 |
```bash
|
| 59 |
pip install mlx-lm
|
| 60 |
|
| 61 |
-
# 直接加载
|
| 62 |
python -m mlx_lm.generate \
|
| 63 |
--model whcl412/LycheeAI-coder-2b-II-pro \
|
| 64 |
--prompt "帮我写个快速排序"
|
|
@@ -67,10 +78,14 @@ python -m mlx_lm.generate \
|
|
| 67 |
python -m mlx_lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
|
| 68 |
```
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
### transformers
|
| 71 |
|
| 72 |
本仓库是 **MLX 4bit 格式**,transformers 不能直接加载。
|
| 73 |
-
需要 HF safetensors 请用
|
| 74 |
|
| 75 |
### Ollama(GGUF)
|
| 76 |
|
|
@@ -470,8 +485,20 @@ The biggest gap in the previous release (1b-II) was that it **couldn't call tool
|
|
| 470 |
**New base model**: switched from MiniCPM5-1B to MiniCPM5-2B (OpenBMB, Apache 2.0, 512K context).
|
| 471 |
|
| 472 |
**This repo is the MLX 4-bit quantized build**, 1.4 GB, ready to run on Apple Silicon.
|
| 473 |
-
|
| 474 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 475 |
|
| 476 |
---
|
| 477 |
|
|
@@ -492,7 +519,7 @@ python -m mlx_lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
|
|
| 492 |
### transformers
|
| 493 |
|
| 494 |
This repo is in **MLX 4-bit format** and cannot be loaded by transformers.
|
| 495 |
-
Use the
|
| 496 |
|
| 497 |
### Ollama (GGUF)
|
| 498 |
|
|
|
|
| 28 |
它最大的用处是:**跑在你自己的机器上,不用联网,能调工具,响应飞快。**
|
| 29 |
|
| 30 |
**本仓库是 MLX 4bit 量化版本**,1.4 GB,Apple Silicon 上开箱即用。
|
| 31 |
+
|
| 32 |
+
**四个版本,按需要选:**
|
| 33 |
+
|
| 34 |
+
| 版本 | 体积 | 用在什么场景 |
|
| 35 |
+
|---|---|---|
|
| 36 |
+
| **MLX 4bit**(本仓库) | 1.42 GB | Mac 上日常跑,最省内存、最快 |
|
| 37 |
+
| [MLX 无损 bf16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16) | 5.03 GB | **要质量基准 / 二次量化 / 继续微调** |
|
| 38 |
+
| [-f16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16) | 5.03 GB | 同上(**同一份权重**,方便 transformers 用户找) |
|
| 39 |
+
| [GGUF](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-GGUF) | 1.45 / 2.50 / 4.69 GB | Ollama、llama.cpp |
|
| 40 |
+
|
| 41 |
+
> **MLX 无损版(bf16)**是给"要在它身上再加工"的人用的:4bit 是有损的,不能作为二次量化或续训的基础。
|
| 42 |
+
> 它和 `-f16` 仓库是**同一份权重**(字节相同),两个名字只是为了让 MLX 用户和 transformers 用户都能找到。
|
| 43 |
+
> 格式是标准 safetensors,**MLX 和 transformers 都能直接加载**。
|
| 44 |
|
| 45 |
---
|
| 46 |
|
|
|
|
| 69 |
```bash
|
| 70 |
pip install mlx-lm
|
| 71 |
|
| 72 |
+
# 直接加载(4bit,1.4 GB)
|
| 73 |
python -m mlx_lm.generate \
|
| 74 |
--model whcl412/LycheeAI-coder-2b-II-pro \
|
| 75 |
--prompt "帮我写个快速排序"
|
|
|
|
| 78 |
python -m mlx_lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
|
| 79 |
```
|
| 80 |
|
| 81 |
+
> **要在 MLX 上用无损版**(二次量化、继续微调、要质量基准),把模型名换成
|
| 82 |
+
> `whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16` 即可(5.03 GB,MLX 直接加载)。
|
| 83 |
+
> 手工拼 prompt 时记得套 chat template 并传 `enable_thinking=False`,否则会复读乱码。
|
| 84 |
+
|
| 85 |
### transformers
|
| 86 |
|
| 87 |
本仓库是 **MLX 4bit 格式**,transformers 不能直接加载。
|
| 88 |
+
需要 HF safetensors 请用无损版 → [`whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16)(或等价的 [`-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16))。
|
| 89 |
|
| 90 |
### Ollama(GGUF)
|
| 91 |
|
|
|
|
| 485 |
**New base model**: switched from MiniCPM5-1B to MiniCPM5-2B (OpenBMB, Apache 2.0, 512K context).
|
| 486 |
|
| 487 |
**This repo is the MLX 4-bit quantized build**, 1.4 GB, ready to run on Apple Silicon.
|
| 488 |
+
|
| 489 |
+
**Four builds, pick by use case:**
|
| 490 |
+
|
| 491 |
+
| Build | Size | Use case |
|
| 492 |
+
|---|---|---|
|
| 493 |
+
| **MLX 4-bit** (this repo) | 1.42 GB | Everyday use on a Mac — fastest, smallest footprint |
|
| 494 |
+
| [MLX lossless bf16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16) | 5.03 GB | **Quality baseline / re-quantization / further fine-tuning** |
|
| 495 |
+
| [-f16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16) | 5.03 GB | Same as above (**identical weights**, named for transformers users) |
|
| 496 |
+
| [GGUF](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-GGUF) | 1.45 / 2.50 / 4.69 GB | Ollama, llama.cpp |
|
| 497 |
+
|
| 498 |
+
> **The MLX lossless (bf16) build** is for anyone who needs to *build on top of* this model —
|
| 499 |
+
> 4-bit is lossy and unsuitable as a base for re-quantization or continued training.
|
| 500 |
+
> It holds **byte-identical weights** to the `-f16` repo; the two names exist so both MLX and
|
| 501 |
+
> transformers users can find it. It's a standard safetensors checkpoint — **loadable by MLX and transformers alike**.
|
| 502 |
|
| 503 |
---
|
| 504 |
|
|
|
|
| 519 |
### transformers
|
| 520 |
|
| 521 |
This repo is in **MLX 4-bit format** and cannot be loaded by transformers.
|
| 522 |
+
Use the lossless build for HF safetensors → [`whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16) (or the equivalent [`-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16)).
|
| 523 |
|
| 524 |
### Ollama (GGUF)
|
| 525 |
|