Instructions to use whcl412/LycheeAI-coder-1b-II-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use whcl412/LycheeAI-coder-1b-II-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Use Docker

docker model run hf.co/whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Llama new
LM Studio
Jan
Ollama
How to use whcl412/LycheeAI-coder-1b-II-GGUF with Ollama:
```
ollama run hf.co/whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M
```
Unsloth Desktop

How to use whcl412/LycheeAI-coder-1b-II-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use whcl412/LycheeAI-coder-1b-II-GGUF with Docker Model Runner:
```
docker model run hf.co/whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M
```

Lemonade

How to use whcl412/LycheeAI-coder-1b-II-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.LycheeAI-coder-1b-II-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use whcl412/LycheeAI-coder-1b-II-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use whcl412/LycheeAI-coder-1b-II-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "whcl412/LycheeAI-coder-1b-II-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

whcl412 commited on Aug 30

Commit

debe49f

verified ·

1 Parent(s): fe87323

docs: 三档版本 + 如何下载指定版本

Browse files

Files changed (1) hide show

README.md +53 -7

README.md CHANGED Viewed

@@ -7,22 +7,68 @@
 ---
-## 文件
-| 文件 | 大小 | 说明 |
-|------|------|------|
-| **`mlx-LycheeAI-coder-1b-II.gguf`** | 2.0 GB | **f16 无损精度**，等同训练时的精度 |
-### 为什么只提供 f16
-实测发现 **1B 模型对量化极其敏感**，量化带来的质量损失远比大模型明显：
 | 问题 | f16（无损） | q8_0 | q4_k_m |
 |------|------------|------|--------|
 | 1+1=3 对吗 | 不是，1+1=2 ✅ | 答成 2+2=4 ⚠️ | 答成 2+2=4 ❌ |
 | Python 是编译型 | 不是，解释器执行 ✅ | 提到 GIL ⚠️ | 说 GIL 是编译器 ❌ |
-1B 模型推理本来就快，2GB 对现代设备毫无压力。**与其提供质量打折的小文件，不如只给最好的一个。** 如果你确实需要 4bit/8bit 版本，可以用 `llama-quantize` 自行量化，或者提 issue 告诉我。
 ---

 ---
+## 📦 三个版本，按设备选
+| 文件 | 大小 | 内存占用 | 用在哪 | 质量 |
+|------|------|---------|--------|------|
+| `lychee-coder2-v3-q4_k_m.gguf` | **656 MB** | ~1 GB | 📱 **老手机**（4GB 内存）、树莓派 | ⚠️ 有损 |
+| `lychee-coder2-v3-q8_0.gguf` | 1.1 GB | ~1.5 GB | 📱 中端手机、轻薄本 | ✅ 接近无损 |
+| `mlx-LycheeAI-coder-1b-II.gguf`（f16） | 2.0 GB | ~2.5 GB | 💻 Mac / PC、高端手机 | ✅ **无损** |
+**原则：设备扛得住就用 f16，空间紧张再往下退。**
+### 📱 手机能跑吗？能。
+1B 模型的优势就是小。q4 版只有 656MB，**一台 4GB 内存的老安卓机就能跑**：
+- **Android**：[MLC Chat](https://mlc.ai/mlc-chat/)（有 APK）或 Termux + llama.cpp
+- **iOS**：[MLC Chat](https://apps.apple.com/app/mlc-chat) 或 llama.cpp
+速度大约每秒十几个字，完全离线、不联网、不花 API 费用。
+### 📥 怎么下载指定版本
+**方法 1：命令行（推荐，可只下单个文件）**
+```bash
+# HuggingFace（需先 pip install huggingface_hub）
+huggingface-cli download whcl412/LycheeAI-coder-1b-II-GGUF \
+    lychee-coder2-v3-q4_k_m.gguf --local-dir ./
+# ModelScope（国内推荐，需先 pip install modelscope）
+modelscope download --model whcl412/LycheeAI-coder-1b-II-GGUF \
+    lychee-coder2-v3-q4_k_m.gguf --local_dir ./
+```
+把文件名换成你想要的即可（去掉文件名就是下载整个仓库）。
+**方法 2：直链下载**
+```bash
+# HuggingFace
+wget https://huggingface.co/whcl412/LycheeAI-coder-1b-II-GGUF/resolve/main/lychee-coder2-v3-q4_k_m.gguf
+# ModelScope
+wget https://www.modelscope.cn/models/whcl412/LycheeAI-coder-1b-II-GGUF/resolve/master/lychee-coder2-v3-q4_k_m.gguf
+```
+**方法 3：网页点下载**
+打开仓库页面的 **Files** 标签 → 点文件名 → 右上角 `Download`。
+---
+### ⚠️ 关于量化的实话
+1B 参数本来就小，**对量化比大模型敏感得多**。实测：
 | 问题 | f16（无损） | q8_0 | q4_k_m |
 |------|------------|------|--------|
 | 1+1=3 对吗 | 不是，1+1=2 ✅ | 答成 2+2=4 ⚠️ | 答成 2+2=4 ❌ |
 | Python 是编译型 | 不是，解释器执行 ✅ | 提到 GIL ⚠️ | 说 GIL 是编译器 ❌ |
+量化越低，事实细节越容易出错，但**日常聊天、写代码、简单问答依然可用**。
+要在老手机上跑，q4 是唯一现实的选择——**能跑起来比什么都强**。
 ---