whcl412 commited on
Commit
996c47a
·
verified ·
1 Parent(s): 60646cc

README: 新增 MLX 无损版 bf16 入口,明确 4bit 为有损、再加工需用无损版

Browse files
Files changed (1) hide show
  1. README.md +34 -7
README.md CHANGED
@@ -28,8 +28,19 @@ library_name: mlx
28
  它最大的用处是:**跑在你自己的机器上,不用联网,能调工具,响应飞快。**
29
 
30
  **本仓库是 MLX 4bit 量化版本**,1.4 GB,Apple Silicon 上开箱即用。
31
- 需要全精度(二次量化、继续微调)→ [`whcl412/LycheeAI-coder-2b-II-pro-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16)(bfloat16,5.03 GB)。
32
- **要用 Ollama / llama.cpp** → [`whcl412/LycheeAI-coder-2b-II-pro-GGUF`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-GGUF)(q4_k_m / q8_0 / f16 三档)。
 
 
 
 
 
 
 
 
 
 
 
33
 
34
  ---
35
 
@@ -58,7 +69,7 @@ library_name: mlx
58
  ```bash
59
  pip install mlx-lm
60
 
61
- # 直接加载
62
  python -m mlx_lm.generate \
63
  --model whcl412/LycheeAI-coder-2b-II-pro \
64
  --prompt "帮我写个快速排序"
@@ -67,10 +78,14 @@ python -m mlx_lm.generate \
67
  python -m mlx_lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
68
  ```
69
 
 
 
 
 
70
  ### transformers
71
 
72
  本仓库是 **MLX 4bit 格式**,transformers 不能直接加载。
73
- 需要 HF safetensors 请用 f16 仓库 → [`whcl412/LycheeAI-coder-2b-II-pro-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16)。
74
 
75
  ### Ollama(GGUF)
76
 
@@ -470,8 +485,20 @@ The biggest gap in the previous release (1b-II) was that it **couldn't call tool
470
  **New base model**: switched from MiniCPM5-1B to MiniCPM5-2B (OpenBMB, Apache 2.0, 512K context).
471
 
472
  **This repo is the MLX 4-bit quantized build**, 1.4 GB, ready to run on Apple Silicon.
473
- For full precision (re-quantization, further fine-tuning) → [`whcl412/LycheeAI-coder-2b-II-pro-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16) (bfloat16, 5.03 GB).
474
- **For Ollama / llama.cpp** → [`whcl412/LycheeAI-coder-2b-II-pro-GGUF`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-GGUF) (q4_k_m / q8_0 / f16).
 
 
 
 
 
 
 
 
 
 
 
 
475
 
476
  ---
477
 
@@ -492,7 +519,7 @@ python -m mlx_lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
492
  ### transformers
493
 
494
  This repo is in **MLX 4-bit format** and cannot be loaded by transformers.
495
- Use the f16 repo for HF safetensors → [`whcl412/LycheeAI-coder-2b-II-pro-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16).
496
 
497
  ### Ollama (GGUF)
498
 
 
28
  它最大的用处是:**跑在你自己的机器上,不用联网,能调工具,响应飞快。**
29
 
30
  **本仓库是 MLX 4bit 量化版本**,1.4 GB,Apple Silicon 上开箱即用。
31
+
32
+ **四个版本,按需要选:**
33
+
34
+ | 版本 | 体积 | 用在什么场景 |
35
+ |---|---|---|
36
+ | **MLX 4bit**(本仓库) | 1.42 GB | Mac 上日常跑,最省内存、最快 |
37
+ | [MLX 无损 bf16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16) | 5.03 GB | **要质量基准 / 二次量化 / 继续微调** |
38
+ | [-f16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16) | 5.03 GB | 同上(**同一份权重**,方便 transformers 用户找) |
39
+ | [GGUF](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-GGUF) | 1.45 / 2.50 / 4.69 GB | Ollama、llama.cpp |
40
+
41
+ > **MLX 无损版(bf16)**是给"要在它身上再加工"的人用的:4bit 是有损的,不能作为二次量化或续训的基础。
42
+ > 它和 `-f16` 仓库是**同一份权重**(字节相同),两个名字只是为了让 MLX 用户和 transformers 用户都能找到。
43
+ > 格式是标准 safetensors,**MLX 和 transformers 都能直接加载**。
44
 
45
  ---
46
 
 
69
  ```bash
70
  pip install mlx-lm
71
 
72
+ # 直接加载(4bit,1.4 GB)
73
  python -m mlx_lm.generate \
74
  --model whcl412/LycheeAI-coder-2b-II-pro \
75
  --prompt "帮我写个快速排序"
 
78
  python -m mlx_lm.server --model whcl412/LycheeAI-coder-2b-II-pro --port 8080
79
  ```
80
 
81
+ > **要在 MLX 上用无损版**(二次量化、继续微调、要质量基准),把模型名换成
82
+ > `whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16` 即可(5.03 GB,MLX 直接加载)。
83
+ > 手工拼 prompt 时记得套 chat template 并传 `enable_thinking=False`,否则会复读乱码。
84
+
85
  ### transformers
86
 
87
  本仓库是 **MLX 4bit 格式**,transformers 不能直接加载。
88
+ 需要 HF safetensors 请用无损版 → [`whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16)(或等价的 [`-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16))。
89
 
90
  ### Ollama(GGUF)
91
 
 
485
  **New base model**: switched from MiniCPM5-1B to MiniCPM5-2B (OpenBMB, Apache 2.0, 512K context).
486
 
487
  **This repo is the MLX 4-bit quantized build**, 1.4 GB, ready to run on Apple Silicon.
488
+
489
+ **Four builds, pick by use case:**
490
+
491
+ | Build | Size | Use case |
492
+ |---|---|---|
493
+ | **MLX 4-bit** (this repo) | 1.42 GB | Everyday use on a Mac — fastest, smallest footprint |
494
+ | [MLX lossless bf16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16) | 5.03 GB | **Quality baseline / re-quantization / further fine-tuning** |
495
+ | [-f16](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16) | 5.03 GB | Same as above (**identical weights**, named for transformers users) |
496
+ | [GGUF](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-GGUF) | 1.45 / 2.50 / 4.69 GB | Ollama, llama.cpp |
497
+
498
+ > **The MLX lossless (bf16) build** is for anyone who needs to *build on top of* this model —
499
+ > 4-bit is lossy and unsuitable as a base for re-quantization or continued training.
500
+ > It holds **byte-identical weights** to the `-f16` repo; the two names exist so both MLX and
501
+ > transformers users can find it. It's a standard safetensors checkpoint — **loadable by MLX and transformers alike**.
502
 
503
  ---
504
 
 
519
  ### transformers
520
 
521
  This repo is in **MLX 4-bit format** and cannot be loaded by transformers.
522
+ Use the lossless build for HF safetensors → [`whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-MLX-bf16) (or the equivalent [`-f16`](https://huggingface.co/whcl412/LycheeAI-coder-2b-II-pro-f16)).
523
 
524
  ### Ollama (GGUF)
525