Hugging Face's logo

YCF-AI
/
CyberTiel-Coder-35B-MLX

Image-Text-to-Text
MLX
Safetensors
English
Chinese
qwen3_5_moe
omlx
apple-silicon
m1
oq
qwen35moe
Mixture of Experts
mtp
speculative-decoding
fixed-mtp
fp16
agentic-coding
abliterated
uncensored
vision
conversational
6-bit
Model card Files
xet
 Community
CyberTiel-Coder-35B-MLX / README.md
YCF-AI's picture
YCF-AI
Refresh the local-stack overview figure (BGE-M3 + BGE-Reranker replace Qwen3-Embedding) and link the new retrieval repos
2d616f3 verified
preview code
|
Raw History Blame Contribute Delete
14.9 kB
metadata 
license: mit
base_model:
  - peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
language:
  - en
  - zh
tags:
  - mlx
  - omlx
  - apple-silicon
  - m1
  - oq
  - qwen35moe
  - moe
  - mtp
  - speculative-decoding
  - fixed-mtp
  - fp16
  - agentic-coding
  - abliterated
  - uncensored
  - vision

CyberTiel-Coder-35B-MLX (FP16 tensors · fixed MTP head · tuned for oMLX on Apple Silicon)

English · 中文

A 35B-A3B MoE coding/agent model (6-bit oQ6e, vision-capable, abliterated) in the exact form we run every day on a Mac Studio M1 Ultra 64 GB with oMLX 0.7.0. Compared with the upstream pack it has two local changes, both measured:

  1. Non-quantized BF16 tensors cast to FP16 (M1/M2 have no native bfloat16). Prefill +28–50 %, decode at 16K context +22 %, quality unchanged (206/260 = 206/260).
  2. Rebuilt MTP head ("fixedmtp": 42-key fused layout) so that oMLX's Lightning MTP speculative decoding runs with an adaptive depth → 94–116 tok/s chat decode.

Everything below is measured on one machine; numbers are not a promise for yours. Full method and raw data: tuning notes dataset · GitHub.

⚠️ Abliterated (refusal-removed) model. Meant for emergency / red-team / research work. Run it inside an OS-level sandbox with restricted network and execution rights, and guard against prompt injection. You are responsible for how you use it.

local stack overview

Updated 2026-10-10: the retrieval pair of this stack is now BGE-M3 8-bit + bge-reranker-v2-m3 (FP16 embedding tables); the previous embedding model, Qwen3-Embedding-0.6B, was retired. 更新于 2026-10-10：本机栈的检索组合现为上述 BGE-M3 8-bit + bge-reranker-v2-m3，旧嵌入模型 Qwen3-Embedding-0.6B 已下线。

At a glance

Item Value
Architecture Qwen3.5-MoE style, 35B total / ~3B active (256 experts, 8 active), vision tower included
Quantization oQ6e mixed (≈6-bit) + non-quantized tensors in FP16 (this repo)
Size on disk ≈ 28 GB (6 shards + MTP head)
Context 131072 configured (native 262144); 16K / 32K / 64K / 128K measured below
Max output 32768 (configured; context and output share one budget)
Speed (M1 Ultra, chat) 107–116 tok/s decode; prefill 4K ≈ 1890, 16K ≈ 1640 tok/s
Thinking On by default; reasoning_effort none/low/medium/xhigh works
Vision Yes (emergency use: OCR digits sometimes wrong, counting weak)
Tool calling Yes
License MIT (upstream card; see LICENSE, README-upstream.md)

Lineage: ornith-ai/Ornith-1.5-35B-A3B → huihui-ai/Huihui-Ornith-…-abliterated (refusal removal) → peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP (rev a443d4e) → our fixedmtp head → FP16 tensor cast (this repo). The previous BF16 build of this repo stays reachable as revision 32e067dd15c07fbfcf841d53efd011f90c6527b7 (recommended for Macs with native bf16, M3 and later — untested by us).

Measured results

1. BF16 → FP16 on M1 (the biggest free win)

fp16 cast

Metric (ABBA, same session) BF16 (as published) FP16 (this repo)
Prefill 1K / 4K / 16K (tok/s) 1085 / 1406 / 1096 1393 / 1891 / 1644
First token at 16K prompt 14.9 s 10.0 s
Decode at 16K context 96 tok/s 117 tok/s
Chat decode (3 prompts) 102.5 / 108.7 109.1 / 112.3
260-question quality set (T=0) 206 / 260 206 / 260

What was cast: 1569 tensors (scales, biases, norms, embeddings, MTP head, vision tower; 2.94 GB). Max |x| = 25.4 (far inside the FP16 range), 26 103 tiny values underflowed to zero. The quantized U32 data and model.safetensors.index.json are unchanged. Script: recipe/to_fp16_v2.py (aborts if any value would overflow). A pure short-decode test showed no gain and misled us once — always measure prefill at 1K/4K/16K and 16K decode.

2. MTP depth (Lightning MTP, chat decode tok/s, BF16 baseline sweep)

mtp depth

Leave mtp_fixed_depth and mtp_adaptive_max_depth empty (0.7.0 model default = adaptive): 94.2 tok/s vs 89.8 for fixed d2, 84.7 for d3, 59.4 with MTP off. Note: oMLX 0.7.0 silently ignores the old mtp_num_draft_tokens field.

3. Long context (hot cache off)

Context Prefill tok/s Decode tok/s
16K 1620 ~100
32K 1320 86–89
64K 957 71
128K 614 (first token 3.6 min) 60

Peak memory with this model alone at 128K: 36.8 GB (fits the default 48 GB Metal limit on a 64 GB Mac).

4. Code quality depends on the thinking mode you use

thinking flip

We rejected a newer upstream requant (09-29) and a rival code model after testing in the thinking-on production mode (LiveCodeBench-60 subset 38 vs 40; 324 code questions 247 vs 242, p = 0.30). Benchmarking with thinking off would have picked the wrong model.

5. What we tried and rejected for this model

TurboQuant KV (decode −20 %), ANE prefill (slower), DFlash / SpecPrefill (no gain), model_type_override=llm to save memory (decode −23 %, image requests silently hallucinate — never do this), hot cache on 64 GB (starves long-context admission).

Recommended oMLX settings

Per-model entry: omlx/model_settings.entry.json (key = model directory name).

{
  "max_context_window": 131072, "max_tokens": 32768,
  "temperature": 0.6, "top_p": 0.95, "top_k": 20, "repetition_penalty": 1.0,
  "enable_thinking": true, "thinking_budget_enabled": false,
  "mtp_enabled": true, "vlm_mtp_enabled": false,
  "ttl_seconds": 1800, "model_alias": "CyberTiel-35B"
}

Global (see the dataset repo's config/ for the full redacted files): hot cache off (hot_cache_max_size: "0"), SSD cache on, max_concurrent_requests: 8, memory_guard_tier: balanced with soft_threshold left at 0.85 (0.70 legacy values make models evict each other), TurboQuant/DFlash/SpecPrefill/ANE prefill all off.

Thinking control

Goal Request field
Thinking off "reasoning_effort": "none" (or chat_template_kwargs: {"enable_thinking": false})
Effort "reasoning_effort": "low" | "medium" | "xhigh" (high/max = xhigh); default medium
Where the reasoning goes separate reasoning_content field

Quick start 

# 1) download (mirror example for mainland China)
HF_ENDPOINT=https://hf-mirror.com hf download YCF-AI/CyberTiel-Coder-35B-MLX \
  --local-dir ~/models/CyberTiel-Coder-35B-MLX
# 2) point oMLX model_dirs at ~/models, merge omlx/model_settings.entry.json into ~/.omlx/model_settings.json, restart oMLX
# 3) chat
curl http://127.0.0.1:8000/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"CyberTiel-35B","messages":[{"role":"user","content":"Write a Python LRU cache."}],"reasoning_effort":"low"}'

The MTP head (model-mtp.safetensors) is used by oMLX's Lightning MTP; we have not tested other runtimes with it. Verify downloaded files with the SHA-256 values in the HF file listing.

Moving to another machine — checklist

  1. Install the same oMLX version (0.7.0 or newer), put the model folder under a model_dirs path.
  2. Merge the settings entry above; keep MTP depth fields empty.
  3. Disable hot cache, keep concurrency 8, keep the memory guard at balanced.
  4. Measure your machine: 3-prompt chat decode, prefill 1K/4K/16K, 16K decode, then (only if you changed something) a fixed quality set with McNemar. Our protocol is in the dataset repo.
  5. If the chip has native bf16 (M3+), compare the BF16 revision with this one before choosing.

Known limits

  • Vision is an emergency capability: digit OCR errors, weak counting. Use a dedicated VLM for precision (MiniCPM-V-4.6 build).
  • Abliteration removes refusals, not hallucinations.
  • All speed numbers: Mac Studio M1 Ultra 64-core GPU, 64 GB, macOS 15.8, oMLX 0.7.0 (build 2987), MLX 0.32.2, mlx-lm 0.31.4.dev, mlx-vlm 0.7.1.

中文说明

English · 中文

35B-A3B MoE 代码/智能体模型（6-bit oQ6e、带视觉、去审查），就是我们每天在 Mac Studio M1 Ultra 64 GB 上用 oMLX 0.7.0 实际运行的版本。相对上游打包有两处本地改动，均有实测：

  1. 非量化 BF16 张量转 FP16（M1/M2 没有原生 bf16）：prefill **+28–50 %**、16K 上下文解码 **+22 %**，质量不变（260 题 206 = 206）。
  2. 重建 MTP 头（fixedmtp，42 键融合布局），配合 oMLX Lightning MTP 自适应深度，对话解码 94–116 tok/s。

所有数字来自同一台机器，不保证在你的机器上一致。完整方法与原始数据见调优笔记数据集、GitHub。

⚠️ 去审查（abliterated）模型，仅用于应急测试 / 红队 / 研究。请放进 OS 级沙箱，限制网络与执行权限，并防范提示注入；使用责任自负。

概览

项目 值
架构 Qwen3.5-MoE 系，总参数 35B / 激活约 3B（256 专家激活 8），含视觉塔
量化 oQ6e 混合（约 6-bit）+ 非量化张量 FP16（本仓库）
体积 约 28 GB（6 个分片 + MTP 头）
上下文 配置 131072（原生 262144）；16K/32K/64K/128K 实测见下
最大输出 32768（上下文与输出共用一个预算）
速度（M1 Ultra） 解码 107–116 tok/s；prefill 4K ≈ 1890、16K ≈ 1640 tok/s
思考模式 默认开；reasoning_effort 支持 none/low/medium/xhigh
视觉 有（应急：OCR 数字偶错、计数弱）
工具调用 有
许可 MIT（上游卡片；见 LICENSE、README-upstream.md）

血统： ornith-ai/Ornith-1.5-35B-A3B → huihui-ai/Huihui-Ornith-…-abliterated (refusal removal) → peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-MLX-oQ6e-MTP（rev a443d4e）→ 自建 fixedmtp 头 → FP16 张量转换（本仓库）。本仓库此前的 BF16 版仍可用 revision 32e067dd15c07fbfcf841d53efd011f90c6527b7 取回（原生支持 bf16 的 M3 及以后机型可优先对比，我们未测）。

实测结果

1. BF16 → FP16（M1 上最大的免费收益）——图见上（英文部分 fp16 cast）。同时段 ABBA：prefill 1K/4K/16K 1085/1406/1096 → 1393/1891/1644 tok/s；16K 首字 14.9 → 10.0 s；16K 上下文解码 96 → 117；对话解码 102.5/108.7 → 109.1/112.3；260 题 206 = 206。 转换了 1569 个张量（scales、biases、norm、embedding、MTP 头、视觉塔；2.94 GB），最大绝对值 25.4，26103 个极小值下溢为 0；量化 U32 数据与 index 不变。脚本 recipe/to_fp16_v2.py（任何值溢出即中止）。只测短上下文纯解码会得出"无收益"的错误结论，必须测 1K/4K/16K prefill 与 16K 解码。

2. MTP 深度——mtp_fixed_depth 与 mtp_adaptive_max_depth 留空（0.7.0 默认=自适应）最优：94.2 tok/s，固定 d2 89.8、d3 84.7、关 MTP 59.4。注意 0.7.0 已静默忽略旧字段 mtp_num_draft_tokens。

3. 长上下文（hot cache 关闭）：16K 1620/~100、32K 1320/86–89、64K 957/71、128K 614（首字 3.6 分钟）/60（prefill/解码 tok/s）。单模型 128K 峰值内存 36.8 GB，64 GB Mac 默认 48 GB Metal 上限内可用。

4. 代码质量取决于你实际使用的思考模式：我们用开思考的生产口径否决了上游 09-29 重量化版与另一款代码模型（LCB60 38 vs 40；324 题 247 vs 242，p=0.30）；关思考评测会选错模型。

5. 试过并否决：TurboQuant KV（解码 −20 %）、ANE prefill（更慢）、DFlash/SpecPrefill（无收益）、model_type_override=llm 省内存（解码 −23 %、图片请求静默幻觉，切勿使用）、64 GB 上开 hot cache（挤占长上下文准入）。

推荐 oMLX 设置

单模型条目见 omlx/model_settings.entry.json（键名=模型目录名），参数见上方英文部分 JSON。全局：hot cache 关（hot_cache_max_size: "0"）、SSD 缓存开、max_concurrent_requests: 8、memory_guard_tier: balanced 且 soft_threshold 保持 0.85（遗留的 0.70 会让模型互相驱逐）、TurboQuant/DFlash/SpecPrefill/ANE prefill 全关。完整脱敏配置见数据集仓库 config/。

思考控制

目标 请求字段
关闭思考 "reasoning_effort": "none"（或 chat_template_kwargs: {"enable_thinking": false}）
力度 "reasoning_effort": "low" | "medium" | "xhigh"（high/max 等同 xhigh），默认 medium
推理内容 走独立的 reasoning_content 字段

快速开始 

# 下载（国内用镜像、直连不走代理）
HF_ENDPOINT=https://hf-mirror.com hf download YCF-AI/CyberTiel-Coder-35B-MLX --local-dir ~/models/CyberTiel-Coder-35B-MLX
# 把 ~/models 加入 oMLX 的 model_dirs，合并 omlx/model_settings.entry.json 到 ~/.omlx/model_settings.json，重启 oMLX

MTP 头 model-mtp.safetensors 由 oMLX Lightning MTP 使用，其他运行时我们未测。下载后请用 HF 文件列表中的 SHA-256 校验。

换机器部署清单

  1. 安装同版本 oMLX（0.7.0 或更新），把模型目录放进 model_dirs。
  2. 合并上面的设置条目，MTP 深度字段留空。
  3. 关 hot cache、并发 8、内存守卫 balanced。
  4. 在你的机器上实测：3 提示词对话解码、1K/4K/16K prefill、16K 解码；若改了东西，再用固定题集 + McNemar 检验。协议见数据集仓库。
  5. 芯片支持原生 bf16（M3+）时，先对比 BF16 版本再选。

已知限制

  • 视觉仅应急：数字 OCR 会错、计数弱；精确任务请用专用 VLM（MiniCPM-V-4.6 版）。
  • 去审查只去掉拒答，不去掉幻觉。
  • 全部速度数据：Mac Studio M1 Ultra 64 核 GPU、64 GB、macOS 15.8、oMLX 0.7.0（build 2987）、MLX 0.32.2、mlx-lm 0.31.4.dev、mlx-vlm 0.7.1。