FrostWolfAI/Frost-Cyber-Mini-1.0-MLX-8bit

Frost-Cyber-Mini 1.0 — MLX (8-bit)

Apple-silicon-native MLX conversion of FrostWolfAI/Frost-Cyber-Mini-1.0, quantized to 8-bit for near-lossless quality.

Pick this if you have 32 GB or more of unified memory. On a 16 GB machine it works but runs at roughly half the speed of the 4-bit release for quality you probably won't notice on a 4B model.

MLX cannot load GGUF — if you want llama.cpp / Ollama / LM Studio's GGUF path, use Frost-Cyber-Mini-1.0-GGUF.

Model details

Base	Qwen3.5-4B text tower (finetuned from huihui-ai/Huihui-Qwen3.5-4B-abliterated)
Architecture	Hybrid attention, 32 layers (24 linear-attention + 8 full-attention)
Hidden / heads	2560 / 16, tied embeddings
Vocab	248,320
Quantization	8-bit affine, group size 64 — 8.502 bits/weight
Size on disk	4.2 GB
Format	MLX safetensors (model_type: qwen3_5)

Conversion note

The upstream repo declares model_type: "qwen3_5_text" (text-only extract of the multimodal Qwen3.5 checkpoint). mlx-lm resolves architectures by importing mlx_lm.models.<model_type> and has no qwen3_5_text module, so loading it directly fails with Model type qwen3_5_text not supported.

This conversion sets model_type to qwen3_5, the module that actually implements this architecture and correctly handles a flat text-only config. This repo loads on a stock mlx-lm install with no patching.

Requirements

• Apple silicon Mac, macOS 13.5+
• Python 3.9+
• mlx-lm >= 0.32
• 32 GB+ unified memory recommended

pip install -U mlx-lm

Usage

from mlx_lm import load, generate

model, tokenizer = load("FrostWolfAI/Frost-Cyber-Mini-1.0-MLX-8bit")

messages = [{"role": "user", "content": "Explain how parameterized queries stop SQL injection."}]

# thinking disabled — direct answer
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False, enable_thinking=False
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=400))

mlx_lm.generate --model FrostWolfAI/Frost-Cyber-Mini-1.0-MLX-8bit \
  --prompt "Explain the OWASP Top 10 A01:2021." --max-tokens 400

mlx_lm.server --model FrostWolfAI/Frost-Cyber-Mini-1.0-MLX-8bit --port 8080

Thinking mode

Qwen3.5 is a hybrid reasoning model and reasons by default — output begins with a thinking block before the answer. For short or tool-style responses set enable_thinking=False in apply_chat_template, or pass "chat_template_kwargs": {"enable_thinking": false} to the server. With thinking left on, budget generous max_tokens or responses will truncate mid-reasoning.

Performance

Measured on Apple M3 (8-core, 16 GB), mlx-lm 0.32.2, macOS 27:

Stage	Throughput
Prompt processing	~119 tok/s
Generation	~19 tok/s
Peak memory	~4.5 GB

What to expect on larger chips

Decoding is memory-bandwidth-bound, so throughput tracks memory bandwidth ÷ 4.2 GB. Estimates below are extrapolations, not measurements:

Chip	Bandwidth	Est. decode
M3 (measured)	~100 GB/s	19 tok/s
M4 Pro	~273 GB/s	~50 tok/s
M4 Max	~546 GB/s	~95–110 tok/s
M5 Max / Ultra	higher	110+ tok/s

On these chips 8-bit is the better default: the model fits easily, and there's little reason to accept 4-bit quantization error.

Intended use and license

Inherits CC-BY-NC-SA-4.0 from the base model: non-commercial use only, share-alike (the same license applies to this conversion).

This is an abliterated offensive-security model with reduced safety filtering — it will produce exploit code, payloads, and attacker tradecraft. Use it only for authorized security testing, defensive research, and education on systems you own or have explicit permission to test.