hermitdave/Hy-Memory-30B-A3B-MLX-8bit

Hy-Memory-30B-A3B — MLX 8-bit

8-bit MLX conversion of tencent/Hy-Memory-30B-A3B, the memory-writer model from Tencent Hunyuan.

Hy-Memory reads a conversation and decides what is worth remembering — names, preferences, constraints, recurring facts — returning them as JSON for an external memory framework (Mem0, MemOS, Hy-Memory). It is a 30B MoE with ~3B active per token.

8.0 bits per weight · 30 GB · effectively lossless

What this is for

This is the quality reference. At 8-bit the model is as close to the bf16 original as makes no practical difference, so it is the right choice when you want maximum extraction fidelity and have the memory to spare — and the right baseline for measuring a smaller quant against.

If you want the same behaviour in 57% of the disk and RAM, use hermitdave/Hy-Memory-30B-A3B-MLX-Mixed4-6 instead: it measured +0.94% perplexity against this model and scored identically (100%) on our extraction eval.

Benchmarks

Measured on an M3 Max (64 GB), wikitext-2, 8 × 256-token deterministic windows:

	8-bit	Mixed 4/6
Size	30 GB	17 GB
Perplexity	14.783	14.923
Extraction recall (15 cases / 70 facts)	100%	100%
Load time	7.0 s	3.3 s
Decode	~51 tok/s	~65 tok/s

Quickstart

pip install -U mlx-lm

from mlx_lm import load, stream_generate

model, tokenizer = load("hermitdave/Hy-Memory-30B-A3B-MLX-8bit")

messages = [
    {
        "role": "system",
        "content": (
            "Extract explicit user facts and preferences from the conversation. "
            "Write each memory as a concise, self-contained statement. "
            "Do not infer or invent information. "
            'Return only a JSON object with a "memories" list of strings.'
        ),
    },
    {
        "role": "user",
        "content": (
            "My name is Alex. I live in Seattle and work as a software engineer. "
            "I mainly use Python and prefer concise answers with code examples. "
            "I'm vegetarian, so please keep that in mind when suggesting meals."
        ),
    },
]

prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_dict=False)
for r in stream_generate(model, tokenizer, prompt=prompt, max_tokens=512):
    print(r.text, end="", flush=True)

Or serve it OpenAI-compatible:

mlx_lm.server --model hermitdave/Hy-Memory-30B-A3B-MLX-8bit --port 8080

Recommended: temperature=0.0 for extraction.

Quantization

Uniform 8-bit, group size 64, affine mode. Norms and the router's expert_bias stay in their original precision; every projection and embedding is quantized.

Converted layer-by-layer from the bf16 source — a 56 GB bf16 checkpoint does not fit alongside a quantized copy on a 64 GB machine, so a streaming converter is used rather than mlx_lm convert.

Notes

• Custom architecture. hy_v3 (HYV3ForCausalLM) is not yet in MLX-LM, so hy_v3.py ships with this repo and is loaded via trust-remote-code.
• Two config corrections were needed: upstream moe_intermediate_size: 1536 disagrees with the checkpoint's own expert weights (all 768-wide), and rope_theta moved into a nested rope_parameters block. Both are fixed here.
• This model is a writer, not a retriever or answerer. Pair it with a general-purpose LLM; it does not replace one.

Attribution

• Base model: tencent/Hy-Memory-30B-A3B by Tencent Hunyuan, Apache-2.0.
• hy_v3 MLX adapter: originally authored by QwQbb for the Hy-MT2 conversion; reused here with a stop_gradient fix on the router indices.
• Conversion and benchmarking: Hermes Agent using mlx-lm 0.31.3.
• Paper: arXiv:2610.22196

Community conversion, not an official Tencent release.

Citation

@misc{hy-memory-30b-a3b,
  title  = {Hy-Memory-30B-A3B},
  author = {Tencent Hunyuan},
  year   = {2026},
  eprint = {2610.22196},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url    = {https://arxiv.org/pdf/2610.22196},
}

License

Apache-2.0, same as upstream.