Interchained/Muse-Glimmer-30B-GGUF

Muse-Glimmer-30B — Built by Interchained

Six GGUF builds. One verified source. One independently built local pipeline.

Official Meta Muse-Glimmer-30B weights, downloaded from the original publisher, converted with Interchained's Hearth tooling, and quantized on our own workstation using a locally compiled llama.cpp. No third-party quants. No mystery provenance. Every artifact in this collection was produced from the same BF16 GGUF master, and the process is documented below.

This is the work we set out to do: take a large model from its original published weights to a full, usable precision ladder—and prove the results on hardware we assembled and configured ourselves.

Built and released by Interchained. Original model architecture and weights by Meta.

The complete lineup

All six files below were built locally and uploaded to this repository.

Artifact	Approximate size	Local validation	Why choose it
Muse-Glimmer-30B-BF16.gguf	55.7 GB	Loaded and generated tokens using hybrid CPU/GPU execution	The high-precision GGUF master used to produce every quant below
Muse-Glimmer-30B-Q8_0.gguf	29.6 GB	Passed end-to-end inference	Higher-precision quant for users with enough memory
Muse-Glimmer-30B-Q6_K.gguf	22.9 GB	Passed end-to-end inference	Strong precision/size option
Muse-Glimmer-30B-Q5_K_M.gguf	19.8 GB	Passed end-to-end inference	Balanced storage and precision
Muse-Glimmer-30B-Q4_K_M.gguf	16.9 GB	Passed end-to-end inference	Our practical starting point for a 32 GB V100
Muse-Glimmer-30B-Q4_0.gguf	16.0 GB	Passed end-to-end inference	Smallest build in this release; fastest in our initial smoke tests

Sizes are approximate decimal GB, based on conversion outputs and observed upload sizes. Actual repository file sizes are authoritative. The master stores BF16 weights—not F16—and is a GGUF conversion of the upstream safetensors, not a byte-for-byte safetensors copy.

Built here, from the original source

We didn't download somebody else's GGUF and put our name on it.

1. Pulled Meta's original safetensors with hearth pull hf:meta-models/Muse-Glimmer-30B (~55.46 GiB fetched). Hearth verified downloaded weight shards; locally computed hashes for metadata without upstream reference checksums should not be confused with independent source authentication.
2. Converted the original weights to a BF16 GGUF master with Interchained's Hearth conversion workflow, using Python 3.12 and a Transformers version supporting the source tokenizer.
3. Quantized directly from that BF16 master with our locally compiled llama-quantize—not from one quant to another.
4. Ran all five quants locally on a Tesla V100 32 GB, using Meta's native ATEM chat template, and captured their completion/timing responses.
5. Loaded and executed the BF16 master locally using hybrid CPU/GPU offload. It generated tokens; the capped test did not reach a final visible response. We report that distinction because the measurements should speak for themselves.
6. Published the full six-file GGUF family for others to download, evaluate, and use.

Build environment

Detail	Recorded setup
Workstation GPU	NVIDIA Tesla V100 / PG500-216, 32 GiB, CUDA capability 7.0
llama.cpp	0.6.0-dev, build 11484, commit d625d3700
Conversion	Interchained Hearth; Python 3.12
Master format	GGUF V3, BF16 intermediate
Model layout	731 tensors, 52 transformer blocks, 202,048 vocabulary entries
Model context metadata	131,072 tokens
Local test date	October 14, 2026
Validated test context	4,096 tokens for quantized inference (not a full-context validation)

Reproduce the precision ladder

Starting from the provided Muse-Glimmer-30B-BF16.gguf, use a compatible llama.cpp llama-quantize executable:

llama-quantize Muse-Glimmer-30B-BF16.gguf Muse-Glimmer-30B-Q8_0.gguf Q8_0
llama-quantize Muse-Glimmer-30B-BF16.gguf Muse-Glimmer-30B-Q6_K.gguf Q6_K
llama-quantize Muse-Glimmer-30B-BF16.gguf Muse-Glimmer-30B-Q5_K_M.gguf Q5_K_M
llama-quantize Muse-Glimmer-30B-BF16.gguf Muse-Glimmer-30B-Q4_K_M.gguf Q4_K_M
llama-quantize Muse-Glimmer-30B-BF16.gguf Muse-Glimmer-30B-Q4_0.gguf Q4_0

The original locally generated BF16 intermediate was named meta-models_Muse-Glimmer-30B-hearth-base-auto.gguf, later copied to the published Muse-Glimmer-30B-BF16.gguf filename.

Measured on our Tesla V100

Five quantized builds loaded, generated coherent final answers, and ended with finish_reason: "stop" using the native ATEM chat template.

GGUF	Quantized tensor payload	Average bits/weight	Observed generation	Completion tokens*	Stop
Q4_0	14.92 GiB	4.60	42.56 tok/s	306	stop
Q4_K_M	15.76 GiB	4.86	38.01 tok/s	298	stop
Q5_K_M	18.44 GiB	5.69	34.02 tok/s	336	stop
Q8_0	27.57 GiB	8.50	27.79 tok/s	365	stop
Q6_K	21.28 GiB	6.56	27.09 tok/s	294	stop

Includes reasoning tokens. Speeds are single-run observations from short introductory prompts, not controlled throughput averages or claims of relative model quality. Model behavior and performance vary by runtime, offload, context, prompt, and hardware.

Our Q4_K_M run reached 38.01 tok/s with a 4,096-token context and GPU offload enabled. A point-in-time nvidia-smi sample showed 16,132 MiB assigned to llama-server.exe. This is an allocation snapshot, not proof of exact offload placement.

BF16 master: what we confirmed

We successfully loaded the BF16 GGUF with hybrid CPU/GPU offload on the 32 GiB V100 and generated tokens through the API. In one constrained request, the server reported:

BF16 observation	Result
Prompt throughput	0.60 tok/s
Generation throughput	0.066 tok/s
Generated tokens	10, in reasoning_content
Final answer	Empty (token cap reached)
Finish reason	length

This confirms the master is loadable and executes inference. It is not a completed end-to-end response test: the deliberately short 10-token allowance ended before the reasoning model produced its visible final answer. Partial GPU offload and CPU execution made the test very slow; these figures are specific to that configuration. The five quantized variants completed their end-to-end smoke tests.

Critical: use the native ATEM chat template

Our initial conversion fell back to a generic ### Instruction: / ### Response: format, resulting in repeated dialogue. Using Meta's native chat_template.jinja with --jinja --chat-template-file corrected the format and yielded clean, naturally terminated final answers on all five quantized builds.

Obtain the upstream template if it isn't bundled with your download:

hf download meta-models/Muse-Glimmer-30B chat_template.jinja --local-dir ./muse-assets

llama-server \
  -m ./Muse-Glimmer-30B-Q4_K_M.gguf \
  --jinja \
  --chat-template-file ./muse-assets/chat_template.jinja \
  -ngl 99 -c 4096 -b 512 \
  --host 127.0.0.1 --port 8080

Test the OpenAI-compatible endpoint:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Muse-Glimmer-30B","messages":[{"role":"user","content":"Hello Muse! Introduce yourself in one sentence."}],"temperature":0.2,"max_tokens":512}'

Check for finish_reason: "stop" and a visible final content field. Muse may emit a separate reasoning_content field that client applications should handle. A sibling Jinja file is not automatically selected by every inference runner; configure your runtime explicitly. Use a recent llama.cpp build with support for this model architecture.

The honest section

• These are our conversions and quants. Meta developed the original model. Interchained built the local acquisition, conversion, quantization, testing, and publication pipeline represented by this release.
• Numbers over hype. Precision differences are real; downstream quality differences require evaluation, not assumptions. The speed figures above are individual measurements.
• Text inference is what we tested. This package does not include the upstream multimodal projector/perception stack (mmproj) or speculative drafter. Don't assume image-input support from these GGUF files alone.
• Long context wasn't stress-tested. The metadata reports 131K context; our tested runtime configuration used 4,096 tokens for the quants.
• Mind your server exposure. The example binds to loopback; don't expose unauthenticated inference endpoints publicly.

Why this exists

Because we wanted GGUF builds we made ourselves, from the original published source, with no third-party weights in the middle. We wanted a transparent chain from safetensors to BF16 master to five working precision options, tested on our own hardware.

Now they exist. Built by Interchained. Enjoy.

Credits and license

GGUF conversion, quantization, tooling, local validation, and release: Interchained.

Original model and base weights: Meta Superintelligence Labs; source: meta-models/Muse-Glimmer-30B. Original weights are attributed to Meta under Apache License 2.0. Include the upstream license and applicable attribution/usage notices when redistributing. This is an independently engineered Interchained distribution, not a Meta-operated release.

Toolchain: Hearth · llama.cpp

---

Base model by Meta. Complete GGUF precision family built, validated, and published by ...Interchained