keras/muse_glimmer_30b

Model Overview

Model Overview

Muse Glimmer is a 30-billion-parameter multimodal causal language model published by Meta Superintelligence Lab, distilled from Muse Spark and purpose-built for autonomous agentic tasks on consumer hardware. The language model is a 52-layer dense decoder with a repeating [Local, Local, Local, Global] attention pattern (sliding window 2048), gated grouped-query attention (32 query / 2 KV heads), SwiGLU feed-forward blocks, and RoPE on local layers only. A ~1.8B-parameter ViT-G/14 perception encoder lets the model accept interleaved text and images, and the context length is 131,072+ tokens. Muse Glimmer ships with a lightweight DFlash block-diffusion drafter — a 5-layer companion network that proposes 16-token blocks for speculative decoding and is used exclusively from the target model's generation loop, never standalone. The model is intended for multi-step reasoning, tool use, agentic coding, and multimodal understanding in local deployments; it is not intended to be deployed as an endpoint by itself without additional system-level guardrails. See the model card below for benchmarks, data sources, and intended use cases.

Weights are released under the Apache 2 License. Keras model code is released under the Apache 2 License.

Links

• [Muse Glimmer Quickstart Notebook](coming soon)
• [Muse Glimmer API Documentation](coming soon)
• Muse Glimmer Model Card
• Muse Glimmer Assistant (DFlash drafter) Model Card
• Muse Glimmer Model Paper
• KerasHub Beginner Guide
• KerasHub Model Publishing Guide

Installation

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the Keras Getting Started page.

Presets

The following model checkpoints are provided by the Keras team. Full code examples for each are available below.

Checkpoints are being uploaded; the table lists the presets defined in KerasHub PR #3100.

Preset name	Parameters	Description
muse_glimmer_30b	29.78B	30B-parameter dense causal decoder with an attached windowed-attention perception encoder, distilled from Muse Spark for local agentic use.
muse_glimmer_30b_assistant	2.56B	MuseGlimmer 30B DFlash Assistant model: 5-layer speculative-decoding drafter for the 30B model. Drafts a 16-token block-diffusion denoised block per forward pass, conditioned on the target model's hidden states. This model must NOT be used standalone. It is designed exclusively as a draft model used from the target model's generation loop.

Example Usage

import keras
import keras_hub
import numpy as np

This variation is the DFlash speculative-decoding drafter for muse_glimmer_30b. It has no vocabulary and no LM head, so it cannot generate text on its own — calling it directly raises NotImplementedError. Load it and pass it as assistant_model to the 30B model's generate(). The drafter proposes 16-token blocks, the target verifies them in parallel, and with a greedy sampler the output is identical to plain greedy decoding.

target = keras_hub.models.MuseGlimmerCausalLM.from_preset(
    "muse_glimmer_30b",
    dtype="bfloat16",
)
assistant = keras_hub.models.MuseGlimmerAssistantCausalLM.from_preset(
    "muse_glimmer_30b",
    dtype="bfloat16",
)
target.compile(sampler="greedy")
target.generate("What is Keras?", max_length=64, assistant_model=assistant)

# Generate with batched prompts.
target.generate(
    ["What is Keras?", "Give me your best brownie recipe."],
    max_length=64,
    assistant_model=assistant,
)

Speculative decoding also works with image prompts. Put one <|patch|> placeholder in the prompt for each image.

image = np.asarray(keras.utils.load_img("cat.jpg"))
target.generate(
    {
        "prompts": "<|patch|>Describe this image.",
        "images": image,
    },
    max_length=512,
    assistant_model=assistant,
)

Load the drafter's backbone directly, for example to inspect its configuration.

backbone = keras_hub.models.MuseGlimmerBackbone.from_preset(
    "muse_glimmer_30b",
    dtype="bfloat16",
)
print(backbone.num_layers, backbone.context_projection_layer_ids)

Example Usage with Hugging Face URI

import keras
import keras_hub
import numpy as np

This variation is the DFlash speculative-decoding drafter for hf://keras/muse_glimmer_30b. It has no vocabulary and no LM head, so it cannot generate text on its own — calling it directly raises NotImplementedError. Load it and pass it as assistant_model to the 30B model's generate(). The drafter proposes 16-token blocks, the target verifies them in parallel, and with a greedy sampler the output is identical to plain greedy decoding.

target = keras_hub.models.MuseGlimmerCausalLM.from_preset(
    "hf://keras/muse_glimmer_30b",
    dtype="bfloat16",
)
assistant = keras_hub.models.MuseGlimmerAssistantCausalLM.from_preset(
    "hf://keras/muse_glimmer_30b",
    dtype="bfloat16",
)
target.compile(sampler="greedy")
target.generate("What is Keras?", max_length=64, assistant_model=assistant)

# Generate with batched prompts.
target.generate(
    ["What is Keras?", "Give me your best brownie recipe."],
    max_length=64,
    assistant_model=assistant,
)

Speculative decoding also works with image prompts. Put one <|patch|> placeholder in the prompt for each image.

image = np.asarray(keras.utils.load_img("cat.jpg"))
target.generate(
    {
        "prompts": "<|patch|>Describe this image.",
        "images": image,
    },
    max_length=512,
    assistant_model=assistant,
)

Load the drafter's backbone directly, for example to inspect its configuration.

backbone = keras_hub.models.MuseGlimmerBackbone.from_preset(
    "hf://keras/muse_glimmer_30b",
    dtype="bfloat16",
)
print(backbone.num_layers, backbone.context_projection_layer_ids)