modilify/Modilify-Mk2-preview-mlx

Modilify Mk2 Preview · MLX

Modilify Mk2 is a rolling-canvas text diffusion model with learned trajectory memory. This release provides complete, unquantized native MLX weights and an inference package for Apple Silicon Macs. It builds on Google DiffusionGemma, adding LoRA adapters, GDN2 memory at two timescales, and a confidence-and-entropy prefix commit policy.

The model revises a 256-token canvas over successive denoising passes, commits a variable-length prefix, and carries memory forward as the canvas advances. Per-position and row-level working states update on every denoising pass; persistent memory updates only when tokens commit.

This is an experimental text-only preview. The complete model is included; no separate base-model download is required. PyTorch is not required.

Download and run

Use Python 3.12 on an Apple Silicon Mac with macOS 26 or newer for the pinned MLX environment. The weights occupy 48.23 GiB. Inference needs additional memory for KV caches, recurrent states, and computation. See local comparison for measured short-request memory usage.

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install "huggingface-hub==1.20.1"

hf download Modilify/Modilify-Mk2-preview-mlx --local-dir ./Modilify-Mk2-preview-mlx
cd Modilify-Mk2-preview-mlx
python -m pip install .

python inference.py --prompt "Why is the sky blue?" \
  --max-new-tokens 256 --max-denoising-steps 256 --stream

The source entrypoint defaults to the model in the same directory. The installed command accepts a local model directory or a Hub model ID:

modilify-mlx --model ./ --prompt "你好，请介绍一下自己。" --stream
modilify-mlx --model Modilify/Modilify-Mk2-preview-mlx \
  --prompt "What is 2 + 3?" --max-new-tokens 128 --max-denoising-steps 128

The Hub ID path downloads the model snapshot into the Hugging Face cache. Later runs reuse the cached files. All inference computation runs locally.

Thinking is enabled by default; use --think False to disable it. --max-new-tokens limits output length; --max-denoising-steps limits denoising work. Reaching either budget may truncate a response. The CLI defaults to 8192 output tokens and the generation configuration's 48 denoising steps. Set both budgets explicitly for your workload. --canvas-length can reduce the canvas from its maximum of 256. Without --stream, the CLI emits JSONL events and a final result with text, token IDs, stop reason, and timing/memory metrics.

Python API

Load once and reuse the runtime for independent requests:

from inference import load_model, generate

runtime = load_model("./")
result = generate(
    runtime,
    "Explain diffusion models in one paragraph.",
    max_new_tokens=256,
    max_denoising_steps=256,
    seed=42,
    think=True,
)
print(result["text"])
print(result["stop_reason"])

Pass messages=[{"role": "user", "content": "..."}] instead of prompt for a conversation. The tokenizer's Modilify template handles system, user, and assistant turns. With thinking enabled, generated text may include reasoning before the answer.

Continuous requests

printf '%s\n' \
  '{"request_id":"a","prompt":"你好","max_new_tokens":128,"max_denoising_steps":128}' \
  '{"request_id":"b","prompt":"What is 2 + 3?","seed":42,"think":false,"max_new_tokens":128,"max_denoising_steps":128}' | \
  modilify-mlx --model ./ --requests-jsonl - \
    --batch-size 4 --pipeline-depth 2 --max-batch-tokens 1024

Each line requires a unique request_id and either prompt or messages. Optional fields are max_new_tokens, max_denoising_steps, seed, and think. The default scheduler interleaves independent rows and keeps their random streams and states separate. --batch-size controls resident requests; --pipeline-depth bounds in-flight denoises.

Release details

Item	Value
Base model	DiffusionGemma 26B-A4B-it text backbone
Saved training step	1250
Memory topology	schema25 / compact_gdn2_v2
Memory scheme	dual_timescale_gdn2_trajectory_memory
Canvas	Up to 256 tokens
Expert routing	8 of 128 experts per token
Dense / expert LoRA rank	16 / 8
Weight precision	BF16, with small trajectory norms and dynamics parameters in FP32
Proposal constraints	top_k 40, min_p 0.05
Commit defaults	failure budget 0.2, target confidence 0.5
Weights	32 Safetensors shards, 1323 tensors

The complete backbone, adapters, and memory parameters are stored in the native MLX parameter layout. Adapters remain unfused to preserve the reference BF16 computation. Loading validates model topology, tensor shape/dtype, and shard SHA256 hashes. export_manifest.json contains the weight index metadata and release provenance; no optimizer state or training data is shipped.

This model uses its own rolling-canvas generation loop. Use the included inference.py, modilify-mlx, or Python API. Generic mlx_lm.generate, Transformers AutoModel.from_pretrained, and standard autoregressive serving backends do not implement this protocol. Image, audio, and video inputs are not supported by this release.

Training, evaluation, and limitations

The text backbone is adapted with dense and expert LoRA and trained trajectory memory. Adaptation uses response supervision over the valid rolling canvas, with terminal, confidence, and commit-readiness objectives. This release uses the saved step 1250 weights. The adaptation dataset is not distributed and its source composition is not documented in this release; refer to the upstream model card for base-model training information.

Local comparisons establish export correctness for the tested cases, not task quality or safety. No capability benchmark score is claimed for this preview, and upstream benchmark scores do not describe this modified model. Long-context quality, broad language coverage, and deployment safety have not been established by those checks.

Local comparison

On 2026-10-06, the package installed in a fresh Python 3.12 environment without PyTorch and ran offline. Two requests matched the strict step 1250 source checkpoint at 1/2/5/10/20 denoising steps, including token IDs, stopping and canvas shifts. At five steps, interleaved B2 also matched independent B1 outputs and all three GDN2 state hashes. Short-request MLX peak allocations were about 50.79 GiB; longer workloads require additional memory.

Intended use and impact statement

This preview is intended for research on text diffusion, trajectory memory, local generation, and reproducible inference experiments. Generated statements can be incorrect, biased, or inappropriate; verify consequential outputs and keep qualified human review for sensitive decisions. The model is not intended to make autonomous high-risk decisions. Do not assume local execution alone provides safety, privacy, or resistance to prompt injection. Deployment should include use-specific testing, appropriate safeguards, and incident monitoring.

License and attribution

Modilify's original contributions and distribution are covered by the Modilify Open Model License 1.0, a custom license. Upstream components retain their own terms; the DiffusionGemma base is published under Apache 2.0. See NOTICE.md for attribution and upstream notices.

Maintained by Modilify. Report model and runtime issues through the Hub discussions.