Instructions to use LocalMuseAI/mlx-flux2-klein-4b-aio-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use LocalMuseAI/mlx-flux2-klein-4b-aio-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download LocalMuseAI/mlx-flux2-klein-4b-aio-4bit --local-dir mlx-flux2-klein-4b-aio-4bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Download conversion-guide.md from LocalMuseAI/mlx-flux2-klein-4b-aio-4bit: direct link, hf CLI and curl.
- Browser
- Download file 21 kB
-
https://huggingface.co/LocalMuseAI/mlx-flux2-klein-4b-aio-4bit/resolve/main/conversion-guide.md
- Command line
-
hf download hf://LocalMuseAI/mlx-flux2-klein-4b-aio-4bit/conversion-guide.md
-
curl -L -o conversion-guide.md https://huggingface.co/LocalMuseAI/mlx-flux2-klein-4b-aio-4bit/resolve/main/conversion-guide.md
FLUX.2 Klein 4B AIO: conversion and LocalMuse integration
Published model
| Field | Value |
|---|---|
| App name / ID | FLUX.2 Klein 4B / flux2_klein_4b_aio_mlx |
| Source | Civitai model 2327389, version 2618128, BF16 file 2511488 |
| Source version | Flux.2-4B-Distilled-AIO, repack by SeeSeeLP / SeeSee21 |
| HF repository | LocalMuseAI/mlx-flux2-klein-4b-aio-4bit |
| Immutable runtime commit | 646b51828f39b2541fbfcc643d6984d199c6f6a5 (first six files unchanged since c2abb5051572bcda02841b3e24af3e3fe9911d57) |
| Runtime / backend | flux2Klein4B4Bit / flux2KleinMLX |
| Payload | 4,560,019,137 bytes (4.56 GB / 4.25 GiB), seven authenticated files |
| Defaults | 1024 × 1024, 4 steps, CFG 1, Euler / Simple |
| Allowed steps | 4–6 |
| Workflows | Text to Image; Image to Image as reference editing. One image per request |
| Device floor | 12 GiB RAM; physical iPhone validation pending |
This is an AIO repack of the official step-distilled Klein 4B, not the 9B model or the undistilled base. The author's recommendations are 4–6 steps, CFG 1, Euler, Simple or normal scheduling, and native 1024 square. LocalMuse implements the Simple variant. Existing Bonsai, Z-Image, SD and SDXL catalog entries retain their previous IDs, manifests, resolutions and defaults.
The model, Qwen3 text encoder and official VAE are Apache-2.0. The source card
permits redistribution and derivatives; exact metadata is in source.json.
Preserve LICENSE, NOTICE, the author attribution and the BFL/Qwen notices
when redistributing converted weights. Runtime code adapted from Duration AI
retains its Apache license and complete upstream NOTICE. ComfyUI is only an
external development reference; no ComfyUI implementation is included in the app.
What is converted
- DiT: 3072 hidden dimensions, 24 heads, five double-stream and twenty single-stream blocks. Attention and FFN matrices use ordinary affine INT4, group 64; sensitive input/output, timestep and modulation layers remain BF16.
- Qwen3-4B: INT4/group64 including row-wise embedding lookup. Raw hidden states after layers 9, 18 and 27 are concatenated to 7680 channels. Layers after the last tap, the final language normalization and language head are unused and omitted. No entire embedding-table dequantization occurs at runtime.
- Tokenizer: official BFL-pinned Qwen3 files, empty thinking chat template, right padding to 512 tokens, causal and padding attention masks.
- VAE: original 32-channel AIO decoder, encoder and packed 128-channel
latent BN statistics retained in FP32; see the projection note below.
Convolution weights change layout to MLX OHWI; attention weights become 2D.
The encoder is a separate
vae_encoder.safetensors(its nestedquant_convincluded), so the six original runtime files stay byte-identical. - Reference editing (Image to Image): the source is fitted to 1024 square,
encoded to the mean of the 64 moment channels, 2×2-patchified, BN-normalized
and packed into 4096 tokens appended to the image stream at RoPE time 10 —
Diffusers'
Flux2KleinPipelinelayout (_prepare_image_ids, scale 10). The transformer output for reference tokens is dropped. Denoising still starts from noise, so there is no strength control. - Scheduling: positive Klein flow velocity, Euler updates with the ComfyUI 10,000-value Simple grid, exponential shift mu 2.02. The terminal zero performs an Euler update; it does not invoke the transformer.
Ordinary INT4 is a separate profile from Bonsai's ternary INT2/group128 format. Neither artifact can be loaded through the other adapter. This profile fixes native 1024 square; it does not enable a size picker, negative prompts, Hires Fix, Face Detail or Inpaint. SD/SDXL retain their existing capabilities.
VAE projections in this AIO
The AIO stores both learned 1×1 VAE projections in Black Forest Labs' nested
layout — vae.encoder.quant_conv.* and vae.decoder.post_quant_conv.* — not
as Diffusers' top-level quant_conv / post_quant_conv (Diffusers'
convert_ldm_vae_checkpoint does not map them either). The first conversion
looked up only the top-level decoder key and therefore restores the decoder
projection from the matching official VAE. On 2026-10-02 the nested AIO
projections, plus the checked encoder and decoder weights, were compared with
that file: every FP32 AIO value rounds exactly to the official BF16 value. The
restored projection is the same learned tensor, so the published decoder is
valid; the decoder file also keeps the AIO's unused nested copy. An identity
projection is invalid and produces severe mosaic. The encoder export uses the
AIO's own FP32 encoder.quant_conv.
The decoder projection is restored from:
- Repository:
black-forest-labs/FLUX.2-klein-4B. - Revision:
e7b7dc27f91deacad38e78976d1f2b499d76a294. - File:
vae/diffusion_pytorch_model.safetensors(168,120,878 bytes). - SHA-256:
ca70d2202afe6415bdbcb8793ba8cd99fd159cfe6192381504d6c4d3036e0f04.
The source AIO's decoder and BN statistics remain intact at FP32 precision.
The official BF16 projection is cast to FP32; this does not recover extra
precision. Both source hashes are embedded in conversion metadata. Never
substitute an unrelated VAE or an identity post-quant projection. See
validation/vae-repair.json and validation/bf16-dit-reference-1024.png.
Reproduce conversion on an Apple Silicon Mac
Commands below run from the project root. Use a task-specific scratch directory; keep the user's original checkpoint unchanged. Allow enough disk space for the 16.13 GB source, 4.56 GB output, environment and build products.
1. Python environment and pinned base resources
KLEIN_WORK="$(mktemp -d /tmp/localmuse-klein.XXXXXX)"
python3.11 -m venv "$KLEIN_WORK/env"
"$KLEIN_WORK/env/bin/python" -m pip install -r Tools/FluxKlein/requirements.txt
"$KLEIN_WORK/env/bin/python" - "$KLEIN_WORK/base" <<'PY'
import sys
from huggingface_hub import snapshot_download
snapshot_download(
repo_id='black-forest-labs/FLUX.2-klein-4B',
revision='e7b7dc27f91deacad38e78976d1f2b499d76a294',
allow_patterns=['tokenizer/*', 'text_encoder/config.json',
'transformer/config.json', 'vae/config.json',
'vae/diffusion_pytorch_model.safetensors', 'LICENSE'],
local_dir=sys.argv[1],
)
PY
The downloaded source file is
~/Downloads/flux2KleinAIO_flux24BDistilledAIO_full_bf16.safetensors:
16,132,318,096 bytes, SHA-256
f9f60d53bb21970e19d31d85eefb294f9b0b5e36669ec81ba2bf52f102a7679e.
The converter checks the entire source file and supplemental VAE before writing.
2. Stream and quantize
"$KLEIN_WORK/env/bin/python" Tools/FluxKlein/convert_klein_aio.py \
--checkpoint "$HOME/Downloads/flux2KleinAIO_flux24BDistilledAIO_full_bf16.safetensors" \
--expected-sha256 f9f60d53bb21970e19d31d85eefb294f9b0b5e36669ec81ba2bf52f102a7679e \
--tokenizer "$KLEIN_WORK/base/tokenizer" \
--vae-base "$KLEIN_WORK/base/vae/diffusion_pytorch_model.safetensors" \
--expected-vae-sha256 ca70d2202afe6415bdbcb8793ba8cd99fd159cfe6192381504d6c4d3036e0f04 \
--output "$KLEIN_WORK/converted"
The converter reuses the streaming safetensors writer from Tools/ZImage without
changing the Z-Image converter. It maps the source FLUX names, splits QKV and
swaps the final adaLN scale/shift halves to match Diffusers. It processes one
matrix at a time; loading the entire BF16 checkpoint is unnecessary. Export
is deterministic with the pinned MLX/Python versions. Compare the resulting
seven-file manifest with Tools/FluxKlein/manifest.json before publishing. Direct-file catalog entries must leave downloadURL and checksum null; each file carries its own URL and hash in downloadFiles. Mixing archive metadata into a direct manifest is rejected by the catalog validator.
Required paths:
runtime_config.json
text_encoder-mlx-4bit/model.safetensors
tokenizer/tokenizer.json
tokenizer/tokenizer_config.json
transformer-mlx-4bit/model.safetensors
vae.safetensors
vae_encoder.safetensors
3. Build validation CLI and run native inference
Use Xcode's macOS executable scheme so MLX's Metal library is bundled. Keep this
package's source-package cache separate from the application's cache: sharing
-clonedSourcePackagesDirPath between different package graphs can prune the
app's llama.xcframework binary artifact. If that occurred, resolve the app
package graph and clean the app's build products before rebuilding.
(
cd Packages/bonsai-runtime
xcodebuild build -scheme klein-validate -destination 'platform=macOS' \
-derivedDataPath "$KLEIN_WORK/runtime-build" \
-clonedSourcePackagesDirPath "$KLEIN_WORK/SourcePackages"
)
MTL_DEBUG_LAYER=1 "$KLEIN_WORK/runtime-build/Build/Products/Debug/klein-validate" \
"$KLEIN_WORK/converted" "$KLEIN_WORK/native-1024.png" 1024x1024 1729 4 \
'Candid photograph of an adult woman in a cream sweater by a studio window, soft evening light, natural skin texture, analog grain.'
Reference editing (Image to Image) uses the same command with --reference;
the image is aspect-filled and center-cropped to the output size like the app:
MTL_DEBUG_LAYER=1 "$KLEIN_WORK/runtime-build/Build/Products/Debug/klein-validate" \
--reference Tools/FluxKlein/validation/native-1024.png \
"$KLEIN_WORK/converted" "$KLEIN_WORK/native-1024-edit.png" 1024x1024 1729 4 \
'Change the cream sweater to a bright red wool sweater, keep the face, pose, window and lighting unchanged.'
The CLI also saves parameters and the final CPU preview. It uses the same library as the iOS app, authenticates the runtime configuration, and validates all tensor shapes and dtypes before evaluating a graph. Encoder, DiT and VAE load sequentially. The decoder uses overlapping 384-pixel windows with CPU cosine blending to bound memory. Tile-local VAE statistics and attention can cause small differences from one-shot full-frame decoding.
4. Independent Transformers/Diffusers comparison
MTL_DEBUG_LAYER=1 "$KLEIN_WORK/runtime-build/Build/Products/Debug/klein-validate" \
--probe --reference Tools/FluxKlein/validation/native-1024.png \
"$KLEIN_WORK/converted" "$KLEIN_WORK/probe" 256x256 1729 4 \
'Candid photograph of an adult woman in a cream sweater by a studio window, soft evening light, natural skin texture, analog grain.'
"$KLEIN_WORK/env/bin/python" Tools/FluxKlein/validate_reference.py \
--checkpoint "$HOME/Downloads/flux2KleinAIO_flux24BDistilledAIO_full_bf16.safetensors" \
--base "$KLEIN_WORK/base" --converted "$KLEIN_WORK/converted" \
--probe "$KLEIN_WORK/probe" --report "$KLEIN_WORK/reference.json" \
--prompt 'Candid photograph of an adult woman in a cream sweater by a studio window, soft evening light, natural skin texture, analog grain.'
The reference streams source modules on CPU, using independent Transformers and Diffusers implementations. It compares source BF16 with native INT4, then repeats the DiT with an independent NumPy unpacking of the quantized matrices to separate quantization loss from Swift-port error. The VAE comparison uses the original AIO FP32 decoder and the same restored official projection. It also verifies exact Swift/Python token IDs and unchanged BN tensors.
Saved results in validation/reference.json:
| Comparison | Relative L2 | Cosine |
|---|---|---|
| Source BF16 Qwen3 vs Swift INT4 | 0.103519 | 0.995114 |
| Source BF16 DiT vs Swift INT4, same conditioning | 0.114535 | 0.993426 |
| Dequantized INT4 DiT in Diffusers vs Swift INT4 | 0.009666 | 0.999953 |
| Restored source FP32 VAE vs Swift FP32 | 0.00000255 | ~1.0 |
The probe also encodes a 256×256 reference and predicts the same noise with its
tokens appended. validate_edit_reference.py compares it with independent
Diffusers code: AutoencoderKLFlux2 loaded with the AIO's FP32 VAE through
Diffusers' own mapping (the nested projections mapped explicitly), the pipeline's
static patchify, BN, packing and _prepare_image_ids helpers, and
Flux2Transformer2DModel with source BF16 and independently dequantized INT4
matrices:
"$KLEIN_WORK/env/bin/python" Tools/FluxKlein/validate_edit_reference.py \
--checkpoint "$HOME/Downloads/flux2KleinAIO_flux24BDistilledAIO_full_bf16.safetensors" \
--base "$KLEIN_WORK/base" --converted "$KLEIN_WORK/converted" \
--probe "$KLEIN_WORK/probe" --report "$KLEIN_WORK/edit-reference.json"
Saved results in validation/edit-reference.json:
| Comparison | Relative L2 | Cosine |
|---|---|---|
| AIO FP32 encoder (Diffusers) vs Swift FP32 | 0.00000258 | ~1.0 |
| Reference tokens (Diffusers helpers) vs Swift | 0.00000258 | ~1.0 |
| Source BF16 DiT vs Swift INT4, with reference | 0.094921 | 0.995487 |
| Dequantized INT4 DiT in Diffusers vs Swift INT4, with reference | 0.010691 | 0.999943 |
| Official BF16 VAE encoder vs Swift FP32 (weight rounding only) | 0.004676 | 0.999989 |
With the reference the predicted velocity differs from the text-only prediction by relative L2 0.822, so the reference conditions the transformer.
These are component comparisons, not proof of identical full-pipeline BF16
images. bf16-dit-reference-1024.png is an additional full-size source DiT
reference with the same INT4 text conditioning and initial noise as the native
run. A ComfyUI seed uses a different RNG implementation and need not produce the
same image as MLX. INT4 and tiled VAE decoding also prevent pixel-exact equivalence.
5. Schedule and regression tests
Packages/bonsai-runtime/Tests/BonsaiRuntimeTests/Fixtures/comfy-klein-simple.json
contains numerical output from unmodified ComfyUI functions at revision
65787d668397d230bf5839d69a0a7239e2dad378. To regenerate, download the external
comfy/samplers.py and comfy/model_sampling.py from that exact revision into a
scratch reference directory, then run:
"$KLEIN_WORK/env/bin/python" Tools/FluxKlein/export_schedule_fixture.py \
--references "$KLEIN_WORK/comfy-reference" \
--revision 65787d668397d230bf5839d69a0a7239e2dad378 \
--output "$KLEIN_WORK/comfy-klein-simple.json"
(
cd Packages/bonsai-runtime
MTL_DEBUG_LAYER=1 xcodebuild test -scheme bonsai-runtime-Package \
-destination 'platform=macOS' -parallel-testing-enabled NO \
-derivedDataPath "$KLEIN_WORK/runtime-build" \
-clonedSourcePackagesDirPath "$KLEIN_WORK/SourcePackages" \
-resultBundlePath "$KLEIN_WORK/package-tests.xcresult"
)
xcodebuild build-for-testing -project LocalMuse.xcodeproj -scheme LocalMuse \
-destination 'generic/platform=iOS' -parallel-testing-enabled NO
xcodebuild test -project LocalMuse.xcodeproj -scheme LocalMuse \
-destination 'platform=iOS Simulator,id=E623201E-7B34-4622-9EF5-CD4E43558F26' \
-parallel-testing-enabled NO -only-testing:LocalMuseTests
Reuse an existing simulator on the newest installed compatible runtime; substitute
its UUID if the listed device is unavailable. The final application run executed 490 Swift Testing tests: 488 passed and the two physical-device opt-in tests were skipped. The full scheme's XCUITests also ran with Klein in the catalog; their only failure, testModalGalleryFitsContentInBothAppearances, is a pre-existing modal-height check that uses no Klein code. The package run passed all 18 tests with Metal API validation enabled. A clean rerun of the native Text to Image render after the encoder work produced a byte-identical native-1024.png with the same 3,606,440,960-byte MLX peak. Saved test and build summaries are in validation/; physical-device tests require an explicit opt-in and are skipped in normal regression runs.
Publication and catalog installation
The source was authenticated, converted and validated before publication. The
six original runtime files were published at c2abb5051572bcda02841b3e24af3e3fe9911d57;
vae_encoder.safetensors and the seven-file manifest were added at
646b51828f39b2541fbfcc643d6984d199c6f6a5 without changing those six files. Every remote byte size and
LFS SHA-256 was compared with the local manifest. Small files were downloaded
and hashed; each pinned public URL was also checked without authorization.
Full evidence: verification.json.
When replacing an artifact or publishing a new checkpoint:
- Validate its source license, actual architecture and complete checkpoint hash.
- Convert in a new directory and finish native/reference checks.
- Authenticate to HF using
hf auth login; keep credentials out of scripts/logs. - Create a private repository, upload the runtime files plus model card, license,
notice, source metadata and manifest with
huggingface_hub.HfApi.upload_folder. - Verify all remote sizes/hashes against the local manifest, publish it, and verify anonymous access to the resulting immutable commit.
- Put that commit and every exact size and SHA-256 hash in the catalog.
Never use
mainin an application download manifest. - Add the preview asset and the allowlist entry in
ModelIdentityIcon. Match metadata inPreviewCatalogand add profile/catalog/lifecycle tests.
Current catalog version is 60; the 24th image model uses this manifest and
profile. The default image model and language catalog remain unchanged. Model
selection is lazy and all generation shares MLExecutionGate. Cancellation
keeps synchronous MLX work registered until it unwinds. The runtime checks
cancellation and the foreground gate before each encoder layer, transformer
block and VAE operation. The 64 MiB MLX cache limit is restored at exit.
Preparation reports real stages: text encoder and image transformer for Text to Image, plus the image encoder between them for Image to Image. Safetensors load lazily, so the transformer weights are evaluated inside their stage. The VAE decoder is loaded only at decoding after transformer eviction. Optional CPU clean-latent previews use the existing request-scoped relay and filtering limits (128 pixels, or 12 protected pixels), without a new neural network or changes to final latents.
Validation and manual iPhone check
Native Swift 1024 × 1024 / four-step inference on an M1 Pro, 16 GB Mac:
- Elapsed: 99.47 seconds, with Metal API validation enabled.
- Peak MLX allocation: 3,606,375,424 bytes (3439.3 MiB / 3.36 GiB).
- Preparation counts:
[0, 1, 2]; four live preview frames, 128 × 128. - Native image:
validation/native-1024.png; parameters and preview saved beside it.
Reference editing at 1024 × 1024 / four steps on the same Mac
(validation/native-1024-edit.png, source native-1024.png):
- Elapsed: 173.81 seconds, with Metal API validation enabled.
- Peak MLX allocation: 4,349,946,880 bytes (4148.4 MiB / 4.05 GiB); the 4096 reference tokens lengthen every transformer step.
- Preparation counts:
[0, 1, 2, 3]; four live preview frames, 128 × 128.
These are Mac measurements, not iPhone speed or total-process-memory claims. The physical-device check is left to the user. The current 12 GiB catalog floor is conservative; do not enable 8 GB devices based on host allocations alone.
For normal manual testing, build/run LocalMuse, download FLUX.2 Klein 4B, keep the default 1024 square / four steps / CFG 1, and use the neutral English prompt above. Check both model-preparation stages, four denoising previews, correct final dimensions and image quality. Then open Image to Image with that result as the source and the edit prompt above: there is no strength control, the prompt is required, preparation shows three stages (text encoder, image encoder, image transformer) and only the requested change should differ. Also check cancellation, returning from the background, and switching back to an existing SD/SDXL or Z-Image model.
A developer-only KleinDeviceTests.installedCheckpointGeneratesAndEditsNativeCanvasWithMetalValidation()
smoke test is available with LOCALMUSE_KLEIN_DEVICE_QA=1 and MTL_DEBUG_LAYER=1.
It renders the prompt above, then reference-edits that render (klein-device-qa
and klein-device-qa-edit PNG/preview/JSON in Caches, including peak MLX bytes).
It uses the installed authenticated model or an explicitly staged cache
directory and does not write gallery or persistence records. It is not enabled
or run automatically.
Cleanup
Preserve source checkpoints in Downloads and these repository files. After
publication and evidence checks, remove only the task's scratch environment,
converted duplicate, external references, validation arrays and build products.
Do not delete installed app models, user gallery data, shared HF caches or the
app's existing SourcePackages cache. This run's temporary package .build
directory and generated default.profraw can also be removed after validation.