--- library_name: diffusers base_model: black-forest-labs/FLUX.1-dev license: other license_name: flux-1-dev-non-commercial-license license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md tags: - lora - flux - compression - text-to-image - diffusers --- # LITHE adapters for FLUX.1-dev This repository holds the 110 rank-16 FLUX.1-dev adapters evaluated in the NeurIPS 2026 paper "LITHE: Lattice-Indexed Twin Hadamard Encoding for Diffusion Personalization" (to appear), stored in the LITHE index format. Code: https://github.com/JianJiang1995/LITHE ## What the files are Each `.lithe` file is one adapter. It contains no float weights: every 64-value block of every LoRA factor is stored as three 9-bit codeword IDs into a fixed Hadamard codebook that is rebuilt from a closed-form rule at load time (nothing is learned or shipped besides the IDs). The ID stream (uint16) is compressed with LZMA (preset 9, extreme) and preceded by a small compressed JSON header with the module names, factor shapes, codebook parameters, checksums and the training metadata listed in `manifest.csv`. * Configuration: Kohya sd-scripts FLUX layout, 304 fused linears per adapter (attention, MLP and modulation linears of all 19 double-stream and 38 single-stream blocks; header field `module_set = kohya-304`), rank 16, alpha 16, K = 512 codewords, 3 residual levels, block size 64. This is the configuration used for the quality results of the paper. * Size: 3,589,632 indices per adapter; 3.23 MB per file on average (355.7 MB for all 110). The decoded bf16 LoRA is about 153 MB. * The adapters were trained natively in the discrete space (straight-through estimator), not quantised after training. ## Usage ```bash git clone https://github.com/JianJiang1995/LITHE.git && cd LITHE pip install -e ".[flux]" huggingface-cli download Fanjue/LITHE --local-dir ./lithe_adapters python scripts/decode_to_lora.py --lithe ./lithe_adapters/indices/dreambooth30/db_cat.lithe --out db_cat_lora.safetensors ``` The decoded file uses Kohya LoRA keys, which the stock diffusers loader converts on load: ```python import torch from diffusers import FluxPipeline pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda") pipe.load_lora_weights("db_cat_lora.safetensors") image = pipe("a photo of a cat on a beach", num_inference_steps=28, guidance_scale=3.5).images[0] ``` Or decode in memory without writing a file: ```python import lithe sd = lithe.to_lora_state_dict(lithe.load_lithe("lithe_adapters/indices/dreambooth30/db_cat.lithe")) pipe.load_lora_weights(sd) ``` `scripts/infer.py --mode peft` and `scripts/serve_demo.py` in the code repository also accept these files. Each file's trigger token and an example prompt are in its header metadata and in `manifest.csv`. The paper evaluated at 1024 x 1024, 28 steps, guidance 3.5. ## Contents | Pool | Folder | Adapters | Trigger tokens | Training | Mean size | Notes | |---|---|---|---|---|---|---| | DreamBooth-30 | `indices/dreambooth30/` | 30 | `` ... `` | 500 steps, 512x512 | 3.31 MB | Subjects of the public DreamBooth dataset (objects and pets), 4 to 6 images each. | | CC101-20 | `indices/cc101_20/` | 20 | `` ... `` | 500 steps, 512x512 | 3.32 MB | 20 subjects sampled from the public CustomConcept101 dataset, one per category, 3 to 10 images each. | | SCIdentity | `indices/scidentity/` | 20 | `` ... `` | 500 steps, 512x512 | 3.27 MB | 20 synthetic animal identities (12 generated images each; images not released). | | SC-40 | `indices/sc40/` | 40 | `` ... ``; `` ... ``; `` ... ``; `` ... ``; `` ... `` | 750 steps, 512x512 | 3.11 MB | 40 synthetic concept-transfer targets: 8 outfits, 8 lighting setups, 8 poses, 8 painter styles, 8 anime styles (20 generated images each; images not released). | Prompt pattern: identity adapters were trained with a single caption such as `a photo of a cat`. SC-40 adapters were trained on per-image captions that contain the trigger and the concept label; `manifest.csv` gives the evaluation prompt used for each (SC-40 groups: anime style 8, lighting 8, outfit 8, painter style 8, pose 8). `manifest.csv` columns: pool, file, concept_id, trigger, description, example_prompt, module_set, n_modules, rank, n_indices, bytes, sha256, train_steps, train_resolution. `SHA256SUMS` lists checksums of all files. ## Data and license * The adapters are derived from FLUX.1-dev and inherit the FLUX.1 [dev] Non-Commercial License. Use them only under its terms. * DreamBooth-30 adapters were trained on the DreamBooth dataset (Ruiz et al., CVPR 2023, https://github.com/google/dreambooth). CC101-20 adapters were trained on CustomConcept101 (Kumari et al., CVPR 2023, https://github.com/adobe-research/custom-diffusion). The images are not redistributed here; see the original datasets for their terms. * SCIdentity and SC-40 adapters were trained on synthetic images generated for the paper. Their training data is not released; the generation protocol is described in the paper appendix. ## Citation ```bibtex @inproceedings{jiang2026lithe, title = {{LITHE}: Lattice-Indexed Twin Hadamard Encoding for Diffusion Personalization}, author = {Jiang, Jian and Celiktutan, Oya and Wang, Yaohui and Ban, Yutong}, booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, year = {2026}, note = {To appear. Code: https://github.com/JianJiang1995/LITHE} } ```