Blame for README.md

---
library_name: diffusers
base_model: black-forest-labs/FLUX.1-dev
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
tags:
- lora
- flux
- compression
- text-to-image
- diffusers
---

# LITHE adapters for FLUX.1-dev

This repository holds the 110 rank-16 FLUX.1-dev adapters evaluated in the NeurIPS 2026 paper "LITHE: Lattice-Indexed Twin Hadamard Encoding for Diffusion Personalization" (to appear), stored in the LITHE index format. Code: https://github.com/JianJiang1995/LITHE

## What the files are

Each `.lithe` file is one adapter. It contains no float weights: every 64-value block of every LoRA factor is stored as three 9-bit codeword IDs into a fixed Hadamard codebook that is rebuilt from a closed-form rule at load time (nothing is learned or shipped besides the IDs). The ID stream (uint16) is compressed with LZMA (preset 9, extreme) and preceded by a small compressed JSON header with the module names, factor shapes, codebook parameters, checksums and the training metadata listed in `manifest.csv`.

* Configuration: Kohya sd-scripts FLUX layout, 304 fused linears per adapter (attention, MLP and modulation linears of all 19 double-stream and 38 single-stream blocks; header field `module_set = kohya-304`), rank 16, alpha 16, K = 512 codewords, 3 residual levels, block size 64. This is the configuration used for the quality results of the paper.
* Size: 3,589,632 indices per adapter; 3.23 MB per file on average (355.7 MB for all 110). The decoded bf16 LoRA is about 153 MB.
* The adapters were trained natively in the discrete space (straight-through estimator), not quantised after training.

## Usage

```bash
git clone https://github.com/JianJiang1995/LITHE.git && cd LITHE
pip install -e ".[flux]"
huggingface-cli download Fanjue/LITHE --local-dir ./lithe_adapters
python scripts/decode_to_lora.py --lithe ./lithe_adapters/indices/dreambooth30/db_cat.lithe --out db_cat_lora.safetensors
```

The decoded file uses Kohya LoRA keys, which the stock diffusers loader converts on load:

```python
import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("cuda")
pipe.load_lora_weights("db_cat_lora.safetensors")
image = pipe("a photo of a <v_sub_001> cat on a beach", num_inference_steps=28, guidance_scale=3.5).images[0]
```

Or decode in memory without writing a file:

```python
import lithe
sd = lithe.to_lora_state_dict(lithe.load_lithe("lithe_adapters/indices/dreambooth30/db_cat.lithe"))
pipe.load_lora_weights(sd)
```

`scripts/infer.py --mode peft` and `scripts/serve_demo.py` in the code repository also accept these files. Each file's trigger token and an example prompt are in its header metadata and in `manifest.csv`. The paper evaluated at 1024 x 1024, 28 steps, guidance 3.5.

## Contents

| Pool | Folder | Adapters | Trigger tokens | Training | Mean size | Notes |
|---|---|---|---|---|---|---|
| DreamBooth-30 | `indices/dreambooth30/` | 30 | `<v_sub_001>` ... `<v_sub_030>` | 500 steps, 512x512 | 3.31 MB | Subjects of the public DreamBooth dataset (objects and pets), 4 to 6 images each. |
| CC101-20 | `indices/cc101_20/` | 20 | `<v_sub_C001>` ... `<v_sub_C020>` | 500 steps, 512x512 | 3.32 MB | 20 subjects sampled from the public CustomConcept101 dataset, one per category, 3 to 10 images each. |
| SCIdentity | `indices/scidentity/` | 20 | `<v_id_001>` ... `<v_id_020>` | 500 steps, 512x512 | 3.27 MB | 20 synthetic animal identities (12 generated images each; images not released). |
| SC-40 | `indices/sc40/` | 40 | `<v_outfit_001>` ... `<v_outfit_008>`; `<v_light_001>` ... `<v_light_008>`; `<v_pose_001>` ... `<v_pose_008>`; `<v_pstyle_001>` ... `<v_pstyle_008>`; `<v_anime_001>` ... `<v_anime_008>` | 750 steps, 512x512 | 3.11 MB | 40 synthetic concept-transfer targets: 8 outfits, 8 lighting setups, 8 poses, 8 painter styles, 8 anime styles (20 generated images each; images not released). |

Prompt pattern: identity adapters were trained with a single caption such as `a photo of a <v_sub_001> cat`. SC-40 adapters were trained on per-image captions that contain the trigger and the concept label; `manifest.csv` gives the evaluation prompt used for each (SC-40 groups: anime style 8, lighting 8, outfit 8, painter style 8, pose 8).

`manifest.csv` columns: pool, file, concept_id, trigger, description, example_prompt, module_set, n_modules, rank, n_indices, bytes, sha256, train_steps, train_resolution. `SHA256SUMS` lists checksums of all files.

## Data and license

* The adapters are derived from FLUX.1-dev and inherit the FLUX.1 [dev] Non-Commercial License. Use them only under its terms.
* DreamBooth-30 adapters were trained on the DreamBooth dataset (Ruiz et al., CVPR 2023, https://github.com/google/dreambooth). CC101-20 adapters were trained on CustomConcept101 (Kumari et al., CVPR 2023, https://github.com/adobe-research/custom-diffusion). The images are not redistributed here; see the original datasets for their terms.
* SCIdentity and SC-40 adapters were trained on synthetic images generated for the paper. Their training data is not released; the generation protocol is described in the paper appendix.

## Citation

```bibtex
@inproceedings{jiang2026lithe,
  title     = {{LITHE}: Lattice-Indexed Twin Hadamard Encoding for Diffusion Personalization},
  author    = {Jiang, Jian and Celiktutan, Oya and Wang, Yaohui and Ban, Yutong},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  note      = {To appear. Code: https://github.com/JianJiang1995/LITHE}
}
```