Jeveration / README.md
linlinlinlinsdada's picture
Model card
48c66dc verified
|
Raw History Blame Contribute Delete
4.5 kB
---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
base_model_relation: adapter
pipeline_tag: text-to-image
language:
- en
tags:
- text-to-image
- image-to-image
- depth-estimation
- image-segmentation
---
# Jeveration: System One Models Now Generate
Weights of Jeveration, a frozen vision-language model that returns a whole 512 x 512 image as one joint decision in a
**single forward pass**. One checkpoint serves text-to-image, image-to-image (HED edge map or depth map to image,
colorization, three styles) and dense prediction (13-class segmentation, depth, surface normals).
- Paper: https://arxiv.org/abs/2611.24738
- Code and usage: https://github.com/Carol-lyh/Jeveration
- Project page: https://carol-lyh.github.io/Jeveration/
- Demo: https://huggingface.co/spaces/linlinlinlinsdada/Jeveration
## Files
| File | Size | What |
|---|---|---|
| `jeveration.pt` | 4.0 GB | LoRA adapters (rank 256) for the frozen Qwen3.5-4B backbone, decision-token inputs, image head, dense heads |
| `decoder512.pt` | 1.2 GB | TA-TiTok's text-aware decoder, fine-tuned for 512-px output (text-to-image) |
| `source_decoder512.pt` | 1.3 GB | the same decoder with a gated source branch (image-to-image) |
| `config.json` | 1 kB | settings of the checkpoint: the models it builds on, sizes, tasks |
The files are PyTorch state dictionaries that hold tensors only (loaded with `weights_only=True`). They are not a
`transformers` or `diffusers` checkpoint and need the code of the repository above.
## All tasks from one pipeline
```bash
git clone https://github.com/Carol-lyh/Jeveration && cd Jeveration
pip install -r requirements.txt
hf download linlinlinlinsdada/Jeveration --local-dir weights
```
The inputs of this example are in the repository's `examples/` folder.
```python
import numpy as np
from PIL import Image
from jeveration import JeverationPipeline
from jeveration.images import load_square, to_gray
pipe = JeverationPipeline("weights/jeveration.pt", decoder="weights/decoder512.pt",
source_decoder="weights/source_decoder512.pt")
# Text-to-image: a list of prompts -> uint8 images [B, 512, 512, 3]
prompt = "A lone pine tree on a small island in a calm lake at dawn."
images = pipe.text_to_image([prompt], pipe.generator(0))
Image.fromarray(images[0]).save("pine.png")
# Structure to image: an edge map ("hed") or a depth map ("depth"), with a caption of the wanted image
edges = load_square("examples/hed_puppy.png")
caption = "A fluffy white Samoyed puppy smiling in a field of yellow flowers on a sunny afternoon."
images = pipe.image_to_image(edges[None], "hed", [caption], generator=pipe.generator(0))
Image.fromarray(images[0]).save("puppy.png")
# Colorization of a grey picture
gray = to_gray(load_square("examples/gray_pancakes.jpg"))
images = pipe.image_to_image(gray[None], "colorize", generator=pipe.generator(0))
Image.fromarray(images[0]).save("pancakes.png")
# Style transfer of a photo: "anime", "ghibli" or "watercolor"
photo = load_square("examples/photo_snowcabin.jpg")
images = pipe.image_to_image(photo[None], "watercolor", generator=pipe.generator(0))
Image.fromarray(images[0]).save("watercolor.png")
# Dense prediction: class index, depth in meters and surface normal of every pixel
room = np.asarray(Image.open("examples/room_1.png").convert("RGB"))
pred = pipe.dense_predict(room[None]) # {"seg": [B, H, W], "depth": [B, H, W], "normal": [B, H, W, 3]}
```
The repository also has a command-line script for each task and a web demo (`python app.py`). Three public models are
downloaded on first use, each under its own license: the backbone `Qwen/Qwen3.5-4B`, the tokenizer
`turkeyju/tokenizer_tatitok_bl128_vq` and the CLIP ViT-L/14-336 text tower `timm/vit_large_patch14_clip_336.openai`.
## Scope
- Images are 512 x 512. Prompts and instructions are English.
- Image-to-image covers the six trained tasks with their trained instructions (`hed`, `depth`, `colorize`, `anime`,
`ghibli`, `watercolor`).
- The dense heads were trained on NYUv2 (indoor scenes, 4:3 pictures); other content or shapes are outside what was
trained.
## License
Apache License 2.0, for these weights and for the code. The models downloaded at run time keep their own licenses.
## Citation
```bibtex
@article{liu2026jeveration,
title = {Jeveration: System One Models Now Generate},
author = {Liu, Yuhe and Li, Qi and Zhang, Shihua and Wang, Xinchao},
journal = {arXiv preprint arXiv:2611.24738},
year = {2026}
}
```