| --- |
| license: apache-2.0 |
| pipeline_tag: text-to-image |
| library_name: pjit-21k |
| tags: |
| - text-to-image |
| - diffusion |
| - pixel-space |
| - research |
| --- |
| |
| # A Text-to-Image Model Trained for $21K<br>(Pyramid-JiT) |
|
|
| **[Sahil Chopra](https://thesahilchopra.com/) 路 [Manu Chopra](https://www.linkedin.com/in/manu-chopra-50360b170/)** 路 *Linum* |
|
|
| [Blog post](https://www.linum.ai/field-notes/pjit-21k) 路 [Code](https://github.com/Linum-AI/pjit-21k) 路 Apache-2.0 |
|
|
|  |
|
|
| *Five samples from the [blog post](https://www.linum.ai/field-notes/pjit-21k)'s gallery. Their |
| prompts and seeds are in the [code repository](https://github.com/Linum-AI/pjit-21k#reproducing-the-samples).* |
|
|
| > [!WARNING] |
| > **Research checkpoint: this is an experimental research artifact, not a production model.** |
| > P-JiT 21K is a 2B-parameter model trained on 660M samples for about $21K of compute, as a |
| > stepping stone towards our next video model. We're releasing it because the results |
| > surprised us for a model this size, and to encourage others to explore efficient training |
| > methods like ours. You should expect rough edges. |
|
|
| > [!IMPORTANT] |
| > **This model cannot write text or generate posters.** It was trained on little to no images |
| > of rendered text: no infographics, PDFs, pages of books, slides or posters. We filtered that |
| > data out on purpose (it isn't useful for our goal of animation tools), so prompts that ask |
| > for words, signs, logos, title cards or posters will not work. |
|
|
| P-JiT 21K is the [P-JiT](https://www.linum.ai/field-notes/pyramid-jit) architecture |
| trained for longer (610M samples at 512px, 50M at 1024px), then post-trained with a variant of |
| [FDFO](https://arxiv.org/abs/2603.12893). |
|
|
| ## Usage |
|
|
| The model was trained on dense captions of 50 to 80 words. The |
| [code repository](https://github.com/Linum-AI/pjit-21k) ships a prompt upsampler that rewrites |
| short prompts into captions of that kind with Claude Opus 5.5 through OpenRouter (it needs an |
| `OPENROUTER_API_KEY`; `--no_upsample` turns it off): |
|
|
| ```bash |
| git clone https://github.com/Linum-AI/pjit-21k && cd pjit-21k |
| pip install -e . |
| |
| # The portrait above (writes outputs/seed_2.png) |
| PROMPT="Close-up portrait of an elderly Middle Eastern woman looking calmly into the lens, her face filling the tall frame. A cream headscarf and the collar of a dark green cardigan frame her chin. Soft overcast daylight smooths the fine lines of her skin, and the olive grove behind melts into silvery blur." |
| python generate.py --no_upsample --aspect_ratio tall --seeds 2 --prompt "$PROMPT" |
| |
| # Your own prompt, upsampled |
| export OPENROUTER_API_KEY=... |
| python generate.py --prompt "oil painting of weathered fisherman smoking a pipe" --aspect_ratio square --seeds 0 |
| ``` |
|
|
| The prompts and seeds of all five samples are in the |
| [code repository](https://github.com/Linum-AI/pjit-21k#reproducing-the-samples). |
|
|
| Sampling defaults: 50 Euler steps, adaptive projected guidance at scale 15 against an empty |
| negative prompt, initial noise at scale 4. The model was trained at five ~1 MP sizes: |
| 1024x1024, 864x1216, 1152x864, 1248x832 and 1280x720 (width x height). Text is encoded by a |
| frozen [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), which downloads separately. |
|
|
| ## Files |
|
|
| | File | | |
| |---|---| |
| | `model.safetensors` | The DiT, fp32 (2.18B parameters, 8.7 GB). The training-only readout heads and PixelREPA adapter are not included | |
| | `config.json` | Architecture, text-encoder layers and sampler defaults | |
|
|
| ## Limitations |
|
|
| A small model (2B DiT, 4B text encoder), so it struggles with full-body shots, many entities |
| (six cars, crowds), interactions between several people, and world knowledge (six strings on a |
| guitar, the dots on a die). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @online{chopra2026pjit21k, |
| title = {Training a Text-to-Image Model for \$21K}, |
| author = {Chopra, Sahil and Chopra, Manu}, |
| year = {2026}, |
| url = {https://www.linum.ai/field-notes/pjit-21k} |
| } |
| ``` |
|
|
| ## Authorship |
|
|
| The code for this model was written by Claude (Anthropic's Opus 5.5 model, running in Claude |
| Code). Linum asked it to extract the model and inference code from Linum's internal experiment |
| repository and to add the prompt upsampler. The model, the training, and the review of the |
| code and this model card are Linum's. |
|
|
| ## About Linum |
|
|
| Linum is a team of two brothers building a tiny-yet-powerful AI research lab. We train our |
| own generative media models from scratch. |
|
|
| **Subscribe to [Field Notes](https://buttondown.com/linum-ai)**: technical deep dives on |
| building generative video models from the ground up, plus updates on new releases from Linum. |
|
|
| **Contact:** hello@linum.ai |
|
|
