Text-to-Image
Diffusers
TensorBoard
Safetensors
Swedish
moss-image
handwriting
historical-documents
archives
htr
synthetic-data
Instructions to use Gabriel/diffu-moss-r6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Gabriel/diffu-moss-r6b with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Gabriel/diffu-moss-r6b", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
Model card: how to draw with a sample page (a run-file line, the caption form, the draw settings, one command)
Browse files
README.md
CHANGED
|
@@ -52,10 +52,32 @@ uv run diffu-run --jobs run.jsonl --draw-settings draw.json --model ../moss-diff
|
|
| 52 |
```
|
| 53 |
|
| 54 |
`diffu-moss-init` reads `export.json` and writes the sample's size into the model directory, so the sample page is
|
| 55 |
-
sized as in training (`legible`: a whole page's sample at the drawn page's pixel count).
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
Recommended drawing: about 1 MP, 50 steps, guidance 3, shift 5, block 22 skipped in the unconditional pass (MOSS-Image's
|
| 61 |
own recipe), and **reference guidance 2** (a third pass toward the sample: `--reference-guidance 2` in
|
|
|
|
| 52 |
```
|
| 53 |
|
| 54 |
`diffu-moss-init` reads `export.json` and writes the sample's size into the model directory, so the sample page is
|
| 55 |
+
sized as in training (`legible`: a whole page's sample at the drawn page's pixel count).
|
| 56 |
+
|
| 57 |
+
### With a sample page
|
| 58 |
+
|
| 59 |
+
The sample page is an ordinary scan (JPEG or PNG) of a page whose **paper, ink and look** the drawn page should have:
|
| 60 |
+
best a page of the same volume as the layout, or any page whose look you want. It does not set the handwriting (see
|
| 61 |
+
Limitations). Each page job in `run.jsonl` is one line of JSON: the layout (`template`, a PAGE/ALTO file, or `layout`,
|
| 62 |
+
the lines' boxes in pixels), the text (left out: the layout's own), the style, the sample page under `reference`, and
|
| 63 |
+
an id:
|
| 64 |
+
|
| 65 |
+
```json
|
| 66 |
+
{"id": "court-0001", "template": "templates/0105.xml", "text": "Anno 1649 den 6 Aprilis Tå laga tingh höltz medh Allmogen ...", "style": {"provenance": {"title": "the court records of Jämtlands domsaga", "years": "1649"}, "look": {"greyscale": false, "paper": "cream", "ink": "black"}}, "reference": {"image": "samples/court-sample.jpg", "caption": "sample"}}
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
`"caption": "sample"` is the prompt form this model was trained on most (the sample page alone, nothing named; the
|
| 70 |
+
style's title and look are then left out of the prompt). `"sample_look"` adds the look in words beside the sample.
|
| 71 |
+
`draw.json` holds the draw settings, for example `{"steps": 50, "device": "cuda", "reference_guidance_scale": 2}`.
|
| 72 |
+
Each page comes out as `<id>.jpg` with its `<id>.alto.xml` and `<id>.page.xml`. `docs/INFERENCE.md` section 5
|
| 73 |
+
describes every field, with a worked example of each combination.
|
| 74 |
+
|
| 75 |
+
One page from the command line, without a run file:
|
| 76 |
+
|
| 77 |
+
```bash
|
| 78 |
+
uv run diffu-generate --model ../moss-diffu --templates templates/0105.xml --text-from text.txt \
|
| 79 |
+
--reference samples/court-sample.jpg --reference-caption sample --reference-guidance 2 --whole-page --out pages
|
| 80 |
+
```
|
| 81 |
|
| 82 |
Recommended drawing: about 1 MP, 50 steps, guidance 3, shift 5, block 22 skipped in the unconditional pass (MOSS-Image's
|
| 83 |
own recipe), and **reference guidance 2** (a third pass toward the sample: `--reference-guidance 2` in
|