MarioTheKnight's picture
v2.0.0: 53 labels, guideline v2.4, new weights
959c0b8 verified
|
Raw History Blame Contribute Delete
24.1 kB
---
license: apache-2.0
language:
- en
base_model: SmilingWolf/wd-eva02-large-tagger-v3
base_model_relation: finetune
pipeline_tag: image-classification
tags:
- lighting
- image-tagging
- multi-label-classification
- anime
- illustration
- booru
- danbooru
- e621
- wd-tagger
- eva02
- dataset-captioning
- distillation
---
# Booru Lighting Tagger v2.0.0
Booru Lighting Tagger adds **lighting tags** to anime-style illustrations: where the light comes from, how
contrasted, soft or colored it is, which light effects and shadows show, and what lights the scene. It
outputs 53 booru-style tags (`backlighting`, `rim_light`, `chiaroscuro`, `bloom`, `caustics`,
`dappled_sunlight`, `candlelight`…) and a tonal key (`dark`, `dim_lighting`, `high_key`).
![Night portrait: tonal key dim_lighting; tags night, cool_lighting, glowing, rim_light, dramatic_lighting, complex_lighting, bloom, hard_lighting, overlighting](examples/night-portrait.jpg)
![Beach at sunset: tonal key normal; tags sunlight, rim_light, hard_lighting, sunset, bloom, diffraction_spikes, golden_hour, backlighting](examples/beach-sunset.jpg)
![Mecha under a starry sky: tonal key normal; tags glowing, dramatic_lighting, diffraction_spikes, light_rays, blue_light, night, light_particles](examples/mecha-starfield.jpg)
![Mountain valley: tonal key normal; tags sunlight, dappled_sunlight, hard_lighting, shade](examples/mountain-valley.jpg)
*Outputs of v2.0.0 at the default thresholds, on four AI-generated illustrations that are not part of the
training data. Each bar is the score out of 10; the white tick is the label's threshold. Grey bars are labels
that are not validated against human answers (see [Labels](#labels)). The last line lists the tags that the
`recall` preset would add.*
Booru tags describe what is in an image much better than how it is lit. Most images with a strong
backlight or a dramatic chiaroscuro are not tagged as such, so generative models trained on these captions
have no reliable handle on lighting. This tagger is meant to complete the captions of image datasets, so
that lighting becomes something you can prompt for.
- **Base model**: [WD EVA02-Large Tagger v3](https://huggingface.co/SmilingWolf/wd-eva02-large-tagger-v3)
by SmilingWolf. 16 of its 24 transformer blocks were fine-tuned, under a new lighting head.
- **Teacher**: Claude Opus 5.5 rated about 25,400 images from 0 to 10 on each label, following a written
guideline; the tagger learned these ratings.
- **Checked against a human**: on blind human answers, the tagger ranks images better than its teacher
(mean AP 0.85 against 0.77 on the 51 labels measured). 38 of the 53 labels reach 70 % precision and
30 % recall (41 for the teacher).
- **New in 2.0**: 14 more labels, among them `soft_lighting`, `hard_lighting`, `bloom`, `light_rays`,
`vignetting` and the colored lights, a revised guideline, and about 6,000 more training images. Tags of
v1.x and v2.x are not consistent with each other: see [Versioning](#versioning).
- **License**: Apache 2.0, like the base model.
The project report, with the goal, method, experiments, results and limitations, is in
[REPORT.md](REPORT.md).
## Quick start
```bash
pip install -U huggingface_hub # provides the hf command
hf download MarioTheKnight/booru-lighting-tagger --local-dir booru-lighting-tagger
pip install -r booru-lighting-tagger/requirements.txt
python booru-lighting-tagger/inference.py my_image.png
```
```
my_image.png: sunlight, rim_light, hard_lighting, sunset, bloom, diffraction_spikes, golden_hour, backlighting
```
At the first run, the base model (1.2 GB) is downloaded from the Hub and cached. If you need a specific
CUDA build of PyTorch, install it first ([pytorch.org](https://pytorch.org/get-started/locally/)).
### Tag a dataset
```bash
# Add the missing lighting tags to each image's .txt caption (the file is created if needed)
python booru-lighting-tagger/inference.py dataset/ --recursive --append-to-captions
# Same, with spaces instead of underscores ("rim light")
python booru-lighting-tagger/inference.py dataset/ -r --append-to-captions --spaces
# Keep every score, to apply your own thresholds
python booru-lighting-tagger/inference.py dataset/ -r --jsonl lighting.jsonl
```
| Option | Effect |
|---|---|
| `-r`, `--recursive` | Look into subfolders. |
| `--append-to-captions [EXT]` | Add the missing tags at the end of `<image name>.txt` (or `EXT`), comma-separated. Existing tags are kept and never duplicated (`rim_light` and `rim light` count as the same tag). |
| `--write-captions [EXT]` | Write the lighting tags alone to `<image name>.lighting.txt` (or `EXT`), replacing that file. |
| `--jsonl FILE` | Write every result to a JSON Lines file: all scores, tonal key probabilities, tags. |
| `--spaces` | Write `rim light` instead of `rim_light`. |
| `--preset recall` | More tags: about 1.5 times as many, a little less precise (see [Threshold presets](#threshold-presets)). |
| `--validated-only` | Emit only the 38 validated labels (see [Labels](#labels)). |
| `--no-tonal-key` | Do not emit `dark`, `dim_lighting` or `high_key`. |
| `--seeds N` | Use only the first N of the 3 seeds: faster, slightly less accurate. |
| `--repo ID`, `--revision v2.0.0` | Download the model from the Hub instead of reading it next to the script. |
| `--device`, `--precision`, `--batch-size`, `--workers` | Hardware settings (`--help` for details). |
Formats read: PNG, JPEG, WebP, BMP, GIF (first frame), TIFF. An unreadable file is reported and skipped;
the run goes on and exits with code 2.
### Python
```python
import sys
from huggingface_hub import snapshot_download
model_dir = snapshot_download("MarioTheKnight/booru-lighting-tagger", revision="v2.0.0")
sys.path.insert(0, model_dir)
from inference import LightingTagger
tagger = LightingTagger(model_dir) # device="auto", seeds=None (all 3), preset="default"
for result in tagger.tag_files(["a.png", "b.jpg"], batch_size=8):
print(result["file"], result["tags"])
from PIL import Image
result = tagger.tag_images([Image.open("a.png")])[0]
print(result["scores"]["rim_light"]) # 0 to 10
result = tagger.tag_images([Image.open("a.png")], preset="recall")[0] # more tags, for this call only
```
### Output
One result per image (abridged; the beach image above):
```json
{
"file": "beach.png",
"tags": ["sunlight", "rim_light", "hard_lighting", "sunset", "bloom", "diffraction_spikes", "golden_hour", "backlighting"],
"scores": {"backlighting": 4.01, "rim_light": 5.36, "sidelighting": 0.15, "...": "..."},
"tonal_key": "normal",
"tonal_key_probabilities": {"dark": 0.0, "dim_lighting": 0.0, "normal": 1.0, "high_key": 0.0},
"error": null
}
```
- **`scores`**: for each of the 53 labels, the rating the teacher would most likely give, from 0 (absent)
to 10 (obvious and dominant). It is not a probability. On the teacher's scale, 3 means "hesitant,
rather no", 4 to 6 "real but subtle or secondary", 7 or 8 "clearly present". The full scale is in
[REPORT.md](REPORT.md#the-teacher-claude-opus-55).
- **`tags`**: the labels whose score reaches their threshold, highest score first, then the tonal key
unless it is `normal`. By default, each threshold makes the tagger emit as many tags as the teacher did
on the validation images. Thresholds are listed in `tagger_config.json`.
### Threshold presets
The teacher is cautious: when a tag is emitted it is usually right, but many images that deserve it are
missed. The `recall` preset (`--preset recall`, or `preset="recall"` in Python) lowers the thresholds to
emit about 1.5 times as many tags as the teacher on the validation images. Four labels keep their default
threshold, because their precision on the human tests falls below 60 % otherwise: `sidelighting`,
`soft_lighting`, `overexposure` and `night`. A recall threshold is never higher than the default one: the
recall preset only adds tags.
| Preset | Mean precision | Mean recall |
|---|---:|---:|
| `default` | 89 % | 55 % |
| `recall` | 81 to 84 % | 63 % |
Means over the labels measured on the human tests. The four exceptions were chosen on these tests, so the
recall preset's figures are a little optimistic there: 84 % precision on the tests, 81 % in
cross-validation (exceptions chosen on half of the images, measured on the other half). With the recall
preset, `silhouette`, `volumetric_lighting`, `colored_lights` and `shadow` reach 70 % precision and 30 %
recall. Per-label figures are in [REPORT.md](REPORT.md#threshold-presets). For finer control, keep every
score with `--jsonl` and apply your own thresholds.
### Speed and memory
| | 3 seeds (default) | 1 seed |
|---|---|---|
| GPU, RTX PRO 6000 Blackwell, batches of 16 | 39 images/s; 3.8 GB of VRAM, 4.6 GB at peak | 85 images/s; 1.3 GB, 2.1 GB at peak |
| CPU, 16 threads | about 3 s per image (measured with v1, same architecture) | not measured |
On a GPU, the tagger runs in bfloat16 as in training (float16 on GPUs without bfloat16). On a CPU it runs
in float32. The thresholds were set for the average of the 3 seeds; with fewer seeds they still apply, a
little less precisely (a single seed: mean validation AP 0.725 to 0.728, against 0.745 for the average).
## Labels
<!-- labels-table:start -->
| Family | Tag | What it means | Status | Precision | Recall | Human positives |
|---|---|---|---|---:|---:|---:|
| Direction | `backlighting` | Main light behind the subject, facing the viewer | validated | 86 % | 59 % | 32 |
| Direction | `rim_light` | Bright edge outlining the subject | low recall | 75 % | 11 % | 28 |
| Direction | `sidelighting` | Light from one side, the other half in marked shadow | low precision | 50 % | 13 % | 23 |
| Direction | `overlighting` | Light from above, with marked downward shadows | validated | 94 % | 34 % | 50 |
| Direction | `underlighting` | Light from below (lamp, fire, screen) | validated | 100 % | 35 % | 31 |
| Direction | `spotlight` | A targeted beam isolates the subject | validated | 71 % | 83 % | 12 |
| Direction | `crack_of_light` | Light entering a dark space through a gap | validated | 100 % | 58 % | 12 |
| Contrast | `high_contrast` | Deep blacks against bright highlights, few midtones | validated | 86 % | 44 % | 27 |
| Contrast | `low_contrast` | Compressed tones: soft, hazy, washed-out | low recall | 100 % | 14 % | 14 |
| Contrast | `chiaroscuro` | Light and shadow structure the whole composition | validated | 100 % | 49 % | 35 |
| Contrast | `silhouette` | Subject reduced to a dark outline against light | low recall | 100 % | 17 % | 18 |
| Quality | `soft_lighting` *(new)* | Volumes modeled by soft, gradual shadows | low precision | 63 % | 74 % | 25 |
| Quality | `hard_lighting` *(new)* | Sharp-edged shadows and marked highlights | validated | 90 % | 65 % | 16 |
| Quality | `dramatic_lighting` | Light staged for mood, like a film shot | validated | 75 % | 79 % | 42 |
| Quality | `volumetric_lighting` | Light visible in the air: mist, dust, beams | low recall | 91 % | 21 % | 48 |
| Quality | `complex_lighting` *(new)* | Two or more distinct sources light the subject | validated | 79 % | 45 % | 10 |
| Color | `warm_lighting` | Clearly warm light tinting the subject | validated | 100 % | 59 % | 17 |
| Color | `cool_lighting` | Clearly cool light tinting the subject | low recall | 100 % | 11 % | 18 |
| Color | `golden_hour` | Low golden sun after sunrise or before sunset | validated | 80 % | 76 % | 21 |
| Color | `colored_lights` *(new)* | Several saturated lights of different colors | low recall | 88 % | 28 % | 17 |
| Color | `red_light` | Saturated red light tinting the scene | validated | 91 % | 74 % | 27 |
| Color | `orange_light` | Saturated orange light tinting the scene | low recall | 100 % | 19 % | 21 |
| Color | `yellow_light` *(new)* | Saturated yellow light tinting the scene | validated | 100 % | 35 % | 20 |
| Color | `green_light` | Saturated green light tinting the scene | low precision | 67 % | 92 % | 13 |
| Color | `blue_light` *(new)* | Saturated blue light tinting the scene | validated | 91 % | 62 % | 16 |
| Color | `purple_light` | Saturated purple light tinting the scene | validated | 88 % | 70 % | 20 |
| Color | `pink_light` *(new)* | Saturated pink light tinting the scene | validated | 91 % | 56 % | 18 |
| Effects | `light_rays` *(new)* | Distinct beams of light crossing the air | validated | 100 % | 54 % | 14 |
| Effects | `light_particles` | Shining particles floating in the air | validated | 82 % | 49 % | 37 |
| Effects | `lens_flare` | Lens artifact: streaks, halos, aligned circles | validated | 100 % | 43 % | 14 |
| Effects | `bloom` *(new)* | Blurry glow spilling over nearby edges | validated | 78 % | 78 % | 20 |
| Effects | `overexposure` | Highlights blown out to pure white | does not work | – | 0 % | 24 |
| Effects | `diffraction_spikes` | Star-shaped spikes around a bright source | validated | 91 % | 77 % | 13 |
| Effects | `caustics` | Rippling light patterns cast by water or glass | validated | 94 % | 74 % | 23 |
| Effects | `refraction` | Image bent or shifted through water or glass | validated | 100 % | 54 % | 13 |
| Effects | `dispersion_(optics)` *(new)* | Light split into rainbow colors (prism, crystal) | validated | 94 % | 88 % | 17 |
| Effects | `subsurface_scattering` | Light glowing through skin, leaves, wax | low precision | 50 % | 10 % | 10 |
| Effects | `vignetting` *(new)* | Edges and corners of the image gradually darken | validated | 100 % | 71 % | 20 |
| Shadows | `shadow` | Strong or striking cast shadows | low recall | 100 % | 18 % | 11 |
| Shadows | `shade` *(new)* | Subject in the shade while strong light hits the rest | not measured | 75 % | 38 % | 8 |
| Shadows | `dappled_sunlight` | Patches of light through foliage | validated | 95 % | 86 % | 21 |
| Shadows | `window_shadow` | Light and shadow pattern from a window or blinds | validated | 89 % | 75 % | 32 |
| Shadows | `colored_shadow` *(new)* | Shadows with a clear hue, not just darker | not measured | – | – | – |
| Shadows | `shaded_face` *(new)* | Upper face or eyes in shadow | validated | 91 % | 60 % | 24 |
| Shadows | `partially_shaded_face` | Face split into a lit half and a shaded half | validated | 89 % | 33 % | 24 |
| Sources | `sunlight` | Sunlight emphasized | validated | 100 % | 38 % | 26 |
| Sources | `moonlight` | Pale moonlight at night | validated | 81 % | 100 % | 17 |
| Sources | `candlelight` | Scene lit mostly by candles | validated | 90 % | 95 % | 19 |
| Sources | `fire_light` | Scene lit by the orange glow of a fire | validated | 100 % | 73 % | 26 |
| Sources | `screen_light` | A visible screen lights a character | validated | 88 % | 91 % | 23 |
| Sources | `glowing` | An object or character emits light | validated | 85 % | 74 % | 23 |
| Sources | `night` | Night scene | validated | 92 % | 79 % | 14 |
| Sources | `sunset` | Sun near the horizon, orange or red sky | validated | 100 % | 92 % | 37 |
<!-- labels-table:end -->
Precision and recall are measured at the tagger's default thresholds against blind answers from one human
annotator, on about 1,600 test images never seen in training (near-duplicates included). The test sets are
small: the figures of a single label can move by 10 to 20 points.
- **validated** (38 labels): at least 70 % precision and 30 % recall, on at least 10 human positives.
`complex_lighting` passes with exactly 10 positives: treat it as fragile.
- **low recall** (8): the tag is usually right when emitted, but it misses many cases. The `recall` preset
brings `silhouette`, `volumetric_lighting`, `colored_lights` and `shadow` over the bar, and `rim_light`
and `low_contrast` close to it.
- **low precision** (4): the tag is emitted too often. `soft_lighting` was kept as a trial: when the
guideline was set, the teacher's own ratings never reached 75 % precision against the human. Use it with
care.
- **does not work** (1): `overexposure` is never emitted at its threshold, and its ranking is weak (AP
0.49). It is kept in the outputs for compatibility; do not rely on it. In `tagger_config.json` its status
is `low_recall`.
- **not measured** (2): `shade` has only 8 human positives in the tests. `colored_shadow` was not measured
against the human on purpose: human answers on colored shadows were not reliable enough to serve as a
reference, so the teacher's judgment is the reference for this label.
The **tonal key** (`dark`, `dim_lighting`, `normal`, `high_key`) is a single choice per image. It agrees
with the teacher on 88 % of the validation images, but has no human reference.
The exact definitions, as given to the teacher, are in `tagger_config.json`. They share a few rules: the
light is judged, not the drawing style (cel-shading, inking or an overall color grading do not count); an
artificial source (screen, candles, fire) or its glow must be visible; the sun and the moon may be off
frame.
Tag names follow Danbooru where the tag exists there, otherwise e621.
## How it was made
1. **Taxonomy.** 53 lighting labels in 7 families, plus the tonal key, each with a written definition.
2. **Teacher.** Claude Opus 5.5 rated each image from 0 to 10 on every label, following a fixed
guideline (v2.4). The guideline was revised several times after discussing its disagreements with a
human; versions 2.2 to 2.4 recovered the 13 labels that v1 had to leave out and added
`dispersion_(optics)`.
3. **Human calibration.** One annotator answered yes or no, blind and one label at a time, on test images.
These answers set a threshold per label on the teacher's ratings and serve as the test reference.
4. **Training data.** About 25,400 images from Danbooru, Gelbooru, e621, yande.re and Konachan, all rated
by the teacher with guideline v2.4 (about 29 M tokens): the v1 images, rated again, and a new batch of
5,932 images aimed at the weakest labels. Half of the v1 images and a third of the new batch were drawn
at random, the rest by booru tag to enrich the rare labels. No explicit images, no AI-generated images,
no comic pages. 23,426 images were used for training and 1,946 for validation; images too close to a
test image were removed.
5. **Student.** WD EVA02-Large Tagger v3 with its last 16 blocks unfrozen (202 M trainable parameters).
For each label, the head predicts P(rating ≥ t) for t = 1 to 10, and the score is their sum. 8 epochs,
AdamW, horizontal flips only, since color or crop augmentations would change the lighting. Three seeds,
whose scores are averaged.
6. **Thresholds.** For each label, the threshold that gives as many tags as the teacher on the validation
images.
Details, experiments and ablations: [REPORT.md](REPORT.md). An example training script, for reference,
shows the method: [training/README.md](training/README.md).
## Results
| Model | Validation: mean AP against the teacher (53 labels) | Tests: mean AP against the human (51 labels) | Validated labels |
|---|---:|---:|---:|
| One seed alone | 0.725 to 0.728 | not measured | – |
| **This model** (3 seeds averaged) | **0.745** (95 % CI 0.730–0.770) | **0.853** | **38** |
| Teacher (Claude Opus 5.5) | – | 0.769 | 41 |
Mean validation AP by family: light direction 0.64, contrast 0.76, light quality 0.81, light color 0.72,
light effects 0.74, shadows 0.66, sources and time of day 0.91.
The tagger beats its teacher on the human tests partly because it learns the teacher's average rating,
which smooths out the teacher's run-to-run noise, and because its scores are continuous where the
teacher's ratings are whole numbers. It still validates three labels fewer than the teacher: its default
thresholds copy the teacher's cautious tagging rate, and the teacher's own thresholds were set on these
same human answers, which favors it. See the report for the caveats.
These figures are **not comparable with v1.x**: the labels, the guideline, the teacher's ratings and the
human references all changed.
## Limitations
- **Domain.** Trained on anime, manga, furry and game-style illustrations from image boards, with a few
hundred 3D renders. Photos, comic pages and AI-generated images were not in the training data, and the
tagger was not measured on them.
- **Ratings.** General and sensitive images, plus a few hundred e621 images rated questionable. No explicit
image was used; behavior on explicit images was not measured.
- **Recall over precision.** The default thresholds copy the teacher's tagging rate, which is cautious:
when a tag is emitted it is usually right, but many images that deserve it are missed. Use
`--preset recall`, or `--jsonl` and your own thresholds, if you need more recall.
- **It imitates a teacher.** The tagger reproduces the judgment of Claude Opus 5.5 under one guideline,
including its blind spots. The human reference comes from a single annotator.
- **Weak spots.** Light direction (validation AP 0.64) and shadows (0.66) are the weakest families.
Fifteen labels are not validated: `rim_light`, `low_contrast`, `silhouette`, `volumetric_lighting`,
`cool_lighting`, `colored_lights`, `orange_light` and `shadow` have a low recall; `sidelighting`,
`soft_lighting`, `green_light` and `subsurface_scattering` have a low precision; `overexposure` does not
work; `shade` and `colored_shadow` are not measured against the human.
- **Resolution.** Images are padded to a square and seen at 448 × 448 px. Fine effects (small caustics,
thin rims) can be lost, especially on very tall or wide images.
- **Cost.** Three EVA02-Large copies make it about three times slower than a single tagger; `--seeds 1`
trades a little accuracy for speed.
## Intended use
- Completing the captions of image datasets before training or fine-tuning generative models.
- Searching, filtering or describing a collection of illustrations by lighting.
- Pre-sorting images before a more expensive annotation. This works less well than in v1: on random
validation images, recovering 90 % of the teacher's tags on every label means sending it 79 % of the
images, because `soft_lighting`, `hard_lighting` and `bloom` are frequent. Without these three labels,
52 % of the images are enough.
It is not meant to judge artistic quality, and it was not designed for photographs.
## Versioning
Versions are `MAJOR.MINOR.PATCH`. MAJOR changes the labels or their meaning, MINOR brings new weights for
the same labels, and PATCH only touches thresholds, scripts or documentation. Version 2.0.0 is a major
version: 14 labels were added and several definitions changed, so captions made with v1.x and v2.x should
not be mixed. Each version is a tag of this repository; pin one with `--revision v2.0.0` or
`revision="v2.0.0"` (`v1.0.2` for the previous model). History: [CHANGELOG.md](CHANGELOG.md).
## License and attribution
Apache License 2.0 ([LICENSE](LICENSE)), for the weights, the code and the documentation.
This model is a modified version of
[WD EVA02-Large Tagger v3](https://huggingface.co/SmilingWolf/wd-eva02-large-tagger-v3) by SmilingWolf,
released under the same license. The changes are listed in [NOTICE](NOTICE). The weight files contain only
the fine-tuned parameters; the rest is loaded from the original repository, at the revision used for
training.
The training labels were produced with Claude Opus 5.5 by Anthropic. The training images belong to their
artists and are not distributed here.
## Citation
```bibtex
@misc{booru_lighting_tagger_2026,
title = {Booru Lighting Tagger: lighting tags for anime-style illustrations},
author = {MarioTheKnight},
year = {2026},
version = {2.0.0},
howpublished = {\url{https://huggingface.co/MarioTheKnight/booru-lighting-tagger}}
}
```