forkger/manga-panel-rfdetr

Manga Panel Detector (RF-DETR-Medium, Core ML and ONNX)

An object detector for manga pages with three classes: panel, text (speech and caption regions) and body (character figures). It is a fine-tune of RF-DETR Medium by forkger, trained on Manga109-s. It comes in two formats: native Core ML (FP16) for iPhone and iPad and ONNX (FP32) for Android, Windows, Linux and the web.

To test panel-by-panel reading with it, the model was integrated into KMReader, a separate open-source app, as an experiment for a community feature request. The model only outputs boxes, so any app can use it.

Files

File	Format	Size
manga-panel-rfdetr-med-576-fp16.mlpackage	Core ML ML Program (FP16), iOS 16+ / macOS 13+	58 MB
manga-panel-rfdetr-med-576.onnx	ONNX (FP32), opset 17	121 MB
NOTICE	Attribution that must stay with the model

Both formats come from the same trained checkpoint.

Input

Both formats take one page resized to 576×576 RGB. Resize the whole page directly (stretch, no letterbox).

Core ML: one image input named image. Scaling (1/255) and ImageNet normalization are built into the model. Pass plain RGB pixels.

ONNX: one float32 tensor named input, shape [1, 3, 576, 576] (NCHW). Normalization is not built in:

1. Divide the RGB values by 255.
2. Subtract the ImageNet mean [0.485, 0.456, 0.406] and divide by the std [0.229, 0.224, 0.225], per channel.
3. Reorder from HWC to CHW and add the batch dimension.

Output

Both formats return the same two arrays (FP16 in Core ML, FP32 in ONNX):

ONNX name	Shape	Meaning
dets	[1, 300, 4]	300 candidate boxes as (cx, cy, w, h), normalized to [0, 1] of the page
labels	[1, 300, 5]	Class logits per candidate

The Core ML converter named the outputs automatically, so match them by shape there.

Class channels: 1 = panel, 2 = text, 3 = body. Channels 0 and 4 are unused.

Decoding: apply a sigmoid to the logits and keep a candidate for a class when its score passes the threshold. No non-maximum suppression is needed (DETR-style set prediction). Thresholds used in KMReader:

Class	Threshold
panel	0.40
text	0.50
body	0.50

Reading order, overlap cleanup and zoom framing are not part of the model. KMReader's panel mode is one open-source implementation of them.

Compute units (Core ML)

On A14 and newer, load with MLComputeUnits.all to use the Neural Engine. Some older chips (A12) cannot build the model for the Neural Engine or GPU and fail to load with error -5; fall back to .cpuAndGPU, then .cpuOnly. In the iOS Simulator use .cpuOnly, since other settings return invalid scores there.

The first load on a device builds and caches a device-specific version: a few seconds on A18, a minute or two on A14. Later loads are fast.

Performance

Accuracy on 4 Manga109-s books (724 pages) held out from training and validation, measured on the PyTorch model before conversion:

Class	AP@[.5:.95]	AP@50
panel	0.911	0.977
text	0.866	0.976
body	0.667	0.901
all	0.815	0.951

Speed per page on device (Core ML, FP16), measured in KMReader's panel mode:

Device	Compute units	Time per page
iPhone 12 mini (A14)	Neural Engine (.all)	~85–230 ms
iPhone XR (A12)	CPU fallback (.cpuOnly)	~730 ms
iPhone 16 (A18)	Neural Engine (.all)	~30–170 ms (typically ~50 ms)

Training

• Base: RF-DETR-Medium, COCO-pretrained checkpoint.
• Data: Manga109-s (v2026), 87 volumes (15,588 single pages; two-page spreads were split). Panels come from the frame annotations, text from text, bodies from body.
• Split by book, with no book in more than one split: 70 train, 13 validation, 4 test.
• Resolution 576, 50 epochs, checkpoint chosen by best EMA validation mAP@[.5:.95].

Limitations

• Borderless panels are detected less reliably. Manga109 annotates bordered frames best, and the model inherits that. A reader should treat "no panels found" as "show the whole page".
• Manga109-s contains only Japanese manga, so expect lower accuracy on webtoons, Western comics and very different art styles.
• text is a region detector without text recognition. body boxes mark figures, not who they are.

License and attribution

• The model weights are released under Apache-2.0 by forkger, fine-tuned from RF-DETR by Roboflow (Apache-2.0).
• The model was trained on the Manga109-s dataset. Its terms require that the use of Manga109-s is indicated clearly wherever results such as pre-trained models are published.
• If you use, republish or convert this model, keep the NOTICE file with it and credit this model in your app or project, for example: "Panel detection by Manga Panel Detector (forkger), trained on Manga109-s".
• This repository contains no Manga109-s images or annotations.

Citation

If you use this model, please cite it along with Manga109 and RF-DETR:

@misc{forkger2026mangapanel,
  author       = {forkger},
  title        = {Manga Panel Detector (RF-DETR-Medium)},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/forkger/manga-panel-rfdetr}}
}

@inproceedings{baek2026mangav26,
  author    = {Baek, Jeonghun and Miyai, Atsuyuki and Onohara, Shota and Ikuta, Hikaru and Aizawa, Kiyoharu},
  title     = {Manga109-v2026: Revisiting Manga109 Annotations for Modern Manga Understanding},
  booktitle = {Culture {\times} AI Workshop at ICML 2026},
  year      = {2026},
  url       = {https://arxiv.org/abs/2605.21182}
}

@article{multimedia_aizawa_2020,
  author  = {Aizawa, Kiyoharu and Fujimoto, Azuma and Otsubo, Atsushi and Ogawa, Toru and Matsui, Yusuke and Tsubota, Koki and Ikuta, Hikaru},
  title   = {Building a Manga Dataset ``Manga109'' with Annotations for Multimedia Applications},
  journal = {IEEE MultiMedia},
  volume  = {27},
  number  = {2},
  pages   = {8--18},
  year    = {2020},
  doi     = {10.1109/mmul.2020.2987895},
  url     = {https://arxiv.org/abs/2005.04425}
}

@article{mtap_matsui_2017,
  author  = {Matsui, Yusuke and Ito, Kota and Aramaki, Yuji and Fujimoto, Azuma and Ogawa, Toru and Yamasaki, Toshihiko and Aizawa, Kiyoharu},
  title   = {Sketch-based Manga Retrieval using Manga109 Dataset},
  journal = {Multimedia Tools and Applications},
  volume  = {76},
  number  = {20},
  pages   = {21811--21838},
  year    = {2017},
  doi     = {10.1007/s11042-016-4020-z},
  url     = {https://arxiv.org/abs/1510.04389}
}

@inproceedings{robinson2026rfdetr,
  author    = {Robinson, Isaac and Robicheaux, Peter and Popov, Matvei and Ramanan, Deva and Peri, Neehar},
  title     = {RF-DETR: Real-Time Detection Transformer},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2511.09554}
}