mednasserallah/manga109-segmentation-bubble-onnx

Manga speech-bubble segmentation (YOLO11-nano) — ONNX export

This is not an original model. It is an ONNX export of huyvux3005/manga109-segmentation-bubble, published here because the original repository ships PyTorch weights only — and exporting the ONNX requires ultralytics, which requires PyTorch, which is more to install than most applications that want to run this model.

All credit for the model belongs upstream. If you use it, cite the original and the datasets it was trained on (below), not this repository.

What changed

Nothing but the format. No retraining, no fine-tuning, no quantisation, no change to the weights.

pip install ultralytics onnx
yolo export model=best.pt format=onnx imgsz=1024 opset=17

Note the input size. The original was trained at 1600×1600 and this is exported at 1024, which was measured rather than assumed. Over 433 text blocks of five manga series, put through the same downstream geometry:

One block in 433 for 2.3× the time. 1600 is slightly better on dense scanned work and slightly worse on clean digital releases, which is shuffling rather than a difference. Export it yourself at another size with the command above if your pages are unusual.

Outputs

A YOLO11 segmentation head, so two tensors rather than one:

A balloon's mask is sigmoid(coefficients @ prototypes), cropped to that balloon's own box. The crop is not optional: the prototypes are global, so an uncropped mask lights up faintly across the whole page. Ultralytics does this in its own post-processing; anything decoding the raw outputs has to do it too.

The head is not end-to-end, so non-maximum suppression is still required.

Usage

import numpy as np
import onnxruntime
from PIL import Image

session = onnxruntime.InferenceSession(
    "manga109_segmentation_bubble_1024.onnx", providers=["CPUExecutionProvider"]
)

# Letterbox the page into 1024×1024 on a (114, 114, 114) canvas, then:
tensor = np.asarray(canvas, dtype=np.float32).transpose(2, 0, 1)[None] / 255.0
proposals, prototypes = session.run(None, {"images": tensor})

A complete decode — letterbox, NMS, per-box mask crop, and the mapping back into the page's own pixels — is in UchiManga's backend/app/services/ocr/bubble_model.py, which is what this export was made for.

Verifying this file

sha256  593cae61f4c9ffc773fee34d55c28069390d19415c24dc04600f983311618ea3

Licence and credit

Apache-2.0, following the upstream model.

The training data carries its own terms, and they are not Apache-2.0. The model was trained on MS92/MangaSegmentation and on Manga109:

• —MangaSegmentation's annotations and images are copyrighted by Minshan Xie, who grants academic and commercial use provided that credit is carried in any publication, reproduction or redistribution. That credit is this paragraph.
• —Manga109 is licensed for academic use; commercial use requires the separate Manga109-s agreement. Anyone putting this model to commercial use should satisfy themselves about that directly rather than relying on the Apache-2.0 tag on the weights, which is the model author's declaration about the weights and not the dataset owners' about the data.

Nothing in this repository redistributes either dataset.

Citation

Cite the upstream model and the datasets, not this export.

@misc{mangalens2024,
  title  = {MangaLens: YOLO11n Speech Bubble Segmentation Model},
  author = {MangaLens Team},
  year   = {2024},
  publisher = {Hugging Face},
  url    = {https://huggingface.co/huyvux3005/manga109-segmentation-bubble}
}

@misc{xie2024mangasegmentation,
  title  = {Advancing Manga Analysis: Comprehensive Segmentation
            Annotations for the Manga109 Dataset},
  author = {Minshan Xie},
  year   = {2024},
  doi    = {10.57967/hf/2581},
  url    = {https://huggingface.co/datasets/MS92/MangaSegmentation}
}

@article{mtap_matsui_2017,
  title   = {Sketch-based Manga Retrieval using Manga109 Dataset},
  author  = {Matsui, Yusuke and Ito, Kota and Aramaki, Yuji and
             Fujimoto, Azuma and Ogawa, Toru and Yamasaki, Toshihiko and
             Aizawa, Kiyoharu},
  journal = {Multimedia Tools and Applications},
  volume  = {76},
  number  = {20},
  pages   = {21811--21838},
  year    = {2017}
}

@article{multimedia_aizawa_2020,
  title   = {Building a Manga Dataset ``Manga109'' with Annotations for
             Multimedia Applications},
  author  = {Aizawa, Kiyoharu and Fujimoto, Azuma and Otsubo, Atsushi and
             Ogawa, Toru and Matsui, Yusuke and Tsubota, Koki and
             Ikuta, Hikaru},
  journal = {IEEE MultiMedia},
  volume  = {27},
  number  = {2},
  pages   = {8--18},
  year    = {2020}
}