Datasetpublic

anonymous-webtoon-sfx/soundsense-webtoon

Webtoon Sound-Moment Annotations Human-curated annotations of where a vertical-scroll webtoon should make a sound, with a reference audio clip attached to each moment. Built to evaluate whether vision-language models can judge, from a static drawing alone, that a depicted moment is audible. Companion resource to the paper "SoundSense: Visual Sound Grounding in Comics and Webtoons". Webtoons have no prior sound-effect annotation, and unlike manga onomatopoeia these labels are… See the full description on the dataset page: https://huggingface.co/datasets/anonymous-webtoon-sfx/soundsense-webtoon.

sourceHugging Facecc-by-4.0updated 21d agoView on Hugging Face

2likes146downloads

Dataset Card

Webtoon Sound-Moment Annotations

Human-curated annotations of where a vertical-scroll webtoon should make a sound, with a reference audio clip attached to each moment. Built to evaluate whether vision-language models can judge, from a static drawing alone, that a depicted moment is audible. Companion resource to the paper "SoundSense: Visual Sound Grounding in Comics and Webtoons".

Webtoons have no prior sound-effect annotation, and unlike manga onomatopoeia these labels are made from depicted events, not drawn sound words — so a moment can be marked even when the artist lettered nothing.

What is included

—`annotations.jsonl` — 359 curated sound moments. Each is a strip-space bounding box with a category, a short sound label, a trigger point (where in the box the sound lands while scrolling), whether drawn onomatopoeia happens to be present (has_sfx_text), and one or more reference audio clips.
—`audio/` — 421 reference clips (420 generated, 1 uploaded).
—`works.json` — per-work metadata and the public source URL.

Images are not included. These are coordinate-and-label annotations describing the source webtoons; use source_url to view the originals.

Works

Work (genre)	Moments	Episodes	Source
Renmianyu (horror)	108	4	https://mojoin.com/comics/132#chapter
Chailang (wuxia)	137	3	https://mojoin.com/comics/63#chapter
Wuqing Waisong (sci-fi)	114	3	https://mojoin.com/comics/212#chapter

Schema (`annotations.jsonl`)

json

{
  "work": "Chailang",                     // romanized title
  "work_original": "柴郎",
  "episode": "序章-執",
  "source_url": "https://mojoin.com/comics/63#chapter",
  "box": {"x": 284, "y": 187, "w": 717, "h": 543},  // pixels in the stitched vertical strip
  "category": "impact",                   // impact|motion|voice|animal|ambient|music
  "sound": "sword clash",                 // short human label
  "has_sfx_text": false,                  // did the artist draw a sound word here?
  "sfx_text": "",                         // that lettering, if any
  "trigger_frac": 0.5,                    // 0=top .. 1=bottom of the box
  "note": "",
  "audio": [
    {"file": "audio/<hash>__name.mp3", "sha1": "...", "bytes": 12345,
      "generated": true, "prompt": "a single sharp metallic clash ..."}
  ],
  "audio_default": "audio/<hash>__name.mp3"  // clip used for preview
}

Boxes are in the coordinate system of the stitched vertical strip; per-episode strip dimensions and segment counts are in works.json.

Provenance and licensing

—Annotations (boxes, labels, prompts): released under CC-BY-4.0. They were curated by a professional annotator, who reviewed every strip segment in full; a VLM was used only to accelerate candidate marking, and every moment was confirmed, positioned and given a reference clip by hand.
—Reference audio: 420 clips were generated with ElevenLabs (sound-effects/v2) under a paid plan and are redistributed here; each carries the exact prompt that produced it, so any clip is reproducible. 1 clip(s) were uploaded by the annotator.
—Source images: not redistributed. The webtoons are © their respective creators and publisher (mojoin); we release only derived coordinates/labels and link to the source.

Citation