Modelpublic

CuriousDragon/ettin-17m-sponsor-scriptsmith-replay3

sourceHugging Facecc-by-nc-sa-4.0updated 6d agoView on Hugging Face

0likes17downloads

Model Card

Ettin 17M sponsor detector (ScriptSmith replay, 3 epochs)

Experiment only. Three epochs from v4 over the ScriptSmith train split. Gains are concentrated on the ScriptSmith test set while the independent full-video pilot does not improve, indicating adaptation to the new source rather than a general improvement. Superseded by the combined and 1-epoch replay models.

—Status: experiment (overfits ScriptSmith; not recommended)
—Base encoder: jhu-clsp/ettin-encoder-17m @ 987607455c61e7a5bbc85f7758e0512ea6d0ae4c
—Architecture: ModernBERT-family Ettin 17M (ModernBertForTokenClassification)
—Labels: O, B-SPONSOR, I-SPONSOR, L-SPONSOR, U-SPONSOR
—Repo: CuriousDragon/ettin-17m-sponsor-scriptsmith-replay3

Use with Transformers

python

from transformers import AutoModelForTokenClassification, AutoTokenizer

repo = "CuriousDragon/ettin-17m-sponsor-scriptsmith-replay3"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo)

For full YouTube transcripts, tokenize into overlapping 768-token windows (128-token overlap), decode BILOU spans, map characters back onto caption cues, and merge gaps of 24 normalized characters or 1500 ms. See the Flow ml/sponsor_detection inference pipeline.

Cross-evaluation, raw decoder (threshold 0)

Held-out set	Window P/R/F1	Span IoU 0.5 P/R/F1	Coverage F1
Xenova test	0.938 / 0.868 / 0.901	0.895 / 0.829 / 0.861	0.913
ScriptSmith test	0.912 / 0.813 / 0.860	0.850 / 0.748 / 0.796	0.860

Calibrated decoder threshold 0.80. Frozen 39-video mixed pilot: temporal span IoU 0.5 F1 0.857; temporal coverage F1 0.896.

Intended use

On-device or local inference over English video transcripts to propose paid sponsor ranges. In Flow, SponsorBlock remains authoritative when it has segments; these weights are a fallback / shadow detector.

Limitations

—English transcripts only.
—Target is external paid sponsorships. Self-promotion, memberships, merchandise, and interaction reminders are treated as negatives.
—SponsorBlock labels are noisy and incomplete. Evaluations against them are weak labels, not ground truth.
—Full-video reconstruction interpolates timestamps inside caption cues.
—Window-level scores do not by themselves skip video; Flow stitches overlapping 768-token windows with a 128-token overlap.

ScriptSmith corpus

This model additionally used the `ScriptSmith/sponsorblock-youtube-metadata-2024` transcript/metadata dataset (CC BY 4.0 compilation; its YouTube auto-captions remain under YouTube's terms). Those transcripts are rolling auto-captions that were reconstructed before alignment, so caption boundaries are approximate. Superseded by `CuriousDragon/ettin-17m-sponsor-combined`.

License

These weights are released under CC BY-NC-SA 4.0.

The base encoder `jhu-clsp/ettin-encoder-17m` is MIT. Fine-tuning used SponsorBlock labels (CC BY-NC-SA 4.0) and the pinned `Xenova/sponsorblock-768` window corpus. Non-commercial use and share-alike apply to this derivative.

This project is not affiliated with SponsorBlock.