Modelpublic

CuriousDragon/ettin-17m-sponsor-scriptsmith-replay

sourceHugging Facecc-by-nc-sa-4.0updated 6d agoView on Hugging Face

0likes18downloads

Model Card

Ettin 17M sponsor detector (ScriptSmith replay, 1 epoch)

Challenger. One low-learning-rate epoch from v4 over the 209,390-window ScriptSmith auto-caption train split only. Slightly better recall than the combined model on the Xenova test set. Calibrated decoder threshold 0.75. Kept for head-to-head comparison on a larger frozen video set.

—Status: challenger
—Base encoder: jhu-clsp/ettin-encoder-17m @ 987607455c61e7a5bbc85f7758e0512ea6d0ae4c
—Architecture: ModernBERT-family Ettin 17M (ModernBertForTokenClassification)
—Labels: O, B-SPONSOR, I-SPONSOR, L-SPONSOR, U-SPONSOR
—Repo: CuriousDragon/ettin-17m-sponsor-scriptsmith-replay

Use with Transformers

python

from transformers import AutoModelForTokenClassification, AutoTokenizer

repo = "CuriousDragon/ettin-17m-sponsor-scriptsmith-replay"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo)

For full YouTube transcripts, tokenize into overlapping 768-token windows (128-token overlap), decode BILOU spans, map characters back onto caption cues, and merge gaps of 24 normalized characters or 1500 ms. See the Flow ml/sponsor_detection inference pipeline.

Cross-evaluation, raw decoder (threshold 0)

Held-out set	Window P/R/F1	Span IoU 0.5 P/R/F1	Coverage F1
Xenova test	0.936 / 0.872 / 0.903	0.894 / 0.833 / 0.862	0.915
ScriptSmith test	0.911 / 0.802 / 0.853	0.842 / 0.732 / 0.783	0.854

Calibrated decoder threshold 0.75. Frozen 39-video mixed pilot: temporal span IoU 0.5 P 0.941 / R 0.842 / F1 0.889; temporal coverage F1 0.905.

Intended use

On-device or local inference over English video transcripts to propose paid sponsor ranges. In Flow, SponsorBlock remains authoritative when it has segments; these weights are a fallback / shadow detector.

Limitations

—English transcripts only.
—Target is external paid sponsorships. Self-promotion, memberships, merchandise, and interaction reminders are treated as negatives.
—SponsorBlock labels are noisy and incomplete. Evaluations against them are weak labels, not ground truth.
—Full-video reconstruction interpolates timestamps inside caption cues.
—Window-level scores do not by themselves skip video; Flow stitches overlapping 768-token windows with a 128-token overlap.

ScriptSmith corpus

This model additionally used the `ScriptSmith/sponsorblock-youtube-metadata-2024` transcript/metadata dataset (CC BY 4.0 compilation; its YouTube auto-captions remain under YouTube's terms). Those transcripts are rolling auto-captions that were reconstructed before alignment, so caption boundaries are approximate. Related: `CuriousDragon/ettin-17m-sponsor-combined`

License

These weights are released under CC BY-NC-SA 4.0.

The base encoder `jhu-clsp/ettin-encoder-17m` is MIT. Fine-tuning used SponsorBlock labels (CC BY-NC-SA 4.0) and the pinned `Xenova/sponsorblock-768` window corpus. Non-commercial use and share-alike apply to this derivative.

This project is not affiliated with SponsorBlock.