algerian-nlp/Algerian-Youtube-Comments

Algerian Youtube Comments

55,365 raw YouTube comments on Algeria-related videos for Darija social-text modeling, from the Algerian NLP Collective. Counted 2026-09-17 via the Hub datasets-server (/info?dataset=algerian-nlp/Algerian-Youtube-Comments: 55,365 train rows) and re-counted row-by-row with datasets streaming (load_dataset("algerian-nlp/Algerian-Youtube-Comments", split="train", streaming=True): 55,365 rows).

The default config answers: how do Algerians actually write in YouTube comments — dialect, French, Arabizi, and noise included.

from datasets import load_dataset

ds = load_dataset("algerian-nlp/Algerian-Youtube-Comments", "default", split="train")

default — raw comments in per-epoch parquet files

55,365 rows, all in train, across 885 per-epoch files (data/train_epoch_000001.parquet through data/train_epoch_001535.parquet, non-contiguous numbering, file list from the Hub API 2026-09-17). No dev or test split ships with this repo.

The gap this config fixes is stated, not measured by us: raw unfiltered social text as a pretraining-style resource. 55,347 of 55,365 comments (99.97%, counted by streaming filter len(comment_text) > 100) exceed 100 characters — this is long-comment data, not short replies.

Key on (comment_id): ids are platform-assigned and globally unique. The text is RAW: no normalisation, no PII stripping, and no language filtering were applied — row 0 is plain French with zero Darja (Hada youcef avec sa chaine est parmi les premiers youtubeurs algériens MAIS AUSSI parmi les plus banales...). What breaks if ignored: any dialect classifier trained here without filtering learns French-first; any PII audit assumption fails — phone numbers and handles are in scope of the raw text.

Scoring protocol

Language-modeling perplexity on a split you define, with tokenizer and seed beside the number. Unmeasured: no baseline scores have been produced by us on this protocol.

Results

Unmeasured. No baseline results scored by us in the protocol above.

Usage

default: raw Algerian social text for modeling.

from datasets import load_dataset

ds = load_dataset("algerian-nlp/Algerian-Youtube-Comments", "default", split="train")
ds[0]
# comment_id: UgyHm5TNV3CCXwBVflZ4AaABAg
# comment_text: Hada youcef avec sa chaine est parmi les premiers youtubeurs ...

# Long comments (measured 2026-09-17, streaming, 55,365 rows scanned):
subset = ds.filter(lambda row: len(row["comment_text"]) > 100)
len(subset)  # 55347

What this dataset does NOT contain — and what breaks if you assume it does. This repo previously shipped with NO card at all. There is no video id column (join video context via the YouTube API on comment_id), no timestamp, no like count, no language tag, and no per-row source id, licence, or retrieval date. Do not assume Darja: French-only rows are common by direct observation.

Script & code-switching distribution

Unmeasured. No per-script counts were published and none were counted by us. French-only, Arabic-script, and mixed rows all occur by direct observation of streamed rows.

Provenance and limits

• —The default config is scraped YouTube comments from 2,649 videos (build ledger processed_tracker.json in this repo: video_ids count 2,649, epoch_counter 1536). Video list, scrape date, and scraper are otherwise undocumented.
• —Per-row provenance is missing: rows carry no source id beyond the platform comment_id, no licence, and no retrieval date.
• —Ledger gap. The tracker lists 55,378 comment ids but only 55,365 rows load (13 fewer) — some tracked comments never became rows, cause undocumented.
• —Filtration audit. None exists: this is raw text by inspection. Spam, bots, ads, and duplicate comments are all in scope until someone measures them.
• —Bounded, not cleared. Personal data, hate speech, and copyrighted text cannot be excluded: web text is used as published with no documented PII sweep. Research-use limits apply — see the organisation card.
• —Out of scope. Representativeness of Algerian YouTube as a whole was not established; video selection bias decides what this set is, and the selection is unpublished.
• —Decontamination of any training corpus against this dataset is unmeasured.

Citation

@misc{algerian_nlp_algerian_youtube_comments,
  title  = {Algerian Youtube Comments: 55,365 raw Algeria-related comments},
  author = {Algerian NLP Collective},
  year   = {2026},
  url    = {https://huggingface.co/datasets/algerian-nlp/Algerian-Youtube-Comments}
}

No DOI, arXiv ID, or named author is published on the repo. Upstream: the comment authors and video creators on YouTube — their rights are unaffected by this card.

Licence

Unknown. No licence tag or licence statement ships with this repo, so no grant can be quoted here. Scraped platform text carries the platform's terms plus the commenters' rights — do not redistribute derivatives until the owner declares a licence; this card sets unknown rather than inventing one.