CoolFace
Datasetpublic

ManBib/DUE-v2-quality-filtered

removed markdown formatting, links, emojis, multiple white spaces replaced user and channel tags with @user and #channel deduplicated (not fuzzy only exact) basic filtering: word count filter(min_words=3, max_words=200) mean word length filter(min_mean_word_length=2.0, max_mean_word_length=12.0) long word filter(max_word_length=200) whitespace filter (max_white_space_ratio=0.25)

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes271downloads

ManBib/DUE-v2-quality-filtered · main · files are served by the source, never re-hosted here