ManBib/DUE-v2-quality-filtered
removed markdown formatting, links, emojis, multiple white spaces replaced user and channel tags with @user and #channel deduplicated (not fuzzy only exact) basic filtering: word count filter(min_words=3, max_words=200) mean word length filter(min_mean_word_length=2.0, max_mean_word_length=12.0) long word filter(max_word_length=200) whitespace filter (max_white_space_ratio=0.25)
Create README.md
Upload data/train-00055-of-00056.parquet with huggingface_hub
Upload data/train-00054-of-00056.parquet with huggingface_hub
Upload data/train-00053-of-00056.parquet with huggingface_hub
Upload data/train-00052-of-00056.parquet with huggingface_hub
Upload data/train-00051-of-00056.parquet with huggingface_hub
Upload data/train-00050-of-00056.parquet with huggingface_hub
Upload data/train-00049-of-00056.parquet with huggingface_hub
Upload data/train-00048-of-00056.parquet with huggingface_hub
Upload data/train-00047-of-00056.parquet with huggingface_hub
Upload data/train-00046-of-00056.parquet with huggingface_hub
Upload data/train-00045-of-00056.parquet with huggingface_hub
Upload data/train-00044-of-00056.parquet with huggingface_hub
Upload data/train-00043-of-00056.parquet with huggingface_hub
Upload data/train-00042-of-00056.parquet with huggingface_hub
Upload data/train-00041-of-00056.parquet with huggingface_hub
Upload data/train-00040-of-00056.parquet with huggingface_hub
Upload data/train-00039-of-00056.parquet with huggingface_hub
Upload data/train-00038-of-00056.parquet with huggingface_hub
Upload data/train-00037-of-00056.parquet with huggingface_hub
Upload data/train-00036-of-00056.parquet with huggingface_hub
Upload data/train-00035-of-00056.parquet with huggingface_hub
Upload data/train-00034-of-00056.parquet with huggingface_hub
Upload data/train-00033-of-00056.parquet with huggingface_hub
Upload data/train-00032-of-00056.parquet with huggingface_hub
Upload data/train-00031-of-00056.parquet with huggingface_hub
Upload data/train-00030-of-00056.parquet with huggingface_hub
Upload data/train-00029-of-00056.parquet with huggingface_hub
Upload data/train-00028-of-00056.parquet with huggingface_hub
Upload data/train-00027-of-00056.parquet with huggingface_hub
Upload data/train-00026-of-00056.parquet with huggingface_hub
Upload data/train-00025-of-00056.parquet with huggingface_hub
Upload data/train-00024-of-00056.parquet with huggingface_hub
Upload data/train-00023-of-00056.parquet with huggingface_hub
Upload data/train-00022-of-00056.parquet with huggingface_hub
Upload data/train-00021-of-00056.parquet with huggingface_hub
Upload data/train-00020-of-00056.parquet with huggingface_hub
Upload data/train-00019-of-00056.parquet with huggingface_hub
Upload data/train-00018-of-00056.parquet with huggingface_hub
Upload data/train-00017-of-00056.parquet with huggingface_hub
Upload data/train-00016-of-00056.parquet with huggingface_hub
Upload data/train-00015-of-00056.parquet with huggingface_hub
Upload data/train-00014-of-00056.parquet with huggingface_hub
Upload data/train-00013-of-00056.parquet with huggingface_hub
Upload data/train-00012-of-00056.parquet with huggingface_hub
Upload data/train-00011-of-00056.parquet with huggingface_hub
Upload data/train-00010-of-00056.parquet with huggingface_hub
Upload data/train-00009-of-00056.parquet with huggingface_hub
Upload data/train-00008-of-00056.parquet with huggingface_hub
Upload data/train-00007-of-00056.parquet with huggingface_hub