LyraNovaHeart/TheRoyalCarpet-Deduped-Compressed

The Royal Carpet: Cleaned Original Dataset: KaraKaraWitch/the-royal-carpet Thanks KaraKaraWitch! Why? The original dataset is 40GB in size (at least in decimal), which is quite the large set! 40GB would likely overwhelm most systems since I doubt most people have more than 32GB of RAM. Not to mention too, this dataset contained a redundant row, html, which is just a exact copy of the text row with HTML crap, removing that row brought down most of the dataset to… See the full description on the dataset page: https://huggingface.co/datasets/LyraNovaHeart/TheRoyalCarpet-Deduped-Compressed.