asigalov61/clean-songs-lyrics-dataset
Clean Songs Lyrics Dataset 1.53M+ clean songs lyrics with songs titles and artists names Dataset info This is the combined, deduped, cleaned, and sanitized aggregation of three large lyrics datasets Each lyric was deduplicated Each lyric was checked to be in range of 256 bytes <-> 8192 bytes Each lyric was checked for profanities with alt-profanity-check Each lyric was ASCII… See the full description on the dataset page: https://huggingface.co/datasets/asigalov61/clean-songs-lyrics-dataset.
2119
Clean Songs Lyrics Dataset
1.53M+ clean songs lyrics with songs titles and artists names
Dataset info
This is the combined, deduped, cleaned, and sanitized aggregation of three large lyrics datasets
Each lyric was deduplicated
Each lyric was checked to be in range of 256 bytes <-> 8192 bytes
Each lyric was checked for profanities with alt-profanity-check
Each lyric was ASCII sanitized for conistency
Fast dataset loading
from datasets import load_dataset
import tqdm
def to_row_dicts_in_batches(ds, batch_size=20000):
"""
Convert a Dataset into a list of dicts (one per example), reading in batches.
"""
rows = []
cols = ds.column_names
for batch in tqdm.tqdm(ds.iter(batch_size=batch_size)):
# batch[col] is a list of values for that column
# zip them together to get individual examples
for record in zip(*(batch[col] for col in cols)):
rows.append(dict(zip(cols, record)))
return rows
ds = load_dataset("asigalov61/clean-songs-lyrics-dataset", split="train")
clean_lyrics_dataset = to_row_dicts_in_batches(ds)
print(len(clean_lyrics_dataset))
print(clean_lyrics_dataset[0])