latam-gpt/red_pajama_es_hq

RedPajama's High Quality Spanish subset What is this? The following is a high-quality dataset distilled from the Spanish subsection of RedPajama-Data-v2, created using the methodology proposed in FineWEB-Edu. Usage from datasets import load_dataset ds = load_dataset("latam-gpt/red_pajama_es_hq") Filtering by quality score Documents in this corpus are scored on academic quality from 2.5 to 5, with higher scores indicating better… See the full description on the dataset page: https://huggingface.co/datasets/latam-gpt/red_pajama_es_hq.