datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
meta-llama_Llama-3.1-8B-Instruct-jdgfct-ReadabilitySlimPajama-Meta-rater-Readability-30B
Top 30B token SlimPajama Subset selected by the Readability rater
This repository contains the dataset described in the paper Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models.
Code: https://github.com/opendatalab/Meta-rater
Dataset Description
This dataset contains the top 30B tokens from the SlimPajama-627B corpus, selected using the Readability dimension of the PRRC (Professionalism, Readability, Reasoning, Cleanliness) framework.… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/SlimPajama-Meta-rater-Readability-30B.nvidia_NVLM-D-72B-jdgfct-Readabilitymeta-llama_Llama-3.1-70B-Instruct-jdgfct-Readabilityreadability-es-hackathon-pln-public
Dataset Card for [readability-es-sentences]
Dataset Description
Compilation of short Spanish articles for readability assessment.
Dataset Summary
This dataset is a compilation of short articles from websites dedicated to learn Spanish as a second language. These articles have been compiled from the following sources:
Coh-Metrix-Esp corpus (Quispesaravia, et al., 2016): collection of 100 parallel texts with simple and complex variants in Spanish. These texts… See the full description on the dataset page: https://huggingface.co/datasets/somosnlp-hackathon-2022/readability-es-hackathon-pln-public.low-readability-text
Low Readability Text Dataset
This dataset consists of high-complexity English web text with an estimated readability at or above the U.S. Grade 12 level. The content typically features advanced, highly technical prose or verbose syntactical structures, making it well-suited for researching complex language understanding and automation.
Primary Use Cases
Text Simplification: Training and evaluating models to translate complex text into plain English.
Information… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/low-readability-text.CommonLit-Ease-of-Readabilitycode-readability-krod
Java Code Readability Merged & Modified
This dataset contains 69276 Java code snippets along with a readability score, mined from Github and automatically processed and labelled.
You can download the dataset using Hugging Face:
from datasets import load_dataset
ds = load_dataset("se2p/code-readability-krod")
The snippets are not split into train and test (and validation) set. Thus, the whole dataset is in the train set:
ds = ds['train']
ds_as_list = ds.to_list() # Convert the… See the full description on the dataset page: https://huggingface.co/datasets/se2p/code-readability-krod.readability-es-caes
Dataset Card for [readability-es-caes]
Dataset Description
Dataset Summary
This dataset is a compilation of short articles from websites dedicated to learn Spanish as a second language. These articles have been compiled from the following sources:
CAES corpus (Martínez et al., 2019): the "Corpus de Aprendices del Español" is a collection of texts produced by Spanish L2 learners from Spanish learning centers and universities. These text are produced by students… See the full description on the dataset page: https://huggingface.co/datasets/somosnlp-hackathon-2022/readability-es-caes.code-readability-merged
Java Code Readability Merged Dataset
This dataset contains 421 Java code snippets along with a readability score, aggregated from several scientific papers [1, 2, 3].
You can download the dataset using Hugging Face:
from datasets import load_dataset
ds = load_dataset("se2p/code-readability-merged")
The snippets are not split into train and test (and validation) set. Thus, the whole dataset is in the train set:
ds = ds['train']
ds_as_list = ds.to_list() # Convert the dataset to… See the full description on the dataset page: https://huggingface.co/datasets/se2p/code-readability-merged.dpo-q2572b-a70b-jllm3-Readability-Aproduction-caption-readability-fixtures
SHAR Production Caption Readability Fixtures
Explicitly synthetic subtitle-cue fixtures for deterministic character-per-second validation. The paired MIT tool is production-caption-readability-check.
SHAR Production is an AI-hybrid video production studio: https://sharprod.com/
Each row has an expected outcome at max_cps: 20. These fixtures contain no client, production, or personal data.
cebuano-readabilitySource: https://github.com/imperialite/cebuano-readability
We asked permission from one of the authors to include this dataset to our catalog effort. We copy a portion of the README in this dataset card.
Baseline Readability Assessment Model for Cebuano
This repository contains the code and datasets from Bloom, Let's Read Asia, and Department of Education (DepEd) websites used for developing the first ML-based baseline for readability assessment in the Cebuano language described… See the full description on the dataset page: https://huggingface.co/datasets/filbench/cebuano-readability.dpo-Llama31-70b-NVLM-72b-Llama3-Readabilitydpo-qwen2572b-athene70b-jdg-Llama3-ReadabilityreadabilityDescription: This dataset comprises approximately 200,000 paragraphs and readability metrics from each of four sources:
HuggingFace's Fineweb-Edu
Ronen Eldan's TinyStories
Wikipedia-2023-11-embed-multilingual-v3 (English only)
ArXiv Abstracts-2021.
Each paragraph falls within the character range of 50 to 2000.
Format: JSON, with each row representing a paragraph and containing both the text and its corresponding readability grade.
Features:
Text: A paragraph of text from one of the… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/readability.osman_readability
Dataset Card for [Dataset Name]
Dataset Summary
[More Information Needed]
Supported Tasks and Leaderboards
[More Information Needed]
Languages
[More Information Needed]
Dataset Structure
Data Instances
[More Information Needed]
Data Fields
[More Information Needed]
Data Splits
[More Information Needed]
Dataset Creation
Curation Rationale
[More Information Needed]
Source Data… See the full description on the dataset page: https://huggingface.co/datasets/arbml/osman_readability.Qwen_Qwen2-7B-Instruct-jdgfct-ReadabilityFinRAD_Financial_Readability_Assessment_Dataset
FinRAD: Financial Readability Assessment Dataset - 13,000+ Definitions of Financial Terms for Measuring Readability
This repository contains the dataset mentioned in the paper: FinRAD: Financial Readability Assessment Dataset - 13,000+ Definitions of Financial Terms for Measuring Readability (presented at The Financial Narrative Processing Workshop colocated with LREC-2022, Marseille, France).
In addition to this, data collection & cleaning scripts, embedding extraction & model… See the full description on the dataset page: https://huggingface.co/datasets/sohomghosh/FinRAD_Financial_Readability_Assessment_Dataset.oncology-readability-collapse-risk-v0.3
What this dataset does
This dataset tests whether a model can detect pre-cancer instability risk from loss of signal readability rather than from stress burden alone.
The task is not cancer diagnosis.
The task is to classify whether a synthetic tissue ecology has entered readability collapse risk.
Core Stability Idea
The dataset represents a stability-transition hypothesis.
Cancer vulnerability may begin when tissue regulation loses the ability to correctly read… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/oncology-readability-collapse-risk-v0.3.Persian-Readability-Dataset
Dataset Summary
This is a re-upload of the Persian Text Readability Dataset, originally created and published by Mohammadi & Khasteh (2020). It provides sentence-level readability annotations for Persian (Farsi) texts. Each data point includes:
A text in Persian
A label (readability level):
0 for easy
1 for medium
2 for hard
A rater profile: the average readability label distribution of the raters who annotated that specific text
All texts included have over 80% agreement… See the full description on the dataset page: https://huggingface.co/datasets/AlirezaF138/Persian-Readability-Dataset.dpo-nemotron70b-nvlm72b-jdg-Llama3-Readabilitydpo-qwen2572b-llama3170b-jdg-Llama3-Readabilityreadability_ori
Dataset Card for readability_ori
This dataset has been created with Argilla. As shown in the sections below, this dataset can be loaded into your Argilla server as explained in Load with Argilla, or used directly with the datasets library in Load with datasets.
Using this dataset with Argilla
To load with Argilla, you'll just need to install Argilla as pip install argilla --upgrade and then use the following code:
import argilla as rg
ds =… See the full description on the dataset page: https://huggingface.co/datasets/annachaaang/readability_ori.Dorn_Code_Readability
Software Readability Dataset
This repository contains the dataset used to build and evaluate the readability model presented in:
A General Software Readability Model
Jonathan Dorn & Westley Weimer, University of Virginia
The dataset consists of human-annotated code snippets sampled from real open-source projects and labeled for perceived readability. It is the largest such dataset collected for software readability research to date.
📦 Dataset Summary… See the full description on the dataset page: https://huggingface.co/datasets/rufimelo/Dorn_Code_Readability.advanced-readability-analysis
Advanced Readability Analysis
This dataset provides rich syntactic and lexical complexity features calculated from English text snippets. It is designed to help researchers study the underlying factors that influence reading difficulty, especially in cases where traditional readability formulas yield conflicting results.
The source text is pulled from the training split of the agentlans/readability dataset.
The linguistic annotations and complexity metrics were computed using a… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/advanced-readability-analysis.aozorabunko_readability_scorereadability_wp