datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
pal-resultsnepalitext-language-model-dataset
Dataset Card for "nepalitext-language-model-dataset"
Dataset Summary
"NepaliText" language modeling dataset is a collection of over 13 million Nepali text sequences (phrases/sentences/paragraphs) extracted by combining the datasets: OSCAR , cc100 and a set of scraped Nepali articles on Wikipedia.
Supported Tasks and Leaderboards
This dataset is intended to pre-train language models and word representations on Nepali Language.
Languages
The data is… See the full description on the dataset page: https://huggingface.co/datasets/Sakonii/nepalitext-language-model-dataset.cis5300-language-models
CIS 5300 Language Models Dataset
Dataset for Homework 3 of CIS 5300 (Natural Language Processing) at Penn.
Cities config
Country-of-origin classification over short city-name strings, drawn from
nine countries (Afghanistan, China, Germany, Finland, France, India, Iran,
Pakistan, South Africa).
from datasets import load_dataset
cities = load_dataset("CCB/cis5300-language-models", "cities")
Split
Rows
Has labels?
train
12,392
yes
validation
1,548
yes
test
1… See the full description on the dataset page: https://huggingface.co/datasets/CCB/cis5300-language-models.core-language-model-data
Core Language Model
A small coding + general-purpose language model built from scratch in pure NumPy — no PyTorch, no JAX, no autograd library. Every piece (autograd engine, tokenizer, attention, layer norm) is hand-written and gradient-checked.
This is a from-first-principles build of a transformer language model, aimed at understanding every formula well enough to reimplement it without reference.
Why pure NumPy?
The goal isn't a competitive model — it's a fully… See the full description on the dataset page: https://huggingface.co/datasets/karthik-2905/core-language-model-data.language_model_frReAPR-Automatic-Program-Repair-via-Retrieval-Augmented-Large-Language-ModelsThis is the Retrieval dataset used in the paper "ReAPR: Automatic Program Repair via Retrieval-Augmented Large Language Models"
Anthropic_HH_Golden
Dataset Card for Anthropic_HH_Golden
This dataset is constructed to test the ULMA technique as mentioned in the paper Unified Language Model Alignment with Demonstration and Point-wise Human Preference (under review, and an arxiv link will be provided soon). They show that replacing the positive samples in a preference dataset by high-quality demonstration data (golden data) greatly improves the performance of various alignment methods (RLHF, DPO, ULMA). In particular, the ULMA… See the full description on the dataset page: https://huggingface.co/datasets/Unified-Language-Model-Alignment/Anthropic_HH_Golden.Baize-TCM-Corpus-for-Large-Language-Models-V3
白泽中医药大模型语料库
版本：3.0语料数量：157,438 条语言：中文领域：中医药（Traditional Chinese Medicine, TCM）格式：问答对（QA Pair）用途：中医药大模型训练、知识问答系统、语义理解研究
📚 简介
“白泽中医药大模型语料库”是一个专注于中医药领域的高质量问答语料集合，旨在支持中医药知识的数字化、智能化应用。语料库共包含 157,438 条 经过整理与校对的问答对，涵盖中医基础理论、中药学、方剂学、诊断学、针灸推拿、经典医籍、临床实践等多个子领域。
本语料库可广泛应用于：
中医药大语言模型的预训练与微调
智能问答系统开发
医学自然语言处理任务（如实体识别、关系抽取）
中医药知识图谱构建
🧩 数据内容
每条语料为一个标准的问答对，格式如下：
{
"instruction": "广义转录组和狭义转录组在定义上的主要区别是什么？",
"input": "",
"output":… See the full description on the dataset page: https://huggingface.co/datasets/DigitalIntelligenceCenter-of-ICMM/Baize-TCM-Corpus-for-Large-Language-Models-V3.QwQ_32B_Preview_language_model_generation_not_confirmedmultimodal-vision-language-video-models-2026
👁️ Multimodal Vision-Language & Video Foundation Models Dataset (2026 Edition)
A structured research dataset featuring 1,000 domain-verified research papers and code repositories focused on Multimodal Vision-Language Models (VLM), Video Foundation Models, Diffusion Transformers (DiT), Visual Grounding, and World Simulators.
Built with Universal Scientific Engine V15.1 Gold, providing 47 schema attributes with verified repository attribution, modality capability matrix, vision… See the full description on the dataset page: https://huggingface.co/datasets/beatsprom/multimodal-vision-language-video-models-2026.nepalitext-language-model-dataset
Dataset Card for "nepalitext-language-model-dataset"
Dataset Summary
"NepaliText" language modeling dataset is a collection of over 13 million Nepali text sequences (phrases/sentences/paragraphs) extracted by combining the datasets: OSCAR , cc100 and a set of scraped Nepali articles on Wikipedia.
Supported Tasks and Leaderboards
This dataset is intended to pre-train language models and word representations on Nepali Language.
Languages
The data is… See the full description on the dataset page: https://huggingface.co/datasets/Arpuuu/nepalitext-language-model-dataset.languagemodelsforRNNdecompositionThis repository is for the paper "Decomposing a Recurrent Neural Network into Modules for Enabling Reusability and Replacement". To use the data, there are two directories:
language datasets: Contains the necessary Tatoeba files used for the experiments. We have experimented with 4 languages(English, French, Italian and German).
language_models: Contains all trained language models and scripts to train them. It's organized in this way: language_models/{X}: contains language models for X… See the full description on the dataset page: https://huggingface.co/datasets/astha/languagemodelsforRNNdecomposition.ShareRobotrepro-how-much-can-language-models-memorize-traces
Agent traces
Agent sessions published from a Trackio Logbook.
Spatial-Blind-Spots-in-Vision-Language-Modelslicense: mit
model_evaluated:
name: Qwen3-VL-2B-Instruct
url: https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct
evaluation_notebook:
https://www.kaggle.com/code/wajidhassanmoosa/blind-spot-qwen3-2b
evaluation_setup: |
The model evaluated in this study is Qwen3-VL-2B-Instruct.
Evaluation was conducted using the Hugging Face Transformers library
with automatic device mapping (device_map="auto") and "bfloat16" dtype selection.
For each example:
The image was provided as part of a… See the full description on the dataset page: https://huggingface.co/datasets/hassan-wajid/Spatial-Blind-Spots-in-Vision-Language-Models.Instruction-Following-Evaluation-for-Large-Language-Models
Instruction-Following Evaluation Dataset
📜 Overview
This dataset, specifically designed for the evaluation of large language models in instruction-following tasks, is directly inspired by the methodologies and experiments described in the paper titled "Instruction-Following Evaluation for Large Language Models". The dataset's creation and availability on HuggingFace are aimed at enhancing research and application in the field of natural language understanding… See the full description on the dataset page: https://huggingface.co/datasets/harpreetsahota/Instruction-Following-Evaluation-for-Large-Language-Models.ilm_deTowards-Joint-Modeling-of-Dialogue-Response-and-Speech-Synthesis-based-on-Large-Language-Modellanguage_models_lab_2lener_br_finetuning_language_model
Dataset Card for "LeNER-Br language modeling"
Dataset Summary
The LeNER-Br language modeling dataset is a collection of legal texts in Portuguese from the LeNER-Br dataset (official site).
The legal texts were downloaded from this link (93.6MB) and processed to create a DatasetDict with train and validation dataset (20%).
The LeNER-Br language modeling dataset allows the finetuning of language models as BERTimbau base and large.
Language
Portuguese from… See the full description on the dataset page: https://huggingface.co/datasets/pierreguillou/lener_br_finetuning_language_model.details_mervinpraison__tamil-large-language-model-7b-v1.0
Dataset Card for Evaluation run of mervinpraison/tamil-large-language-model-7b-v1.0
Dataset automatically created during the evaluation run of model mervinpraison/tamil-large-language-model-7b-v1.0 on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_mervinpraison__tamil-large-language-model-7b-v1.0.language_modelsilm_esrepro-prototype-transformer-towards-language-model-architectures-interpretable-by-design-traces
Agent traces
Agent sessions published from a Trackio Logbook.
ilm_itaNEJM-AI_Benchmarking_Medical_Language_Models
A Comparative Study of Open-Source Large Language Models
Dataset Overview
Welcome to the dataset repository for our paper, "A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrology." The preprint of the paper can be accessed here.
Files
This repository contains two key files:
NEJM_All_Questions_And_Answers.csv: This file includes all the questions and corresponding answers used in the study.… See the full description on the dataset page: https://huggingface.co/datasets/SeanWu25/NEJM-AI_Benchmarking_Medical_Language_Models.Bangla_Masked_Language_Model_dataset_preprocessedColor-modele-languagelanguagemodelsilm_pol