datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
lilm1-pretrain-mix-32b
LiLM Experiment 3 pretraining corpus
Private research corpus with 32,000,010,072 globally
exact-deduplicated train tokens plus 328,933,246 held-out
tokens. Data are stored as EOS-delimited little-endian uint16 binaries with
aligned Parquet provenance.
This repository combines ODC-By FinePDFs-Edu, CC-BY-4.0 DCLM, ODC-By
SmolLM/FineMath sources, Apache-2.0 UltraX subject to its upstream terms,
per-file permissively licensed Stack-Edu code subject to The Stack v2 terms,
StarCoder2… See the full description on the dataset page: https://huggingface.co/datasets/glouriousgautam/lilm1-pretrain-mix-32b.lucky-initialization-atlas-100m-v2
Lucky initialization atlas v2 evidence
Private live evidence archive for lilywchen/lucky-initialization-atlas-100m-v2. It contains hash-bound configs,
provenance, scalar trajectories, step-zero diagnostics, and final per-sequence losses
after those artifacts complete. It excludes
credentials, caches, raw FineWeb-derived token arrays, optimizer states, and W&B
binary logs.
nemotron_cc_v2_hq_packed4096_200shard
Nemotron-CC-v2 High-Quality, packed to 4096 tokens (train)
Documents from nvidia/Nemotron-CC-v2
High-Quality subset, tokenized with google/t5gemma-2-270m-270m (add_special_tokens=False, EOS
appended per document) and greedily packed into sequences of at most 4096 tokens.
A document is never split across a pack boundary; documents longer than 4096
are truncated to their own pack. Every pack ends on an EOS/document boundary.
Schema
index (int64): running pack id… See the full description on the dataset page: https://huggingface.co/datasets/lillian039/nemotron_cc_v2_hq_packed4096_200shard.nemotron_cc_v2_hq_packed4096
Nemotron-CC-v2 High-Quality, packed to 4096 tokens
5% subset of nvidia/Nemotron-CC-v2
High-Quality documents, tokenized with google/t5gemma-2-270m-270m (add_special_tokens=False, EOS
appended per document) and greedily packed into sequences of at most 4096 tokens.
A document is never split across a pack boundary; documents longer than 4096
are truncated to their own pack. Every pack ends on an EOS/document boundary.
Schema
index (int64): running pack id
input_ids… See the full description on the dataset page: https://huggingface.co/datasets/lillian039/nemotron_cc_v2_hq_packed4096.respect
Retrospective Learning from Interactions (Respect) Dataset
This repository contains the lil-lab/respect data, based on the ACL paper Retrospective Learning from Interactions. For more resources, please see https://lil-lab.github.io/respect and https://github.com/lil-lab/respect.
Sample Usage
You can load the data and associated checkpoints as follows:
from datasets import load_dataset
from transformers import Idefics2ForConditionalGeneration
from peft importPeftModel… See the full description on the dataset page: https://huggingface.co/datasets/lil-lab/respect.medical_texts_simplification
Dataset Card for Medical texts simplification
The dataset consisting of 30 triples (around 800 sentences) of the original text,
human- and ChatGPT-simplified texts was created from a subset Medical Notes Classification dataset.
The original dataset contains medical notes, which come from exactly one of the following five clinical domains:
Gastroenterology, Neurology, Orthopedics, Radiology, and Urology. There are 1239 texts in total in the original dataset.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/liliya-makhmutova/medical_texts_simplification.lilm1-tool-teacher-corpora
LiLM1 tool teacher corpora
This dataset contains synthetic tool-use records generated with Gemma and Qwen
teacher models.
Method
Each teacher received structured tool schemas and task templates. One
configuration preserves the records from each teacher and task set.
Configurations
Configuration
Content
gemma-26b-a4b-function
Gemma function-calling records
qwen-27b-function
Qwen function-calling records
qwen-35b-a3b-function
Qwen MoE… See the full description on the dataset page: https://huggingface.co/datasets/glouriousgautam/lilm1-tool-teacher-corpora.lilm1-230m-posttraining
LiLM1-230M post-training data
This dataset contains the selected post-training data for LiLM1-230M.
Method
The records combine general assistant text with structured tool-use examples.
The configurations preserve the binding stage, the ratio study, and the
selected 4:8 continuation.
Configurations
Configuration
Content
binding-repair
Tool binding data
ratio-study
Three training splits used for ratio selection
ratio-evaluation
Shared… See the full description on the dataset page: https://huggingface.co/datasets/glouriousgautam/lilm1-230m-posttraining.HealthBench-ProX
HealthBench-ProX
Dataset Description
HealthBench-ProX is a multilingual extension of the original HealthBench Professional benchmark for evaluating large language models on realistic healthcare consultation scenarios.
The dataset contains 6,825 evaluation instances organized into 13 language-specific splits:
de (German)
en (English)
fr (French)
hi (Hindi)
ig (Igbo)
ja (Japanese)
ko (Korean)
ms (Malay)
pt (Portuguese)
sw (Swahili)
th (Thai)
zh (Chinese)
zu (Zulu)… See the full description on the dataset page: https://huggingface.co/datasets/li-lab/HealthBench-ProX.nemotron_cc_v2_hq_packed4096_200shard_valid
Nemotron-CC-v2 High-Quality, packed to 4096 tokens (validation)
Documents from nvidia/Nemotron-CC-v2
High-Quality subset, tokenized with google/t5gemma-2-270m-270m (add_special_tokens=False, EOS
appended per document) and greedily packed into sequences of at most 4096 tokens.
A document is never split across a pack boundary; documents longer than 4096
are truncated to their own pack. Every pack ends on an EOS/document boundary.
Schema
index (int64): running pack… See the full description on the dataset page: https://huggingface.co/datasets/lillian039/nemotron_cc_v2_hq_packed4096_200shard_valid.uigen-ui-code-gen
UIGEN UI/UX Code Generation Dataset
This dataset contains UI/UX code generation examples formatted for training code generation models.
Each example consists of a task description and the corresponding HTML/CSS code implementation using Tailwind CSS.
Dataset Structure
The dataset has a single text column containing formatted prompts and completions:
# Task: Generate HTML/CSS code using Tailwind CSS
# Requirements: [specific requirements]
[HTML/CSS code implementation]… See the full description on the dataset page: https://huggingface.co/datasets/lilyzhng/uigen-ui-code-gen.uigen-ui-code-gen-full
UIGEN UI/UX Code Generation Dataset
This dataset contains UI/UX code generation examples formatted for training code generation models.
Each example consists of a task description and the corresponding HTML/CSS code implementation using Tailwind CSS.
Dataset Structure
The dataset has a single text column containing formatted prompts and completions:
# Task: Generate HTML/CSS code using Tailwind CSS
# Requirements: [specific requirements]
[HTML/CSS code implementation]… See the full description on the dataset page: https://huggingface.co/datasets/lilyzhng/uigen-ui-code-gen-full.nepal-law-commission-nepali
⚖️ Nepal Law Commission — Nepali Legal Corpus
Dataset Summary
A cleaned Nepali-language text corpus extracted from official annual reports published by the Nepal Law Commission (lawcommission.gov.np). The corpus spans fiscal years 2067/68 – 2081/82 (approximately 2010–2025), covering legal research, legislative drafting, law reform activities, and policy recommendations.
Each row is a self-contained chunk of Nepali text (~300–1200 characters), filtered from mixed… See the full description on the dataset page: https://huggingface.co/datasets/lilgoose777/nepal-law-commission-nepali.nepali-bible-corpus
🇳🇵 Nepali Bible Corpus (NPIONCB)
A cleaned and structured Nepali-language Bible dataset derived from the New Pioneers International Old and New Covenant Bible (NPIONCB) translation. This corpus is intended for use in pretraining foundational Nepali language models and other NLP research on low-resource Devanagari-script languages.
Dataset Summary
The Nepali Bible is a rich, formally structured source of written Nepali in Devanagari script. This dataset contains… See the full description on the dataset page: https://huggingface.co/datasets/lilgoose777/nepali-bible-corpus.OLA
OLA: Output Language Alignment Benchmark
OLA is a benchmark designed to evaluate LLMs' Output Language Alignment in code-switched interactions
Dataset Structure
OLA consists of two settings: Simple and Complex.
Simple Setting
The Simple setting focuses on intra-sentential code-switching, where the expected response language is the matrix language—the language providing the core grammatical structure into which elements from another language are embedded.… See the full description on the dataset page: https://huggingface.co/datasets/lilolyhh/OLA.satai
Dataset Card for Dataset Name
This dataset card aims to be a base template for new datasets. It has been generated using this raw template.
Dataset Details
Dataset Description
Curated by: [More Information Needed]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Dataset Sources [optional]
Repository: [More… See the full description on the dataset page: https://huggingface.co/datasets/lilsomnus/satai.mof-nepal-nepali
💰 Ministry of Finance Nepal — Nepali Government Finance Corpus
Dataset Summary
A cleaned Nepali-language text corpus extracted from official Ministry of Finance (MoF), Nepal ministry-wise progress reports published on mof.gov.np. The corpus spans fiscal years 2072/73 – 2080/81 (approximately 2015–2024), covering budget implementation, ministry-level expenditure, program progress, and financial reporting across all government ministries of Nepal.
Each row is a… See the full description on the dataset page: https://huggingface.co/datasets/lilgoose777/mof-nepal-nepali.lilanlanHundredCV-Chat
百人对话数据集
HundredCV-Chat: A Dataset of Daily Chatting Developed on HundredCVs
简介
本项目提出一个全新的中文多轮对话数据集（HundredCV-Chat），该数据集由 100 位青年的简历数据集 HundredCVs 开发而来，共包含 24,750 组日常闲聊对话数据。
数据集具有如下特点：
自动化标注：HundredCV-Chat 中的对话均由 Deepseek-V3 大模型生成，不涉及任何人工标注，因此同时保证了大规模数据量和低成本优势。
多样性话题：HundredCV-Chat 中的对话话题涵盖了校园生活、工作经验、兴趣爱好、生活琐事等多个方面，与真实生活联系紧密，尤其适用于开发年轻化应用。
高质量对话：利用 Deepseek 强大的生成能力和全面的知识，HundredCV-Chat 的对话内容在流畅度、拟人性、多样性方面均显著优于现有的开源对话数据集。
数据样例
HundredCV-Chat 含有 24… See the full description on the dataset page: https://huggingface.co/datasets/lilianggui/HundredCV-Chat.moha-nepal-nepali
🇳🇵 MoHA Nepal — Nepali Government Corpus
Dataset Summary
A cleaned Nepali-language text corpus extracted from official PDF documents published by the Ministry of Home Affairs (MoHA), Nepal (moha.gov.np). The corpus covers annual progress reports and quarterly disclosures spanning fiscal years 2076/77 – 2082/83 (approx. 2019–2026).
Each row is a self-contained chunk of Nepali text (~300–1200 characters), cleaned of OCR artifacts and annotated with rich metadata including… See the full description on the dataset page: https://huggingface.co/datasets/lilgoose777/moha-nepal-nepali.lilchatbot-corpus
LilChatBot Corpus
Training corpus for LilChatBot.
Composition
Wikipedia prose (50%)
News articles (Reuters / AP style) (30%)
Literary prose (late 19th / early 20th century) (20%)
Approximate size:
~438MB
~50M tokens (WordLevel tokenizer, 50k vocab)
Formatting
Paragraph-separated text
<bos> and <eos> tokens included
Cleaned for repeated punctuation
ASCII normalized apostrophes
Intended Use
Pretraining small GPT-style models (≈4-6 layers, 128… See the full description on the dataset page: https://huggingface.co/datasets/divilian/lilchatbot-corpus.