datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
code_search_net
Dataset Card for CodeSearchNet corpus
Dataset Summary
CodeSearchNet corpus is a dataset of 2 milllion (comment, code) pairs from opensource libraries hosted on GitHub. It contains code and documentation for several programming languages.
CodeSearchNet corpus was gathered to support the CodeSearchNet challenge, to explore the problem of code retrieval using natural language.
Supported Tasks and Leaderboards
language-modeling: The dataset can be used to… See the full description on the dataset page: https://huggingface.co/datasets/code-search-net/code_search_net.huggingface-spaces-codes
📊 Dataset Description
This dataset comprises code files of Huggingface Spaces that have more than 0 likes as of November 10, 2023. This dataset contains various programming languages totaling in 672 MB of compressed and 2.05 GB of uncompressed data.
📝 Data Fields
Field
Type
Description
repository
string
Huggingface Spaces repository names.
sdk
string
Software Development Kit of the space.
license
string
License type of the space.… See the full description on the dataset page: https://huggingface.co/datasets/Weyaxi/huggingface-spaces-codes.python-codes-25k
License
MIT
This is a Cleaned Python Dataset Covering 25,000 Instructional Tasks
Overview
The dataset has 4 key features (fields): instruction, input, output, and text.It's a rich source for Python codes, tasks, and extends into behavioral aspects.
Dataset Statistics
Total Entries: 24,813
Unique Instructions: 24,580
Unique Inputs: 3,666
Unique Outputs: 24,581
Unique Texts: 24,813
Average Tokens per example: 508
Features… See the full description on the dataset page: https://huggingface.co/datasets/flytech/python-codes-25k.NCERT-Parallel-Dataset-Indiccode-search-net-python
Dataset Card for "code-search-net-python"
Dataset Description
Homepage: None
Repository: https://huggingface.co/datasets/Nan-Do/code-search-net-python
Paper: None
Leaderboard: None
Point of Contact: @Nan-Do
Dataset Summary
This dataset is the Python portion of the CodeSarchNet annotated with a summary column.The code-search-net dataset includes open source functions that include comments found at GitHub.The summary is a short description of what the… See the full description on the dataset page: https://huggingface.co/datasets/Nan-Do/code-search-net-python.Java-GitHub-CodesCodeSearchNetEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/CodeSearchNet.llama-codes-swe-bench-evalsFinance-Conversational-Dataset-Indicsms-spam-collection
SMS Spam Collection v.1
DESCRIPTION
The SMS Spam Collection v.1 (hereafter the corpus) is a set of SMS tagged messages that have been collected for SMS Spam research. It contains one set of SMS messages in English of 5,574 messages, tagged acording being ham (legitimate) or spam.
1.1. Compilation
This corpus has been collected from free or free for research sources at the Web:
A collection of between 425 SMS spam messages extracted manually from the Grumbletext Web… See the full description on the dataset page: https://huggingface.co/datasets/codesignal/sms-spam-collection.CodeSearchNet-ccrEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/CodeSearchNet-ccr.CodeSearchNetRetrieval
CodeSearchNetRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
The dataset is a collection of code snippets and their corresponding natural language queries. The task is to retrieve the most relevant code snippet for a given query.
Task category
t2t
Domains
Programming, Written
Reference
https://huggingface.co/datasets/code_search_net/
Source datasets:
code-search-net/code_search_net
How to evaluate on this task
You can evaluate an embedding… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CodeSearchNetRetrieval.NCERT-Conversational-Dataset-Indictiny-codes
Reasoning with Language and Code
This synthetic dataset is a collection of 1.6 millions short and clear code snippets that can help LLM models learn how to reason with both natural and programming languages. The dataset covers a wide range of programming languages, such as Python, TypeScript, JavaScript, Ruby, Julia, Rust, C++, Bash, Java, C#, and Go. It also includes two database languages: Cypher (for graph databases) and SQL (for relational databases) in order to study the… See the full description on the dataset page: https://huggingface.co/datasets/nampdn-ai/tiny-codes.tsla-historic-pricesLaw-Conversational-Dataset-Indicsih26099-cpse-material-codes
SIH 26099 — Collected Dataset
AI-Driven Standardization & Harmonization of Material Codes Across CPSEs
This workspace holds the data-collection stage only — no model, no training,
no feature engineering. Just raw public sources, their extracted structured
form, and the reference taxonomies/vocabularies the harmonisation step needs.
Collected live on 2026-09-08. All row counts below were verified by reading
the files back with pandas.
1. Headline numbers… See the full description on the dataset page: https://huggingface.co/datasets/sarthak20024/sih26099-cpse-material-codes.Cyber-Conversational-Dataset-IndicLLM-CodeGen
LLMs-Generated Code
This dataset the raw code generated by 10 different Large Language Models (LLMs) in response to the prompts from our research project. The code is organized to facilitate the assessment and comparison of each model's ability to generate secure C/C++ code.
The generated code is divided into two main categories:
Simple Assistant: Code generated by the LLM with no specific security-focused instructions.
Secure Assistant: Code generated by the LLM using prompts that… See the full description on the dataset page: https://huggingface.co/datasets/codesbyusman/LLM-CodeGen.Cyber-Parallel-Dataset-IndicTextInsightBench
TextInsightBench
English | 简体中文
A natural-language data-mining benchmark for agents: 50 tasks, 435,000 task documents and 944,468 unlabeled learning documents.
Each task provides 5,000 or 10,000 texts and a research objective. Agents choose the patterns, populations and comparisons to investigate, then submit up to three findings with complete document assignments, exact quotations, statistics, counterexamples and limitations. Any analysis method is allowed.
Contents… See the full description on the dataset page: https://huggingface.co/datasets/CodeSoulco/TextInsightBench.CodeSearchNet-ccrFinance-Parallel-Dataset-IndicComputer-Science-Conversational-Dataset-Indicstatic-analysis-evalA dataset of 76 Python programs taken from real Python open source projects (top 100 on GitHub),
where each program is a file that has exactly 1 vulnerability as detected by a particular static analyzer (Semgrep), used in the paper Patched MOA: optimizing inference for diverse software development tasks.
OpenAI used the synth-vuln-fixes and fine-tuned
a new version of gpt-4o is now the SOTA on this benchmark. More details and code is available from their repo.
More details on the benchmark… See the full description on the dataset page: https://huggingface.co/datasets/patched-codes/static-analysis-eval.CA-Conversational-Dataset-IndicLaw-Parallel-Dataset-Indiccode_search_net_clean
Dataset Card for "code_search_net_clean"
More Information needed
code_search_net
CodeSearchNet
This is an unofficial reupload of the code_search_net dataset in the parquet format. I have also removed the columns func_code_tokens, func_documentation_tokens, and split_name as they are not relevant. The original repository relies on a Python module that is downloaded and executed to unpack the dataset, which is a potential security risk but importantly raises an annoying warning. As a plus, parquets load faster.
Original model card:
Dataset Card for… See the full description on the dataset page: https://huggingface.co/datasets/claudios/code_search_net.QA-Codes-perfect