datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
codefeedback-stEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-st.codefeedback-mtEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-mt.codefeedback-st-queries-corpusEmploying the CoIR evaluation framework's dataset version, utilize the code below for assessment:
import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel
model_name = "intfloat/e5-base-v2"
# Load the model
model = YourCustomDEModel(model_name=model_name)
# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-st-queries-corpus.codefeedback-st-qrels
Dataset Card for "codefeedback-st-qrels"
More Information needed
Code-Feedback
Dataset Card for CodeFeedback
This is a formatted version of m-a-p/Code-Feedback to store the conversations in the same format as the OpenAI SDK.
SWE-bench__style-3__fs-oracle_large_tokenlength
Dataset Card for "SWE-bench__style-3__fs-oracle"
More Information needed
SWE-bench__style-3__fs-oracle
Dataset Card for "SWE-bench__style-3__fs-oracle"
More Information needed
CodeFeedback-Filtered-Instruction-PythonCodeFeedback-Python105K
Note
This subset is derived from the m-a-p/CodeFeedback-Filtered-Instruction dataset, which contains 156,526 samples.
CodeFeedback-Filtered-Instruction is a curated collection of code instruction queries extracted from four prominent open-source code instruction tuning datasets: Magicoder-OSS-Instruct, Python code subset of ShareGPT, Magicoder-Evol-Instruct, and Evol-Instruct-Code.
From this dataset, we specifically selected only the 104,848 samples that are written in Python.… See the full description on the dataset page: https://huggingface.co/datasets/fxmeng/CodeFeedback-Python105K.CodeFeedbackMT
CodeFeedbackMT
An MTEB dataset
Massive Text Embedding Benchmark
The dataset is a collection of user queries and assistant responses. The task is to retrieve the most relevant response for a given query.
Task category
t2t
Domains
Programming, Written
Reference
https://arxiv.org/abs/2402.14658
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["CodeFeedbackMT"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CodeFeedbackMT.g1_top8_no_swebench_code_feedback_glm47_traces
DCAgent2/g1_top8_no_swebench_code_feedback_glm47_traces
89,094 rows. Aggressively SWE-bench-decontaminated variant of DCAgent/g1_min_episodes_top8_100k_glm47_traces: the entire issue subset (13,783 rows, of which 7,824 were SWE-bench-derived) is dropped and partially replaced with code_feedback traces filtered to ≥5 episodes.
What was changed
Original top8_100k
This dataset
Total rows
100,110
89,094
Issue subset rows
13,783 (with 7,824 from SWE-bench: 2,442… See the full description on the dataset page: https://huggingface.co/datasets/DCAgent2/g1_top8_no_swebench_code_feedback_glm47_traces.codefeedback-mtcodefeedback-mt-queries-corpusEmploying the CoIR evaluation framework's dataset version, utilize the code below for assessment:
import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel
model_name = "intfloat/e5-base-v2"
# Load the model
model = YourCustomDEModel(model_name=model_name)
# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-mt-queries-corpus.codefeedback-mt-qrels
Dataset Card for "codefeedback-mt-qrels"
More Information needed
Code-Feedback-CleanCodeFeedbackST
CodeFeedbackST
An MTEB dataset
Massive Text Embedding Benchmark
The dataset is a collection of user queries and assistant responses. The task is to retrieve the most relevant response for a given query.
Task category
t2t
Domains
Programming, Written
Reference
https://arxiv.org/abs/2407.02883
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_task("CodeFeedbackST")… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CodeFeedbackST.code-feedback-deepseekv2-criticcode-feedback-10k-deepseekv2-critica1_code_codefeedbackseed_code_codefeedback_exploitneulab-code-feedback-sandboxes-traces-terminus-2DCAgent2_terminal_bench_2_DCAgent2_neulab-code-feedback-sandboxes-traces-termineeb38555CodeFeedback-Filtered-Instruction-ko
Dataset Card for "CodeFeedback-Filtered-Instruction-ko"
Translated m-a-p/CodeFeedback-Filtered-Instruction using nayohan/llama3-instrucTrans-enko-8b.
codefeedback-single-turn-reformat-magicodercodefeedback-scoredm-a-p-codefeedback-mistral-largecodefeedback-stneulab-code-feedback-sandboxesru_codefeedbackmozzarella
Mozzarella-0.3.1
Motivation
Mozzarella is a dataset matching issues (= problem statements) and corresponding pull requests (PRs = problem solutions) of a selection of well maintained Java GitHub repositories. The original purpose was to serve as training and evaluation data for ML models concerned with fault localization and automated program repair of complex code bases. However, there might be more use cases that could benefit from this data.
Inspired by SWEBench… See the full description on the dataset page: https://huggingface.co/datasets/feedback-to-code/mozzarella.