datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
CodeFeedback-Filtered-Instruction OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
[🏠Homepage]
|
[🛠️Code]
OpenCodeInterpreter
OpenCodeInterpreter is a family of open-source code generation systems designed to bridge the gap between large language models and advanced proprietary systems like the GPT-4 Code Interpreter. It significantly advances code generation capabilities by integrating execution and iterative refinement functionalities.
For further information and… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction.Code-Feedback OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
[🏠Homepage]
|
[🛠️Code]
Introduction
OpenCodeInterpreter is a family of open-source code generation systems designed to bridge the gap between large language models and advanced proprietary systems like the GPT-4 Code Interpreter. It significantly advances code generation capabilities by integrating execution and iterative refinement functionalities.
For further information and related… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/Code-Feedback.codefeedback-stEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-st.codefeedback-mtEmploying the MTEB evaluation framework's dataset version, utilize the code below for assessment:
import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB
logger = logging.getLogger(__name__)
model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
tasks=[
"AppsRetrieval",
"CodeFeedbackMT",
"CodeFeedbackST",
"CodeTransOceanContest",
"CodeTransOceanDL"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-mt.codefeedback-st-queries-corpusEmploying the CoIR evaluation framework's dataset version, utilize the code below for assessment:
import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel
model_name = "intfloat/e5-base-v2"
# Load the model
model = YourCustomDEModel(model_name=model_name)
# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-st-queries-corpus.codefeedback-st-qrels
Dataset Card for "codefeedback-st-qrels"
More Information needed
Code-Feedback
Dataset Card for CodeFeedback
This is a formatted version of m-a-p/Code-Feedback to store the conversations in the same format as the OpenAI SDK.
SWE-bench__style-3__fs-oracle_large_tokenlength
Dataset Card for "SWE-bench__style-3__fs-oracle"
More Information needed
SWE-bench__style-3__fs-oracle
Dataset Card for "SWE-bench__style-3__fs-oracle"
More Information needed
Code-feedback-sharegpt-renamedCodeFeedback-Filtered-Instruction-PythonCodeFeedback-Python105K
Note
This subset is derived from the m-a-p/CodeFeedback-Filtered-Instruction dataset, which contains 156,526 samples.
CodeFeedback-Filtered-Instruction is a curated collection of code instruction queries extracted from four prominent open-source code instruction tuning datasets: Magicoder-OSS-Instruct, Python code subset of ShareGPT, Magicoder-Evol-Instruct, and Evol-Instruct-Code.
From this dataset, we specifically selected only the 104,848 samples that are written in Python.… See the full description on the dataset page: https://huggingface.co/datasets/fxmeng/CodeFeedback-Python105K.CodeFeedbackMT
CodeFeedbackMT
An MTEB dataset
Massive Text Embedding Benchmark
The dataset is a collection of user queries and assistant responses. The task is to retrieve the most relevant response for a given query.
Task category
t2t
Domains
Programming, Written
Reference
https://arxiv.org/abs/2402.14658
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["CodeFeedbackMT"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CodeFeedbackMT.g1_top8_no_swebench_code_feedback_glm47_traces
DCAgent2/g1_top8_no_swebench_code_feedback_glm47_traces
89,094 rows. Aggressively SWE-bench-decontaminated variant of DCAgent/g1_min_episodes_top8_100k_glm47_traces: the entire issue subset (13,783 rows, of which 7,824 were SWE-bench-derived) is dropped and partially replaced with code_feedback traces filtered to ≥5 episodes.
What was changed
Original top8_100k
This dataset
Total rows
100,110
89,094
Issue subset rows
13,783 (with 7,824 from SWE-bench: 2,442… See the full description on the dataset page: https://huggingface.co/datasets/DCAgent2/g1_top8_no_swebench_code_feedback_glm47_traces.codefeedback-mtcodefeedback-mt-queries-corpusEmploying the CoIR evaluation framework's dataset version, utilize the code below for assessment:
import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel
model_name = "intfloat/e5-base-v2"
# Load the model
model = YourCustomDEModel(model_name=model_name)
# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/codefeedback-mt-queries-corpus.codefeedback-mt-qrels
Dataset Card for "codefeedback-mt-qrels"
More Information needed
Code-Feedback-CleanCodeFeedbackST
CodeFeedbackST
An MTEB dataset
Massive Text Embedding Benchmark
The dataset is a collection of user queries and assistant responses. The task is to retrieve the most relevant response for a given query.
Task category
t2t
Domains
Programming, Written
Reference
https://arxiv.org/abs/2407.02883
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_task("CodeFeedbackST")… See the full description on the dataset page: https://huggingface.co/datasets/mteb/CodeFeedbackST.code-feedback-deepseekv2-criticCode-Feedback-decontaminationA decontaminated version of m-a-p/Code-Feedback.
The excluded (28) files are "contaminated" with only two code segments:
simple GCD function: while b: a, b = b, a % b return a
sum_to_n solution: return sum(range(n + 1))
And reformated to sharegpt.
Decontamination is done in the same way as Magicoder (ie., bigcode decontamination process), which uses a substring-match-finding method to find overlaps between a target dataset and the following standard benchmarks:
HumanEval
MBPP… See the full description on the dataset page: https://huggingface.co/datasets/Leon-Leee/Code-Feedback-decontamination.code-feedback-10k-deepseekv2-critica1_code_codefeedbackseed_code_codefeedback_exploitCodeFeedback-Alpacaneulab-code-feedback-sandboxes-traces-terminus-2DCAgent2_terminal_bench_2_DCAgent2_neulab-code-feedback-sandboxes-traces-termineeb38555CodeFeedback-Filtered-Instruction-ko
Dataset Card for "CodeFeedback-Filtered-Instruction-ko"
Translated m-a-p/CodeFeedback-Filtered-Instruction using nayohan/llama3-instrucTrans-enko-8b.
codefeedback-single-turn-reformat-magicodercodefeedback-scored