Datasetpublic

CoIR-Retrieval/CodeSearchNet-ccr-ruby-qrels

Employing the CoIR evaluation framework's dataset version, utilize the code below for assessment: import coir from coir.data_loader import get_tasks from coir.evaluation import COIR from coir.models import YourCustomDEModel model_name = "intfloat/e5-base-v2" # Load the model model = YourCustomDEModel(model_name=model_name) # Get tasks #all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic- # text2sql","cosqa","codesearchnet"… See the full description on the dataset page: https://huggingface.co/datasets/CoIR-Retrieval/CodeSearchNet-ccr-ruby-qrels.

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes50downloads

Dataset Card

Employing the CoIR evaluation framework's dataset version, utilize the code below for assessment:

python

import coir
from coir.data_loader import get_tasks
from coir.evaluation import COIR
from coir.models import YourCustomDEModel

model_name = "intfloat/e5-base-v2"

# Load the model
model = YourCustomDEModel(model_name=model_name)

# Get tasks
#all task ["codetrans-dl","stackoverflow-qa","apps","codefeedback-mt","codefeedback-st","codetrans-contest","synthetic-
# text2sql","cosqa","codesearchnet","codesearchnet-ccr"]
tasks = get_tasks(tasks=["codetrans-dl"])

# Initialize evaluation
evaluation = COIR(tasks=tasks,batch_size=128)

# Run evaluation
results = evaluation.run(model, output_folder=f"results/{model_name}")
print(results)