CoIR-Retrieval/codetrans-dl

1---2dataset_info:3- config_name: corpus4  features:5  - name: _id6    dtype: string7  - name: partition8    dtype: string9  - name: text10    dtype: string11  - name: language12    dtype: string13  - name: meta_information14    struct:15    - name: framework16      dtype: string17  - name: title18    dtype: string19  splits:20  - name: corpus21    num_bytes: 124385222    num_examples: 81623  download_size: 13533224  dataset_size: 124385225- config_name: default26  features:27  - name: query-id28    dtype: string29  - name: corpus-id30    dtype: string31  - name: score32    dtype: int6433  splits:34  - name: train35    num_bytes: 1275636    num_examples: 56437  - name: test38    num_bytes: 414039    num_examples: 18040  - name: valid41    num_bytes: 165642    num_examples: 7243  download_size: 1362544  dataset_size: 1855245- config_name: queries46  features:47  - name: _id48    dtype: string49  - name: partition50    dtype: string51  - name: text52    dtype: string53  - name: language54    dtype: string55  - name: meta_information56    struct:57    - name: framework58      dtype: string59  - name: title60    dtype: string61  splits:62  - name: queries63    num_bytes: 133193264    num_examples: 81665  download_size: 13241966  dataset_size: 133193267configs:68- config_name: corpus69  data_files:70  - split: corpus71    path: corpus/corpus-*72- config_name: default73  data_files:74  - split: train75    path: data/train-*76  - split: test77    path: data/test-*78  - split: valid79    path: data/valid-*80- config_name: queries81  data_files:82  - split: queries83    path: queries/queries-*84---85Employing the MTEB evaluation framework's dataset version, utilize the code below for assessment:86 87```python88import mteb89import logging90from sentence_transformers import SentenceTransformer91from mteb import MTEB92 93logger = logging.getLogger(__name__)94 95model_name = 'intfloat/e5-base-v2'96model = SentenceTransformer(model_name)97tasks = mteb.get_tasks(98    tasks=[99        "AppsRetrieval",100        "CodeFeedbackMT",101        "CodeFeedbackST",102        "CodeTransOceanContest",103        "CodeTransOceanDL",104        "CosQA",105        "SyntheticText2SQL",106        "StackOverflowQA",107        "COIRCodeSearchNetRetrieval",108        "CodeSearchNetCCRetrieval",109    ]110)111evaluation = MTEB(tasks=tasks)112results = evaluation.run(113    model=model,114    overwrite_results=True115)116print(result)117```