TechxGenus/LeetCode-Contest

1---2license: other3license_name: deepseek4license_link: >-5  https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct/blob/main/LICENSE6task_categories:7- text-generation8language:9- en10tags:11- code12---13 14## LeetCode Contest Benchmark15 16A new benchmark for evaluating Code LLMs proposed by [DeepSeek-Coder](https://arxiv.org/abs/2401.14196), which consists of the latest algorithm problems of different difficulties.17 18## Usage19 20```21git clone https://github.com/deepseek-ai/DeepSeek-Coder.git22cd Evaluation/LeetCode23 24# Set the model or path here25MODEL="deepseek-ai/deepseek-coder-7b-instruct"26 27python vllm_inference.py --model_name_or_path $MODEL --saved_path output/20240121-Jul.deepseek-coder-7b-instruct.jsonl28 29python evaluate_leetcode.py --generation_path output/20240121-Jul.deepseek-coder-7b-instruct.jsonl --result_path output/20240121-Jul.deepseek-coder-7b-instruct.result.jsonl30```31 32### Citation33 34```35@article{guo2024deepseekcoder,36  title   = {DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence},37  author  = {Daya Guo and Qihao Zhu and Dejian Yang and Zhenda Xie and Kai Dong and Wentao Zhang and Guanting Chen and Xiao Bi and Y. Wu and Y. K. Li and Fuli Luo and Yingfei Xiong and Wenfeng Liang},38  year    = {2024},39  journal = {arXiv preprint arXiv: 2401.14196}40}41```42