NamCyan/codebert-base-technical-debt-code-tesoro

1---2library_name: transformers3datasets:4- NamCyan/tesoro-code5base_model:6- microsoft/codebert-base7---8 9# Improving the detection of technical debt in Java source code with an enriched dataset10 11 12## Model Details13 14### Model Description15 16This model is the part of Tesoro project, used for detecting technical debt in source code. More information can be found at [Tesoro HomePage](https://github.com/NamCyan/tesoro.git).17 18- **Developed by:** [Nam Hai Le](https://github.com/NamCyan)19- **Model type:** Encoder-based PLMs20- **Language(s):** Java21- **Finetuned from model:** [CodeBERT](https://huggingface.co/microsoft/codebert-base)22 23### Model Sources24 25- **Repository:** [Tesoro](https://github.com/NamCyan/tesoro.git)26- **Paper:** [To be update]27 28## How to Get Started with the Model29 30Use the code below to get started with the model.31 32```python33from transformers import AutoModelForSequenceClassification, AutoTokenizer34 35tokenizer = AutoTokenizer.from_pretrained("NamCyan/codebert-base-technical-debt-code-tesoro")36model = AutoModelForSequenceClassification.from_pretrained("NamCyan/codebert-base-technical-debt-code-tesoro")37```38 39 40## Training Details41 42- Training Data: The model is finetuned using [tesoro-code](https://huggingface.co/datasets/NamCyan/tesoro-code)43 44- Infrastructure: Training process is conducted on two NVIDIA A100 GPUs with 80GB of VRAM.45 46## Leaderboard47| Model | Model size  | EM            | F1              |48|:-------------|:-----------|:------------------|:------------------|49| **Encoder-based PLMs** |50| [CodeBERT](https://huggingface.co/microsoft/codebert-base)     | 125M       | 38.28             | 43.47             |51| [UniXCoder](https://huggingface.co/microsoft/unixcoder-base)    | 125M       | 38.12             | 42.58             |52| [GraphCodeBERT](https://huggingface.co/microsoft/graphcodebert-base)| 125M       | *39.38*          | *44.21*           |53| [RoBERTa](https://huggingface.co/FacebookAI/roberta-base)      | 125M       | 35.37             | 38.22             |54| [ALBERT](https://huggingface.co/albert/albert-base-v2)       | 11.8M      | 39.32             | 41.99             |55| **Encoder-Decoder-based PLMs** |56| [PLBART](https://huggingface.co/uclanlp/plbart-base)       | 140M       | 36.85             | 39.90             |57| [Codet5](https://huggingface.co/Salesforce/codet5-base)       | 220M       | 32.66             | 35.41             |58| [CodeT5+](https://huggingface.co/Salesforce/codet5p-220m)      | 220M       | 37.91             | 41.96             |59| **Decoder-based PLMs (LLMs)** |60| [TinyLlama](https://huggingface.co/TinyLlama/TinyLlama_v1.1_math_code)    | 1.03B      | 37.05             | 40.05             |61| [DeepSeek-Coder](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base) | 1.28B    | **42.52**         | **46.19**         |62| [OpenCodeInterpreter](https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-1.3B)       | 1.35B             | 38.16             | 41.76             |63| [phi-2](https://huggingface.co/microsoft/phi-2)        | 2.78B      | 37.92             | 41.57             |64| [starcoder2](https://huggingface.co/bigcode/starcoder2-3b)   | 3.03B      | 35.37             | 41.77             |65| [CodeLlama](https://huggingface.co/codellama/CodeLlama-7b-hf)    | 6.74B      | 34.14             | 38.16             |66| [Magicoder](https://huggingface.co/ise-uiuc/Magicoder-S-DS-6.7B)    | 6.74B      | 39.14             | 42.49             |67 68 69## Citing us70```bibtex71@article{nam2024tesoro,72  title={Improving the detection of technical debt in Java source code with an enriched dataset},73  author={Hai, Nam Le and Bui, Anh M. T. Bui and Nguyen, Phuong T. and Ruscio, Davide Di and Kazman, Rick},74  journal={},75  year={2024}76}77```