Lazyhope/python-clone-detection

1---2license: mit3---4# Python clone detection5 6This is a codebert model for detecting Python clone codes, fine-tuned on the dataset shared by [PoolC](https://github.com/PoolC) on [Hugging Face Hub](https://huggingface.co/datasets/PoolC/1-fold-clone-detection-600k-5fold). The original source code for using the model can be found at https://github.com/sangHa0411/CloneDetection/blob/main/inference.py.7 8# How to use9 10To use the model in an efficient way, you can refer to this repository: https://github.com/RepoAnalysis/PythonCloneDetection, which contains a class that integrates data preprocessing, input tokenization, and model inferencing.11 12You can also follow the original inference source code at https://github.com/sangHa0411/CloneDetection/blob/main/inference.py.13 14More conveniently, a pipeline for this model has been implemented, and you can initialize it with only two lines of code:15```python16from transformers import pipeline17 18pipe = pipeline(model="Lazyhope/python-clone-detection", trust_remote_code=True)19```20To use it, pass a tuple of code pairs:21```python22code1 = """def token_to_inputs(feature):23    inputs = {}24    for k, v in feature.items():25        inputs[k] = torch.tensor(v).unsqueeze(0)26 27    return inputs"""28code2 = """def f(feature):29    return {k: torch.tensor(v).unsqueeze(0) for k, v in feature.items()}"""30 31is_clone = pipe((code1, code2))32is_clone33# {False: 1.3705984201806132e-05, True: 0.9999862909317017}34```35 36# Credits37 38We would like to thank the original team and authors of the model and the fine-tuning dataset:39- [PoolC](https://github.com/PoolC)40- [sangHa0411](https://github.com/sangHa0411)41- [snoop2head](https://github.com/snoop2head)42 43# Lincese44 45This model is released under the MIT license.46