thealper2/graphcodebert-code-clone-detection

1---2license: mit3library_name: transformers4pipeline_tag: text-classification5tags:6  - code7  - clone-detection8  - graphcodebert9  - code-similarity10base_model: microsoft/graphcodebert-base11datasets:12  - PoolC/1-fold-clone-detection-600k-5fold13language:14  - code15metrics:16  - accuracy17  - precision18  - recall19  - f120model-index:21  - name: graphcodebert-code-clone-detection22    results:23      - task:24          type: text-classification25          name: Binary code clone detection26        dataset:27          type: PoolC/1-fold-clone-detection-600k-5fold28          name: PoolC/1-fold-clone-detection-600k-5fold29          split: test (group-disjoint half of the `val` fold)30        metrics:31          - type: f132            value: 0.880533          - type: accuracy34            value: 0.874735          - type: precision36            value: 0.84137          - type: recall38            value: 0.92439---40 41# graphcodebert-code-clone-detection42 43Binary code-clone detection. Full fine-tune of44[`microsoft/graphcodebert-base`](https://huggingface.co/microsoft/graphcodebert-base) on45[`PoolC/1-fold-clone-detection-600k-5fold`](https://huggingface.co/datasets/PoolC/1-fold-clone-detection-600k-5fold),46using GraphCodeBERT's data-flow-aware pairwise architecture.47 48Output labels: `0 = not clone`, `1 = clone`.49 50## Architecture51 52Not a generic sequence-pair classifier. The two snippets are encoded53**separately** by one shared GraphCodeBERT encoder, each with its own54graph-guided masked attention, and the two `<s>` vectors are concatenated for55classification:56 57```58Linear(2 x 768 -> 768) -> tanh -> Linear(768 -> 2)59```60 61Per-snippet input layout (length 640):62 63| segment | length | content | `position_idx` |64|---|---|---|---|65| code tokens | 512 | `<s>` + BPE code tokens + `</s>` | `2 .. n+1` |66| data-flow nodes | 128 | one slot per DFG variable node (`<unk>` id) | `0` |67| padding | remainder | `<pad>` | `1` |68 69A data-flow node's input embedding is the **average of the embeddings of the70code tokens it was identified from**. Graph-guided attention allows: code to71code; `<s>`/`</s>` to everything; node to the code tokens it comes from (and72back); node to adjacent nodes.73 74## Preprocessing75 76The dataset contains **Python** snippets, so data flow is extracted with the77`tree-sitter-python` grammar via a port of GraphCodeBERT's `DFG_python`78extractor (comment/docstring stripping -> AST -> variable states ->79`comesFrom` / `computedFrom` edges).80 81| | |82|---|---|83| `code_length` | 512 |84| `data_flow_length` | 128 |85| total sequence length | 640 |86| distinct snippets featurised | 44,950 |87| mean data-flow nodes / snippet | 44.22 |88| snippets with empty data flow | 263 |89| total data-flow edges | 2,481,388 |90| extraction status counts | `{"ok": 44930, "comment_strip_failed": 13, "dfg_failed": 7}` |91 92No example was dropped: a snippet whose data flow could not be extracted is93kept with an empty graph and counted above.94 95## Data splits96 97The repository provides one of 5 predefined folds as `train` + `val`; those groups are disjoint and are kept as-is. `val` is partitioned further into validation/test along problem-group boundaries.98 99`similar` equals `(code1_group == code2_group)` for every row, so the group100columns are a perfect label proxy and are never used as features.101 102| split | source | pairs | positives | negatives | groups |103|---|---|---:|---:|---:|---:|104| train | `train` fold | 50,000 | 25,000 | 25,000 | 240 |105| validation | half of `val` by group | 20,000 | 10,000 | 10,000 | 29 |106| test | other half of `val` by group | 20,000 | 10,000 | 10,000 | 30 |107 108Train/validation/test share **no problem group and no code snippet**; this is109asserted at runtime before training starts. 337,398 pairs of the110held-out fold were dropped because their two snippets fell on opposite sides of111the validation/test group boundary.112 113Class weighting: Measured majority-class share 0.5000 is within the 0.6 threshold, so weighted cross entropy is NOT used.114 115## Training116 117| | |118|---|---|119| optimizer | adamw_torch |120| learning rate | 2e-05 |121| scheduler | linear with 0.1 warmup ratio (938 steps) |122| epochs | 3.0 |123| per-device batch size | 16 |124| gradient accumulation | 1 |125| effective batch size | 16 |126| weight decay | 0.01 |127| gradient clipping | 1.0 |128| mixed precision | fp16 |129| gradient checkpointing | False |130| seed | 42 |131| trainable parameters | 125,236,994 |132| training time | 1.923 h |133| GPU | NVIDIA GeForce RTX 5060 Ti (15.9 GB) |134| torch / transformers | 2.11.0+cu128 / 5.17.0 |135 136Checkpoint selection: best validation **F1** (`load_best_model_at_end=True`,137`metric_for_best_model="f1"`). Best validation F1 = **0.8672**.138The test split was scored once, after selection.139 140## Results141 142| split | accuracy | precision | recall | F1 | TP | TN | FP | FN |143|---|---:|---:|---:|---:|---:|---:|---:|---:|144| validation | 0.8557 | 0.8032 | 0.9422 | 0.8672 | 9,422 | 7,692 | 2,308 | 578 |145| test | 0.8747 | 0.8410 | 0.9240 | 0.8805 | 9,240 | 8,253 | 1,747 | 760 |146 147Test confusion matrix (`[[TN, FP], [FN, TP]]`): `[[8253, 1747], [760, 9240]]`148 149## Usage150 151This checkpoint uses a **custom pairwise head and a graph-guided attention152mask**, so `AutoModelForSequenceClassification` will not reproduce these153results. Use the repository's own model class and preprocessing:154 155```python156import torch157from transformers import AutoTokenizer158from modeling import load_model                      # from this project159from preprocess import build_snippet_features, CloneCollator160from config import Config161 162cfg = Config()163tokenizer = AutoTokenizer.from_pretrained("thealper2/graphcodebert-code-clone-detection")164model = load_model("thealper2/graphcodebert-code-clone-detection").eval()165 166features = build_snippet_features(cfg, [code_a, code_b], tokenizer, num_proc=1)167collator = CloneCollator(features)168batch = collator([(0, 1, 0)])                        # (snippet_a, snippet_b, dummy label)169with torch.no_grad():170    logits = model(**{k: v for k, v in batch.items() if k != "labels"}).logits171label = int(logits.argmax(-1))                       # 0 = not clone, 1 = clone172```173 174## Limitations175 176- Trained on competitive-programming Python solutions grouped by problem;177  "clone" therefore means *solves the same problem*, which is closer to178  semantic (Type-4) similarity than to syntactic copy-paste detection.179- Data flow is extracted with the Python grammar only. Other languages need the180  matching `tree_sitter_<lang>` grammar and `DFG_<lang>` function.181- Snippets longer than 512 BPE tokens are truncated; 885 of182  44,950 distinct snippets hit that limit.183- Both directions of a pair are not explicitly symmetrised; the head sees184  `concat(<s>_1, <s>_2)` in the given order.185