archit11/verl-code-corpus-track-a-file-split

1---2language:3- en4license: apache-2.05task_categories:6- text-generation7- fill-mask8tags:9- code10- python11- verl12- repo-specific-finetuning13pretty_name: Verl Code Corpus (File Holdout Split)14size_categories:15- n<1K16---17 18# archit11/verl-code-corpus-track-a-file-split19 20Repository-specific code corpus extracted from the `verl` project and split by file for training/evaluation.21 22## What is in this dataset23 24- Source corpus: `data/code_corpus_verl`25- Total files: 21426- Train files: 17227- Validation files: 2128- Test files: 2129- File type filter: .py30- Split mode: `file` (file-level holdout)31 32Each row has:33 34- `file_name`: flattened source file name35- `text`: full file contents36 37## Training context38 39This dataset was used for extended pretraining of:40 41- Model repo: `https://huggingface.co/archit11/qwen2.5-coder-3b-verl-track-a-lora`42- Base model: `/root/.cache/huggingface/hub/models--Qwen--Qwen2.5-Coder-3B/snapshots/09d9bc5d376b0cfa0100a0694ea7de7232525803`43- Sequence curriculum: [768, 1024]44- Learning rate: 0.000145- Batch size: 846 47Evaluation from this run:48 49- Baseline perplexity (val/test): 3.1820 / 2.776450- Post-training perplexity (val/test): 2.7844 / 2.237951 52## Load with datasets53 54```python55from datasets import load_dataset56 57ds = load_dataset("archit11/verl-code-corpus-track-a-file-split")58print(ds)59print(ds["train"][0]["file_name"])60```61