sungyub/code-contests-plus-verl

1---2dataset_info:3  features:4  - name: data_source5    dtype: string6  - name: prompt7    list:8    - name: role9      dtype: string10    - name: content11      dtype: string12  - name: ability13    dtype: string14  - name: reward_model15    struct:16    - name: style17      dtype: string18    - name: ground_truth19      dtype: string20  - name: extra_info21    struct:22    - name: index23      dtype: int6424  splits:25  - name: train26    num_bytes: 1073741824027    num_examples: 786128  download_size: 1073741824029  dataset_size: 1073741824030configs:31- config_name: default32  data_files:33  - split: train34    path: data/train-*35license: cc-by-4.036task_categories:37- reinforcement-learning38- text-generation39tags:40- code41- reasoning42- rlhf43- verl44---45 46# Code Contests Plus (VERL Format)47 48This dataset contains 8,432 competitive programming problems from the Code-Contests-Plus dataset, converted to VERL format for reinforcement learning applications. Each problem includes test cases validated through sandbox execution.49 50**Source**: [ByteDance-Seed/Code-Contests-Plus](https://huggingface.co/datasets/ByteDance-Seed/Code-Contests-Plus) (1x config)51 52**License**: MIT53 54## Dataset Structure55 56The dataset follows the VERL format with the following fields:57 58- `data_source` (string): Dataset source identifier ("code-contests-plus")59- `prompt` (list): Chat template format with role/content structure containing the coding problem60- `ability` (string): Task category ("code")61- `reward_model` (dict): Evaluation information62  - `style`: Evaluation method ("rule")63  - `ground_truth`: JSON-encoded test cases with input/output pairs64- `extra_info` (dict): Additional metadata65  - `index`: Example index from original dataset66 67## Test Case Format68 69Each problem includes test cases in the `reward_model.ground_truth` field, stored as JSON with the following structure:70 71```json72{73  "inputs": ["3\n1 2 3\n"],74  "outputs": ["6\n"]75}76```77 78The format consists of two parallel arrays:79- `inputs`: Array of input strings for each test case80- `outputs`: Array of expected output strings corresponding to each input81 82Each problem typically contains between 1 and 32 test cases, validated through sandbox execution during dataset creation.83 84## Data Processing85 86The dataset was created through a multi-step processing pipeline:87 88### 1. Test Case Extraction89- Extracted public test cases from the original dataset90- Validated format and executability91- Filtered problems without valid test cases92 93### 2. Sandbox Validation94- Each problem's test cases were validated using a sandbox environment95- Test input/output pairs verified for correctness96- Only problems with passing validation were included97 98### 3. Size Filtering99- Applied 10MB size limit to test case JSON (encoded)100- Removed overly large problems to ensure efficient processing101- Balanced dataset quality and usability102 103### Processing Statistics104 105- **Total input examples**: 11,690106- **Successfully processed**: 8,432 (72.1% success rate)107- **Total filtered**: 3,258 (27.9%)108  - No test cases: 54 (0.5%)109  - Size filtered (>10MB): 3,204 (27.4%)110- **Processing time**: 69 minutes111- **Configuration used**: 1x (standard difficulty)112 113## Usage114 115```python116from datasets import load_dataset117import json118 119# Load the dataset120dataset = load_dataset("sungyub/code-contests-plus-verl")121 122# Access an example123example = dataset['train'][0]124 125# Get the problem description126problem = example['prompt'][0]['content']127print("Problem:", problem)128 129# Parse test cases130ground_truth = json.loads(example['reward_model']['ground_truth'])131inputs = ground_truth['inputs']132outputs = ground_truth['outputs']133 134print(f"\nNumber of test cases: {len(inputs)}")135print(f"First input: {repr(inputs[0])}")136print(f"Expected output: {repr(outputs[0])}")137```138 139## Example Problem140 141**Problem Description:**142```143Twins144 145square1001 and E869120 are twins, but they are not identical twins...146```147 148**Test Case:**149```python150Input:  ""151Output: "square1001"152```153 154## Statistics155 156- **Total examples**: 8,432157- **Average test cases per problem**: ~10-15158- **Test case range**: 1-32 per problem159- **Dataset size**: ~10 GB uncompressed, ~10 GB compressed (includes test cases)160- **Format**: Parquet (11 shards, ~1GB each)161- **Schema**: VERL-compatible162 163## Data Quality164 165All problems in this dataset have been validated to ensure:166 1671. **Valid test cases**: Each problem has at least one valid test case1682. **Correct input/output pairs**: Test cases verified through sandbox execution1693. **Size constraints**: Test cases are within reasonable size limits (≤10MB)1704. **Format consistency**: All examples follow the same schema structure171 172## Conversion Script173 174The dataset was created using `preprocess_codecontests_verl.py`:175 176```bash177# Standard conversion (used for this dataset)178python preprocess_codecontests_verl.py \179  --dataset-id ByteDance-Seed/Code-Contests-Plus \180  --config 1x \181  --output-dir ./codecontests_verl_full \182  --sandbox-url http://localhost:8080/run_code \183  --batch-size 100184 185# Process with different configuration186python preprocess_codecontests_verl.py \187  --dataset-id ByteDance-Seed/Code-Contests-Plus \188  --config 2x \189  --output-dir ./codecontests_verl_2x \190  --sandbox-url http://localhost:8080/run_code \191  --batch-size 100192 193# Process limited samples for testing194python preprocess_codecontests_verl.py \195  --dataset-id ByteDance-Seed/Code-Contests-Plus \196  --config 1x \197  --output-dir ./codecontests_test \198  --sandbox-url http://localhost:8080/run_code \199  --max-examples 100200```201 202## Related Datasets203 204- [Code Contests Plus (Original)](https://huggingface.co/datasets/ByteDance-Seed/Code-Contests-Plus): Original dataset with competitive programming problems205- [Skywork-OR1-Code-VERL](https://huggingface.co/datasets/sungyub/skywork-or1-code-verl): Similar VERL-format dataset with 14,057 coding problems206 207## Additional Information208 209For more information about VERL format and usage in reinforcement learning, see:210- [VERL Documentation](https://verl.readthedocs.io/en/latest/preparation/prepare_data.html)211- [VERL GitHub Repository](https://github.com/volcengine/verl)212 213## Citation214 215If you use this dataset, please cite the original Code-Contests-Plus dataset:216 217```bibtex218@misc{code-contests-plus,219  title={Code-Contests-Plus},220  author={ByteDance-Seed},221  year={2024},222  publisher={HuggingFace},223  url={https://huggingface.co/datasets/ByteDance-Seed/Code-Contests-Plus}224}225```226 227## License228 229This dataset is released under the MIT License, following the license of the original Code-Contests-Plus dataset.230