Datasetpublic

GPUMODE/categorized_triton_data_permissive

Dataset Description This dataset contains code snippets from Triton-based projects across GitHub, specifically filtered to include only repositories with permissive licenses (MIT, Apache, BSD, etc.). Each entry in the dataset includes: Triton code snippet Repository information File path Commit hash Direct GitHub URL to the source code License information Categorization of the code functionality Dataset Creation The dataset was created by: Collecting Triton… See the full description on the dataset page: https://huggingface.co/datasets/GPUMODE/categorized_triton_data_permissive.

sourceHugging Facemitupdated 2y agoView on Hugging Face

6likes54downloads

Dataset Card

Dataset Description

This dataset contains code snippets from Triton-based projects across GitHub, specifically filtered to include only repositories with permissive licenses (MIT, Apache, BSD, etc.). Each entry in the dataset includes:

—Triton code snippet
—Repository information
—File path
—Commit hash
—Direct GitHub URL to the source code
—License information
—Categorization of the code functionality

Dataset Creation

The dataset was created by:

1.Collecting Triton code snippets from public GitHub repositories
2.Categorizing the code snippets based on functionality (Using claude)
3.Filtering to keep only snippets from repositories with permissive licenses using a custom should_keep_license function

License Information

This dataset is released under the MIT License. However, each code snippet in the dataset comes from a repository with its own specific license (all permissive). The license type for each snippet is included in the dataset.

Permissive licenses included in this dataset:

—MIT
—BSD
—APACHE
—CC0

Format and Usage

The dataset is provided in two formats:

—JSON format (permissive_triton_dataset.json)
—Parquet format (permissive_triton_dataset.parquet)

Sample Data Structure

json

{
  "uuid": "...",
  "file_name": "example_triton_file.py",
  "repo_name": "username/repo",
  "file_path": "path/to/file.py",
  "commit_hash": "abcdef123456",
  "starcount": 42,
  "input": "@triton.jit\ndef example_kernel(...):\n    ...",
  "category": {
    "Functionality": ["Category1", "Category2"]
  },
  "licenses": ["MIT"],
  "github_url": "https://github.com/username/repo/blob/abcdef123456/path/to/file.py"
}

Field Descriptions

Field	Description
`uuid`	Unique identifier for the entry in the dataset
`file_name`	Name of the source code file
`repo_name`	GitHub repository name in format "username/repo"
`file_path`	Path to the file within the repository
`commit_hash`	Git commit hash for the specific version of the file
`starcount`	Number of stars the repository had at the time of data collection
`input`	The actual Triton code snippet
`category`	Categorization of the code functionality (labeled using Claude)
`licenses`	List of permissive license types applicable to this code
`github_url`	Direct URL to view the file on GitHub at the specific commit

Category Types

We consider categories in the following domains: Functionality, Data Type, Performance Objective, Parallelization Strategy, and Memory Access Pattern. We optinally add labels to each of these domains per entry to try and describe the data (using claude).

Loading the Dataset

python

# Using JSON
import json
with open('permissive_triton_dataset.json', 'r') as f:
    dataset = json.load(f)

# Using Parquet
import pandas as pd
df = pd.read_parquet('permissive_triton_dataset.parquet')