hybridfree/github-code-2025

🚀 GitHub Code 2025: The Clean Code Manifesto

A meticulously curated dataset of 1.5M+ repositories representing both quality and innovation in 2025's code ecosystem

🌟 The Philosophy

Quality Over Quantity, Purpose Over Volume

In an era of data abundance, we present a dataset built on radical curation. Every file, every repository, every byte has been carefully selected to represent the signal in the noise of open-source development.

🎯 What This Dataset Is

📊 Dual-Perspective Design

🧹 The Clean Code Promise

# What you WON'T find here:
🚫 Binary files          # No images, executables, models
🚫 Build artifacts       # No node_modules, __pycache__
🚫 Configuration noise   # No .git, IDE files, lock files
🚫 License duplication   # No repetitive legal text
🚫 Minified code         # No compressed/obfuscated content
🚫 Empty files           # No whitespace-only content

📁 Dataset Structure

github-code-2025/
├── 📈 above-2-stars/
│   ├── train_000.parquet
│   ├── train_001.parquet
│   └── ...
└── 🌱 below-2-star/
    ├── train_000.parquet
    ├── train_001.parquet
    └── ...

📊 Schema

{
    "repo_id": "owner/repo_name",    # 📍 Repository identifier
    "file_path": "src/main.py",      # 🗂️ Relative file path
    "content": "def clean_code():",   # 💎 Actual source code
    "size": 1024                     # 📏 File size in bytes
}

🛠️ How to Use

🔥 Quick Start

from datasets import load_dataset

# Load the quality benchmark
quality_ds = load_dataset("nick007x/github-code-2025", "above-2-stars")

# Load emerging trends
emerging_ds = load_dataset("nick007x/github-code-2025", "below-2-star")

# Mix for balanced training
balanced_ds = interleave_datasets([quality_ds, emerging_ds])

🎯 Ideal Use Cases

• —🧠 AI Training: Clean, diverse code for language models
• —📊 Code Analysis: Compare popular vs emerging patterns
• —🔍 Trend Research: 2025 development practices
• —🎓 Education: High-quality examples for learning
• —🛠️ Tool Development: Benchmarking code quality tools

🏗️ Creation Methodology

🎨 Selection Strategy

🚫 What We Filtered Out

File Types Removed:

• —50+ binary extensions (images, models, executables)
• —30+ build/system directories
• —15+ configuration file types
• —All files outside 1KB-5MB range

Quality Checks:

• —✅ UTF-8 text validation
• —✅ Non-empty content check
• —✅ Binary detection
• —✅ Repository structure preservation

🎪 Why This Dataset Matters

💫 The Quality Revolution

We reject the "more data is better" dogma. Instead, we offer:

• —🎯 Intentional Curation: Every file serves a purpose
• —⚖️ Balanced Perspective: Popular + Emerging = Complete picture
• —🧹 Unprecedented Cleanliness: The cleanest code dataset available
• —📅 Temporal Intelligence: 2025-focused for relevance

🤝 Contributing & Feedback

This dataset is a living project. We welcome:

• —🐛 Bug reports and issues
• —💡 Feature requests for future versions
• —📊 Validation of data quality
• —🎯 Suggestions for improvement

📜 License

This dataset aggregates Github repos. Each individual repo maintains its original copyright and license terms (typically various Creative Commons licenses like CC BY, CC BY-NC, etc.). Users must verify and comply with the specific license of any repo they extract and use from this collection. The MIT license in this repository applies only to the dataset compilation and packaging code.

Important: Repository contents maintain their original licenses. Please respect individual project licenses when using this data.

🙏 Acknowledgments

Built with gratitude for the entire open-source community. Every file in this dataset represents hours of dedication from developers worldwide.

⭐ If this dataset helps your research or project, please consider starring the repository!

"In the pursuit of AI that understands code, we must first understand what code is worth learning."