Datasetpublic

Andwwy/rules

rules Natural-language LLM-agent rule files (AGENTS.md, CLAUDE.md, SKILL.md, .cursor/rules/*.mdc, and friends) crawled from public GitHub repositories, with the content stored inline. 2,204,470 files from 45,014 repositories. Each row is one rule file, pinned to the commit SHA it was read at, so link always resolves to the exact bytes in file. Columns column type description file string full text of the rule file content_sha256 string SHA-256 of file… See the full description on the dataset page: https://huggingface.co/datasets/Andwwy/rules.

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes77downloads

Dataset Card

rules

Natural-language LLM-agent rule files (AGENTS.md, CLAUDE.md, SKILL.md, .cursor/rules/*.mdc, and friends) crawled from public GitHub repositories, with the content stored inline.

2,204,470 files from 45,014 repositories. Each row is one rule file, pinned to the commit SHA it was read at, so link always resolves to the exact bytes in file.

Columns

column	type	description
`file`	string	full text of the rule file
`content_sha256`	string	SHA-256 of `file`
`path`	string	path within the source repository
`link`	string	permalink to the file at the pinned commit
`repo`	string	`owner/name` of the source repository
`stars`	int64	stargazer count of the source repository at crawl time
`crawled_time`	timestamp	when the file was fetched

Composition

No single filename dominates — AGENTS.md is under 3% of rows. Skill bundles contribute the most files, while the named manifests reach the most repositories.

family	rows	share	repos
`SKILL.md` (Claude skills)	1,088,809	49.4%	15,835
supporting `.md` inside `.claude/`, `.cursor/`, …	655,713	29.7%	19,415
`agents/*.md` (subagent definitions)	139,997	6.4%	8,592
`.mdc` (Cursor rules)	77,911	3.5%	8,022
`AGENTS.md`	60,466	2.7%	15,924
`CLAUDE.md`	52,717	2.4%	18,869
`.prompt.md`	48,898	2.2%	4,738
`.instructions.md`	34,364	1.6%	4,951
`.cursorrules`	13,234	0.6%	5,278
`GEMINI.md`	10,754	0.5%	3,946
`copilot-instructions.md`	9,476	0.4%	8,618
`.chatmode.md` (Copilot)	7,699	0.3%	1,801
`.windsurfrules`	4,053	0.2%	3,389
`.goosehints`	379	0.02%	323

Usage

python

from datasets import load_dataset

ds = load_dataset("Andwwy/rules", split="train")

Or query the parquet directly, without downloading, from DuckDB:

sql

INSTALL httpfs; LOAD httpfs;

-- ALL rule files, most-starred first
SELECT repo, path, stars
FROM read_parquet('hf://datasets/Andwwy/rules/data/*.parquet')
ORDER BY stars DESC
LIMIT 20;

See what the corpus is made of:

sql

SELECT CASE
         WHEN path ILIKE '%SKILL.md'                THEN 'SKILL.md'
         WHEN path ILIKE '%.mdc'                    THEN 'cursor .mdc'
         WHEN path ILIKE '%AGENTS.md'               THEN 'AGENTS.md'
         WHEN path ILIKE '%CLAUDE.md'               THEN 'CLAUDE.md'
         WHEN path ILIKE '%.prompt.md'              THEN '.prompt.md'
         WHEN path ILIKE '%.instructions.md'        THEN '.instructions.md'
         WHEN path ILIKE '%.chatmode.md'            THEN '.chatmode.md'
         WHEN path ILIKE '%.cursorrules%'           THEN '.cursorrules'
         WHEN path ILIKE '%.windsurfrules%'         THEN '.windsurfrules'
         WHEN path ILIKE '%.goosehints%'            THEN '.goosehints'
         WHEN path ILIKE '%GEMINI.md'               THEN 'GEMINI.md'
         WHEN path ILIKE '%copilot-instructions.md' THEN 'copilot-instructions.md'
         WHEN path ILIKE '%/agents/%'               THEN 'agents/*.md'
         ELSE 'other'
       END                  AS family,
       count(*)             AS files,
       count(DISTINCT repo) AS repos
FROM read_parquet('hf://datasets/Andwwy/rules/data/*.parquet')
GROUP BY family
ORDER BY files DESC;

Narrow to one ecosystem — everything Cursor reads, say:

sql

SELECT repo, path, stars, file
FROM read_parquet('hf://datasets/Andwwy/rules/data/*.parquet')
WHERE path ILIKE '%.mdc' OR path ILIKE '%.cursorrules%'
ORDER BY stars DESC;

Or pull the whole rule set for a single repository:

sql

SELECT path, length(file) AS bytes, link
FROM read_parquet('hf://datasets/Andwwy/rules/data/*.parquet')
WHERE repo = 'anthropics/claude-code'
ORDER BY path;

Collection

Repositories are discovered via the GitHub Code Search API, pinned to a commit via GraphQL, enumerated with the Git Trees API, and the file bodies are fetched from raw.githubusercontent.com at that commit. Code search selects repositories only — it caps at 1,000 results per query and surfaces only a fraction of a repo's matching files — so the file list always comes from the Trees API, which returns an entire repository in one request.

Deduplicated on (repo, path, commit). Identical text appearing in different repositories is kept as separate rows: 2,200,687 distinct files span 1,088,448 distinct contents, and that redundancy is itself signal about what gets copied. Symbolic links, and files whose entire body is a pointer to another file (e.g. @AGENTS.md), are dropped.

Licensing

Each file remains under the license of the repository it came from; link and repo identify the source. No single license applies to the collection as a whole.