balrampandey/qmmit-open-source-agent-commit-index

Repository-Level Measurement of Self-Declared Coding-Agent Commit Signatures

Dataset release: 2026-09-18-v3.0 Schema: 3.0.0 Release stamp: dataset 2026-09-18-v3.0 · ruleset sha256:b2e8889c66f72c18a61839f0bf1a9f77b5481ba2def044dba33c797b2f2bdcae · scanned 2026-09-16

Abstract

This dataset contains 2000 repository-level observations from public Git repositories. Each observation estimates a lower bound on the proportion of non-merge, non-infrastructure-bot commits that contain a deterministic, self-declared coding-agent signature during the fixed calendar window 2026-02-01 through 2026-09-15.

The measurement is intentionally narrower than AI-assisted software development. Git metadata can reveal a signature when an agent participates in creating a commit; it cannot reveal assistance when a person commits manually without a signature. Accordingly, all proportions are reported as floors, not estimates of the amount of code written with AI.

The sample contains 2,236,082 considered commits. The aggregate floor is 8.5% and the median repository floor is 2%. These are descriptive statistics of repository histories, not causal estimates and not measurements of individual developer performance.

Detector validation

A stratified blind sample of 600 commits (300 detector-positive and 300 detector-negative) was independently reviewed and disagreements were adjudicated by the human study lead. Weighted precision is 99.45% (95% Wilson interval 97.77%–99.86%), weighted recall is 72.86%, raw reviewer agreement is 96.83%, and Cohen's kappa is 0.938. Nineteen cases required adjudication. AI-assisted recommendations used during adjudication are retained in the private audit trail; the human study lead reviewed and confirmed every final decision.

Research questions

The dataset supports the following descriptive questions:

1. 1.What lower-bound share of recent commits contains a coding-agent signature?
2. 2.How often are signed commits associated with assisted versus autonomous commit authorship?
3. 3.How concentrated is signed activity across repository contributors?
4. 4.How do these measurements vary across the sampled language ecosystems?

Files

• —data/train-00000-of-00001.parquet: the same 2000 observations as a compact tabular split for the Hugging Face dataset viewer and programmatic loading.
• —data/repositories.jsonl: an equivalent newline-delimited copy for reproducibility outside the Hub viewer.
• —summary.json: dataset-level summary, fixed measurement window, method, and ecosystem aggregates.
• —repository_resolution.json: row-by-row canonical URL, HTTP status, default branch, and HEAD SHA recorded during the v3 release audit.
• —release_manifest.json: SHA-256 checksums for the release's canonical data and validation artifacts.
• —CORRECTIONS.md: investigated data-quality reports and their disposition.
• —assets/: six generated SVG figures used in this card.
• —make_charts.py and validate_dataset.py: reproducible figure generation and release validation scripts.

Data and sampling

Repositories were selected from a public GitHub stars-ranked snapshot of non-fork, non-archived repositories. The ranking is a collection-time sampling frame, not a claim that stars measure software quality or represent the entire open-source population. The final dataset contains 2000 successful scans.

One initially selected repository, ClickHouse/ClickHouse, could not be cloned reliably in the collection environment and was replaced by home-assistant/frontend. The replacement is recorded in the collection workflow; no missing observation was imputed.

Repository identity audit

Before the v3 release, every one of the 2000 github_url values was resolved through GitHub and independently checked with git ls-remote --symref. All 2000 returned HTTP 200, exposed a readable HEAD SHA, matched their repo slug, and resolved to 2000 unique canonical targets. The complete audit—not just its summary—is published in repository_resolution.json.

The audit found one renamed source repository and records its canonical target in the resolution manifest. No two observations resolve to the same canonical repository.

Operational definitions and method

The scanner reads Git commit metadata and applies literal signature rules for Claude Code, Cursor, GitHub Copilot, Devin, OpenAI Codex, Aider, Gemini CLI, and other supported agent signatures. It does not use code-style heuristics, machine-learning authorship classifiers, or perplexity scoring.

considered commits = all commits - merge commits - CI/dependency bot commits
agent-share floor  = agent-attributed commits / considered commits

Agent detection runs before generic [bot] exclusion because some coding agents are GitHub Apps and use bot identities. Matched commits are split into:

• —assisted: a human identity authored the commit and the agent signed it;
• —autonomous: a bot identity authored the commit.

The dataset also reports concentration: distinct agent authors, the largest author's share of agent commits, and an adoption pattern (none, individual, team, or org-wide). Identities themselves are never exported.

Outcome variables

• —agent_share_pct_floor: detected agent-attributed commits divided by considered commits.
• —assisted_pct and autonomous_pct: the two commit-authorship modes divided by considered commits.
• —distinct_agent_authors, top_author_share_of_agent_commits, and adoption_pattern: concentration measures based on internal author counts.
• —low_activity: a quality flag for repositories with fewer than 100 considered commits. These observations remain in the dataset but should be treated as high-variance measurements.

All percentages are rounded to one decimal place by the scanner.

Agent-category commit counts are not necessarily mutually exclusive: a single commit may match more than one literal signature rule. The unique agent_attributed_commits field remains the denominator-safe primary count.

Descriptive figures

Ecosystem spread

[image]

Adoption patterns

[image]

Commit composition

[image]

Distribution of repository floors

[image]

Monthly curve

[image]

Share and concentration

[image]

Generate the figures and validate the release with:

python make_charts.py --parquet data/train-00000-of-00001.parquet --summary summary.json --out assets
python validate_dataset.py data/train-00000-of-00001.parquet summary.json repository_resolution.json

Collection and reproducibility

The reports were generated by qmmit-cli using blobless Git clones and the fixed --since 2026-02-01 window. The public site and scanner source are in the project repositories:

• —https://github.com/pandey019/qmmit-cli
• —https://qmmit.dev

The exported rows contain repository URLs, collection-time language and star metadata, aggregate counts, monthly timelines, agent categories, and public commit hash/subject samples. The raw clone directories and raw scanner artifacts are not part of the published dataset.

Interpretation

The aggregate floor should not be interpreted as the prevalence of AI-written code. A low or zero value is compatible with substantial unobserved AI assistance if developers commit without agent signatures. Conversely, a high value can reflect the workflow of a small number of contributors. The concentration fields are therefore necessary for interpreting the share.

The sample is not a random sample of all repositories. Repository age, activity, popularity, language, and Git workflow may affect both inclusion and the probability that a signature remains in history. Comparisons across ecosystems are descriptive and should not be treated as causal comparisons.

Validity threats and limitations

1. 1.This is a lower bound. Squash merges, rebases, and manual commits can strip signatures.
2. 2.A zero result means no matching signature was observed, not that AI was not used.
3. 3.Commit counts are not lines of code; a tiny fix and a large refactor each count as one commit.
4. 4.Low-activity repositories are included for coverage and marked with low_activity; their percentages are noisy.
5. 5.Repository popularity and language metadata are collection-time context, not causal variables.
6. 6.Do not use this dataset to rank, surveil, or identify individual developers.

The signature rules are versioned with the scanner but are not a complete catalogue of all coding agents. Squash merges and rebases can remove trailers; manual commits can leave no signal. Commit counts also weight a one-line fix and a large refactor equally.

Ethics and privacy

Contributor names and email addresses are used transiently by the scanner for aggregate concentration calculations and are deliberately excluded from the export. The dataset must not be joined with external identity data to produce individual-level rankings. Public commit subjects and hashes are included only as reproducibility samples and may contain third-party text.

Public commit subjects and hashes are included only as reproducibility samples. They may contain repository-authored text and remain subject to the source repository's license and terms. Users must independently review source-repo licenses before redistributing those fields.

Citation

@dataset{qmmit_agent_commit_index_2026,
   title = {Repository-Level Measurement of Self-Declared Coding-Agent Commit Signatures},
   author = {Pandey, Balram},
  year = {2026},
      version = {2026-09-18-v3.0},
   url = {https://huggingface.co/datasets/balrampandey/qmmit-open-source-agent-commit-index}
}

License

The qmmit code is MIT-licensed. This dataset is a collection of derived repository-level measurements plus public repository metadata and sample commit fields. No single license is asserted over third-party repository content; please observe each source repository's license and GitHub terms when using or redistributing the samples.