dengyixuan/openhands-commit-noise-databases

OpenHands Commit Noise Databases

This dataset contains commit-retrieval databases for 12 SWE-bench repositories at five noise ratios: 0%, 25%, 50%, 75%, and 100%.

Each archive expands to noise_NNN/<repository>/ directories containing:

• —commits.db: SQLite commit records
• —commits.faiss: normalized inner-product FAISS index
• —commits.meta.jsonl: FAISS row-to-commit metadata
• —commits.index_meta.json: embedding and index configuration

The 0% archive is an exact file-level copy of the original database. The noisy datasets use deterministic nested selections with seed 42, so lower-ratio selections are subsets of higher-ratio selections. Of 644,099 planned noisy-field values, 175 generation gaps use the unchanged source field and are recorded in source_fallbacks.json.

The four generated ratios contain 323,850 commit rows each, including 322,072 indexed rows with a non-empty problem description. Embeddings use BAAI/bge-large-en-v1.5 with dimension 1024, normalization enabled, and inner-product search.

Use SHA256SUMS to verify downloads. database_validation_manifest.json records exhaustive SQLite/JSON equality checks, FAISS counts, metadata checks, and database file hashes for the generated ratios. noise_000_snapshot.json records hashes for the original 0% files.

Extract an archive with:

tar --zstd -xf commit_databases_noise_000.tar.zst