Odeinjul/turing-100m-static-search-eval-lfs-backup-20260720-0825

turing-100m-static-search-eval

Static 100M search-evaluation dataset with original vectors, original query/ground-truth files, a rebuilt hnswlib HNSW index, and generated PQ artifacts.

Files

• —Base: base.fbin
• —HNSW index: index_m_32_ef_500
• —Query: orig_query_100k.fbin
• —Ground truth: groundtruth.bin
• —Checksums: checksums.sha256
• —Kaggle metadata is provided via dataset-metadata.json during upload and is stored by Kaggle as dataset configuration, not as a listed data file.

Data Sources

• —base.fbin -> base.fbin from shurangwu/turing-100m version 2
• —unif_query_100k.fbin -> orig_query_100k.fbin from shurangwu/turing-100m version 2
• —unif_groundtruth_100k.bin -> groundtruth.bin from shurangwu/turing-100m version 2

Source license/terms note: Original source terms apply; Kaggle metadata uses license other.

Dataset Properties

• —Vector count: 100000000
• —Dimension: 100
• —Dtype: float32
• —Metric: l2
• —Query count label: 100k
• —Query count: 100000

Environment

• —CPU threads used: 96
• —Python: Python 3.10.12
• —hnswlib package version: unknown
• —hnswlib git commit: fe873b1cee484795bd254088c0d4579f567f1979
• —faiss: 1.14.3
• —numpy: 2.2.6
• —kaggle: unknown
• —kagglehub: 1.0.2
• —vectordb-cxl commit: 73e82318a4c346b64b8c645d030ac86696351632

HNSW Index

• —Parameters: M=32, ef_construction=500
• —Index dtype: float32
• —hnswlib space: l2
• —Thread count: 96
• —Build start: 2026-07-04 19:04:47 UTC
• —Build end: 2026-07-04 22:04:59 UTC
• —Build elapsed seconds: 10811.74
• —Built from original base vectors: true
• —Prebuilt Kaggle index reused: false

Primary Search

• —Parameters: ef_search=150, k=10
• —Thread count: 96
• —Recall@10: 0.946972
• —Batch throughput qps: 17779.257758
• —Latency mean_ms: 5.389480
• —Latency p50_ms: 4.369865
• —Latency p95_ms: 7.192574
• —Latency p99_ms: 16.896276
• —Instrumented latency from batch run: True
• —Search elapsed seconds: 5.624532

Optional ef_search Sweep

Not run unless a search_sweep.json file is included and referenced in the status log.

PQ

Implementation path: cpp

Fallback used: False

Thread count: 96

Elapsed seconds: 3072.976

Final m values: m25, m50, m100

Kaggle upload file names: pq_m25.pqcodes/.pqmeta/.pqcodebook, pq_m50.pqcodes/.pqmeta/.pqcodebook, pq_m100.pqcodes/.pqmeta/.pqcodebook

Attribution

The original Turing-ANNS-1B vectors are credited to Microsoft Turing and were obtained from the Big ANN 2021 benchmark collection.

The original source data is subject to the original Microsoft Turing-ANNS terms for non-commercial research; the original terms are reproduced in LICENSE.

We generated the indexes ourselves.