Odeinjul/deep-100m-batch-update-eval

deep-100m-batch-update-eval

Batch-update evaluation workload package generated from deep-100m-static-search-eval.

Dataset

• —Source static dataset: deep-100m-static-search-eval
• —Vector count: 100,000,000
• —Dimension: 96
• —Dtype: float32
• —Metric: l2
• —Initial update index: 80,000,000 vectors with external labels equal to A = P[0:80M]
• —Update order: update_order.u32, a seed-42 permutation of source IDs [0, 100M)
• —Insert vector source: base_permuted.fbin, where row j equals base.fbin[P[j]]

Traces

• —insert-20: starts from A, then inserts P[80M:100M] in twenty 1,000,000-vector batches.
• —delete-20: starts from the static 100M state, then deletes P[80M:100M] in twenty 1,000,000-vector batches.
• —mixed-replace-100: keeps 80M live vectors for 100 rounds; each round deletes a cyclic 1,000,000 source-ID slice and inserts base_permuted.fbin row ranges: batches 1-20 use rows 80M:100M, then batches 21-100 use rows 0:80M. Insert external IDs use existing P[80M:100M] labels for the first 20 rounds, then new labels in [100,000,000, 180,000,000).

Batch JSON files use compact descriptors (range, u32_slice, and u32_cyclic_slice) instead of inline million-ID arrays. Insert external_ids continue to express user-visible source IDs. Insert vector_refs are row ranges in the reordered insert source and must be read from base_permuted.fbin.

Ground Truth

Checkpoint ground truth is produced by filtering the static source ground truth in source-distance order through the checkpoint owner map. Files are exact top-10 only when every query retains at least 10 active candidates from the static source GT depth. If a checkpoint cannot provide top-10 for every query, the package writes a matching .invalid.json marker instead of padding.

Files

• —workload.json: workload contract.
• —static-workload-reference.json: immutable static-search-eval references.
• —source_manifest.json: generation manifest.
• —update_order.u32: seed-42 source-ID permutation.
• —initial/index_80m_m32_efc500: HNSW index built from base[A] with labels A.
• —groundtruth/active_80m.bin: initial 80M checkpoint GT, or active_80m.invalid.json.
• —initial/layout-sidecar/index_80m_m32_efc500.*: optional runtime layout sidecar for the initial HNSW index.
• —initial/pq/pq_m<M>.*: initial 80M PQ artifacts reordered for initial/index_80m_m32_efc500 internal IDs.
• —initial_pq_manifest.json: source static PQ files and validation samples for the reordered initial PQ artifacts.
• —base_permuted.fbin: reordered insert vector source, present when insert vector_refs are row ranges.
• —reordered_insert_manifest.json: source, formula, size, and sample-check manifest for base_permuted.fbin.
• —traces/*/trace.json: trace metadata.
• —traces/*/batches/*.json: compact batch descriptors.
• —traces/*/groundtruth/*: checkpoint GT or invalid markers.
• —checksums.sha256: checksums for generated package files.

Static PQ codebooks and metadata are reused. initial/pq/pq_m<M>.pqcodes contains only 80,000,000 rows and is ordered by the initial HNSW internal ID, so it can be used directly with initial/index_80m_m32_efc500.

PQ row-order compatibility

Exhaustive validation confirmed that the corrected M24, M48, and M96 initial PQ files use the initial index's hnsw_internal_id row order for all 80,000,000 rows.

For initial-index internal row j, let e be the external label stored in initial/index_80m_m32_efc500. Its PQ row must encode base.fbin[e]. When using the corrected static PQ, obtain its source row through the inverse static index mapping: initial_PQ[j] = static_PQ[inverse_static_mapping[e]]. Neither j nor e can be used directly as a static PQ row in general.

The previously published assertion that the initial PQ was unaffected by the static row-order defect was incorrect. Independent checks against the actual initial-index vectors found wrongly paired rows in M24, M48, and M96, despite valid headers, checksums, and an internal-order declaration. The correction changes PQ row assignments; it does not retrain codebooks or alter the HNSW index, update order, insert vectors, traces, or checkpoint ground truth.

initial_pq_manifest.json records the corrected source mapping, sample rows, and correction provenance. Validate the sidecar against actual index labels, compare every initial PQ row to the correctly mapped static row, and include nonidentity mappings when independently encoding sampled vectors. A stored manifest declaration or its sample coordinates alone is not a correctness proof.

For relocated downloads, pass the local static package root explicitly to the preparation package validator; historical authoring paths are provenance only. Insert vector_refs remain offsets in base_permuted.fbin, independent of HNSW internal IDs and externally visible labels.