Odeinjul/deep-10m-batch-update-eval

deep-10m-batch-update-eval

Batch-update evaluation workload package generated from deep-10m-static-search-eval.

Dataset

• —Source static dataset: deep-10m-static-search-eval
• —Vector count: 10,000,000
• —Dimension: 96
• —Dtype: float32
• —Metric: l2
• —Initial update index: 8,000,000 vectors with external labels equal to A = P[0:8M]
• —Update order: update_order.u32, a seed-42 permutation of source IDs [0, 10M)
• —Insert vector source: base_permuted.fbin, where row j equals base.fbin[P[j]]

Traces

• —insert-20: starts from A, then inserts P[8M:10M] in twenty 100,000-vector batches.
• —delete-20: starts from the static 10M state, then deletes P[8M:10M] in twenty 100,000-vector batches.
• —mixed-replace-100: keeps 8M live vectors for 100 rounds; each round deletes a cyclic 100,000 source-ID slice and inserts base_permuted.fbin row ranges: batches 1-20 use rows 8M:10M, then batches 21-100 use rows 0:8M. Insert external IDs use existing P[8M:10M] labels for the first 20 rounds, then new labels in [10,000,000, 18,000,000).

Batch JSON files use compact descriptors (range, u32_slice, and u32_cyclic_slice) instead of inline million-ID arrays. Insert external_ids continue to express user-visible source IDs. Insert vector_refs are row ranges in the reordered insert source and must be read from base_permuted.fbin.

Ground Truth

Checkpoint ground truth is produced by filtering the static source ground truth in source-distance order through the checkpoint owner map. Files are exact top-10 only when every query retains at least 10 active candidates from the static source GT depth. If a checkpoint cannot provide top-10 for every query, the package writes a matching .invalid.json marker instead of padding.

Files

• —workload.json: workload contract.
• —static-workload-reference.json: immutable static-search-eval references.
• —source_manifest.json: generation manifest.
• —update_order.u32: seed-42 source-ID permutation.
• —initial/index_8m_m32_efc500: HNSW index built from base[A] with labels A.
• —groundtruth/active_8m.bin: initial 8M checkpoint GT, or active_8m.invalid.json.
• —initial/layout-sidecar/index_8m_m32_efc500.*: optional runtime layout sidecar for the initial HNSW index.
• —initial/pq/pq_m<M>.*: initial 8M PQ artifacts reordered for initial/index_8m_m32_efc500 internal IDs.
• —initial_pq_manifest.json: source static PQ files and validation samples for the reordered initial PQ artifacts.
• —base_permuted.fbin: reordered insert vector source, present when insert vector_refs are row ranges.
• —reordered_insert_manifest.json: source, formula, size, and sample-check manifest for base_permuted.fbin.
• —traces/*/trace.json: trace metadata.
• —traces/*/batches/*.json: compact batch descriptors.
• —traces/*/groundtruth/*: checkpoint GT or invalid markers.
• —checksums.sha256: checksums for generated package files.

Static PQ codebooks and metadata are reused. initial/pq/pq_m<M>.pqcodes contains only 8,000,000 rows and is ordered by the initial HNSW internal ID, so it can be used directly with initial/index_8m_m32_efc500.