pkhw2023/openshape-contamination-axes-checkpoints

OpenShape Benchmark Contamination Artifacts (BMVC 2026, Anonymous Release)

Released anonymously for BMVC 2026 double-blind review. Authorship and provenance will be revealed on acceptance.

This release accompanies a BMVC 2026 submission analyzing benchmark contamination in 3D representation learning. It contains the training prune masks, training/eval configs, eval-time per-step metrics, training logs, NN-proxy predictions, and the best-LVIS checkpoint for the counterfactual training runs used to support the paper's headline claims.

What's in this release

This is split across two anonymous repos:

File map (paper claim → file)

The paper analyzes how four kinds of training-set "leakage" affect downstream LVIS zero-shot top-1 accuracy. Each run trains an OpenShape PointBERT encoder against a different prune mask of the training corpus, then evaluates on the same held-out LVIS split. The "T1" stack is single-stack (PointBERT-only); "T2-MLP" adds a small text-tower MLP, matching the paper's primary three-stack reported numbers.

Full per-row mapping is in CLAIM_TO_FILE_MAP.md (copied from the paper supplementary).

Repo layout

masks/                      4 × .npy boolean prune masks (one per run)
configs/                    Training YAML configs (one per checkpoint)
metrics/                    Per-step train + per-epoch eval JSONL logs
checkpoints/                best_lvis.pt for 5 runs (Model repo only)
code/                       Training launcher + losses + mask-build scripts
predictions/                NN-proxy per-UID predictions + Uni3D gap decomposition
splits/                     LVIS eval split + ModelNet40 test split
logs/                       Raw training logs (.log) for all 11 train runs
README.md                   This file
CLAIM_TO_FILE_MAP.md        Per-paper-claim file mapping
REPRODUCE.md                Runbook for reproducing every numeric claim
CITATIONS.md                External data dependencies (Cap3D, etc.)
LICENSE                     CC-BY-4.0

What's NOT here

• —Run-A T1, Run-A T2-MLP, Run-B v2 T1, Run-C T1, Run-0 T1 checkpoints — these training runs were on temporary pod scratch storage that was reclaimed before snapshot. Per-step metrics, training logs, masks, and configs ARE in this release for all of them, so the runs are fully reproducible from the bundled code (see REPRODUCE.md).
• —OpenShape training corpus point clouds — not redistributed (~700 GB); download from the original OpenShape release per CITATIONS.md.
• —Cap3D caption corpora — required for the own-caption proxy (Table 1); download per CITATIONS.md. Reproducible aggregates for the captioner-sweep rows are bundled at data/audit/G6_captioner_sweep_own.md (in the supplementary ZIP, not this HF repo).

How to load

from huggingface_hub import snapshot_download
import torch, numpy as np, json

# Data repo
data_path = snapshot_download(
    repo_id="<anon-org>/openshape-contamination-axes-data",
    repo_type="dataset",
)
mask = np.load(f"{data_path}/masks/run_C.npy")
config = open(f"{data_path}/configs/run_c_v3_t2_mlp_config.yaml").read()

# Model repo
ckpt_path = snapshot_download(
    repo_id="<anon-org>/openshape-contamination-axes-checkpoints",
)
state = torch.load(f"{ckpt_path}/run_c_v3_t2_mlp_best_lvis.pt", map_location="cpu")
# State dict keys follow the standard OpenShape PointBERT layout

License

• —Code, masks, configs, metrics, logs, predictions, splits: CC-BY-4.0.
• —Checkpoints: released under the same license as the upstream OpenShape weights (MIT — see <https://github.com/Colin97/OpenShape_code/blob/master/LICENSE>).

Citation

To be added after the double-blind review period.