submissionpaper1234/mixture-then-select-selections

Frozen Qwen3-8B Selections

This dataset contains the exact selection metadata and pool indices used for a frozen cross-scale data-selection experiment. The subsets were selected using Qwen3-8B-derived information and can be transferred unchanged to a tokenizer-compatible larger target model.

The companion training and evaluation code is:

<https://github.com/submissionpaper1234/mixture-then-select-reproducibility>

Critical Interpretation

The selected instruction text was not generated by Qwen3-8B. Qwen3-8B was the selection-source model used to derive allocations or rankings over an existing instruction-tuning pool.

This repository publishes indices and metadata only. It does not redistribute the mixed-source instruction text or any model weights.

Contents

Coverage

The artifact contains 52 static subsets, each with exactly 1,000 unique indices:

DIDS-SFT is intentionally absent from the static bundle. DIDS performs online, target-model-dependent sampling, so representing it as a frozen Qwen3-8B subset would change the baseline.

Canonical Pool Contract

Indices have meaning only for the exact ordered pool below:

The original build used 13 public instruction-tuning datasets, but exact upstream dataset revisions were not recorded. Therefore, rebuilding from the current upstream main revisions is not guaranteed to reproduce the canonical bytes. The provided tools fail closed unless the supplied pool matches the record count, byte size, and SHA-256.

Use With the Target-Scale Pipeline

Download this dataset:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="submissionpaper1234/mixture-then-select-selections",
    repo_type="dataset",
    local_dir="mixture-then-select-selections",
)

Copy or link selection_bundle.json into the code repository:

cp mixture-then-select-selections/selection_bundle.json \
  mixture-then-select-reproducibility/selections/selection_bundle.json

The code package validates the bundle hash, every index list, the target pool, and the complete 65-row matrix before launching.

Materialize One or More Static Subsets

To export records from an independently obtained canonical pool:

python3 materialize_subsets.py \
  --pool-file /absolute/path/to/train_pool.json \
  --selection-id submitted_mts \
  --selection-id regmix_sft \
  --output-dir ./materialized

To materialize all 52:

python3 materialize_subsets.py \
  --pool-file /absolute/path/to/train_pool.json \
  --all \
  --output-dir ./materialized

Materialized records preserve the selection-list order. Each export includes a manifest binding it to the canonical pool and selection hashes.

Frozen Transfer Rules

For a valid cross-scale comparison:

1. 1.Freeze these static subsets before observing target-scale results.
2. 2.Train every static method on its exact 1,000 indices.
3. 3.Do not use the target model to rescore, rerank, recalibrate, or replace a frozen static subset.
4. 4.Keep training and evaluation settings common across methods.
5. 5.Run DIDS natively online and separately disclose its unique-example access, exposures, and compute.
6. 6.Evaluate on IFEval, MMLU, GSM8K, and ARC-Challenge under the companion protocol.

Validation

python3 validate_artifacts.py

The validator checks:

• —The sealed bundle and table hashes.
• —Exactly 52 selections and 52,000 table rows.
• —Exactly 1,000 unique in-range indices per subset.
• —Every per-selection SHA-256.
• —Agreement between the nested bundle and tabular representation.
• —Absence of private filesystem provenance.

Limitations

• —Raw training text is not included because it combines upstream datasets with distinct licenses and terms.
• —Exact upstream revisions were not retained in the original pool build.
• —Static selections transfer Qwen3-8B selection information; they are not native target-model recalibrations.
• —Selection indices are unusable with a differently ordered or reformatted pool.
• —Task-conditioned Modern3 and TAP profiles should be interpreted using their declared objective names rather than as one universal selector.

Licensing

The selection indices, normalized metadata, manifests, and documentation are released under CC BY 4.0. The helper scripts are released under the MIT License. These licenses do not apply to upstream instruction datasets, models, or benchmarks. Obtain each upstream resource under its original terms.