Datasetpublic

jayzou3773/less-is-moe-s1-calibration-128-seq8192

Less-is-MoE S1K calibration data — 128 samples, seq_length 8192 This is the fixed calibration artifact used to prune GPT-OSS-120B, Qwen3.5-122B-A10B, and the Gemma-4-26B-A4B causal language tower. It uses the same 128 source rows as the full-length variant: yentinglin/s1K-1.1-trl-format revision 58a01564d278477da20ead1bcf1cde8e31f36251, train, followed by Dataset.shuffle(seed=1234) and the first 128 nonempty messages rows. For pruning, concatenate messages[].content with one… See the full description on the dataset page: https://huggingface.co/datasets/jayzou3773/less-is-moe-s1-calibration-128-seq8192.

sourceHugging Faceotherupdated 4d agoView on Hugging Face

0likes95downloads

Dataset Card

Less-is-MoE S1K calibration data — 128 samples, seq_length 8192

This is the fixed calibration artifact used to prune GPT-OSS-120B, Qwen3.5-122B-A10B, and the Gemma-4-26B-A4B causal language tower. It uses the same 128 source rows as the full-length variant: yentinglin/s1K-1.1-trl-format revision 58a01564d278477da20ead1bcf1cde8e31f36251, train, followed by Dataset.shuffle(seed=1234) and the first 128 nonempty messages rows.

For pruning, concatenate messages[].content with one space, tokenize with the model's tokenizer, retain the first 8192 tokens, and do not pad. The two model-specific *-input_ids.jsonl files are the exact tensors consumed by the gradient scorer. Their manifests record hashes, full pre-truncation lengths, and per-sample lengths.

—GPT-OSS: 915,210 input tokens; 79 of 128 rows truncated.
—Qwen3.5: 934,661 input tokens; 87 of 128 rows truncated.
—Gemma 4: 933,487 input tokens; 86 of 128 rows truncated.

The upstream S1K dataset does not declare a license. This repository records that status as license: other; users must review the upstream terms.