SII-LancelotXie/DRIFT_LFRP

DRIFT_LFRP

DRIFT_LFRP is a curriculum-style dataset constructed from the English Wikipedia snapshot dated November 1, 2023. The source dataset is released by Wikimedia on Hugging Face: https://huggingface.co/datasets/wikimedia/wikipedia

The original data comes from Wikipedia snapshots provided by Wikimedia.

Each example corresponds to a cleaned Wikipedia entry segment. The dataset is organized by token-length intervals, where token counts are computed using Qwen2Tokenizer.

Associated Paper

This dataset is the official resource for the paper: [Decoupled Reasoning with Implicit Fact Tokens (DRIFT): A Dual-Model Framework for Efficient Long-Context Inference](https://arxiv.org/abs/2602.10021).

Dataset Structure

Each subset corresponds to a token-length interval:

• —64_128
• —128_256
• —256_512
• —512_1024

Each subset contains:

• —train
• —validation
• —test

Note: The additional subsets (1024-2048, 2048-4096 and 4096-8192 tokens) are currently used for constructing the QAFT (Query-Aware Fine-Tuning) task and were not utilized in the LFRP (Latent Fact Reconstruction Pretraining) data. However, as they share the same data format, they are included here for consistency and future extensions.

Data Fields

Usage

from datasets import load_dataset

# 加载特定长度区间的数据集，例如 64-128
dataset = load_dataset("SII-LancelotXie/DRIFT_LFRP", "64_128")

# 查看数据
print(dataset["train"][0])

Citation

If you find this dataset or the DRIFT framework useful in your research, please cite our work:

@misc{xie2026decoupledreasoningimplicitfact,
      title={Decoupled Reasoning with Implicit Fact Tokens (DRIFT): A Dual-Model Framework for Efficient Long-Context Inference}, 
      author={Wenxuan Xie and Yujia Wang and Xin Tan and Chaochao Lu and Xia Hu and Xuhong Wang},
      year={2026},
      eprint={2602.10021},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={[https://arxiv.org/abs/2602.10021](https://arxiv.org/abs/2602.10021)}, 
}