Apppublic

Yyk040316/long-context-icl

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes

rootIntegrate_Codedatasetssave_datasets.py

save_datasets.py20 linesDownload Raw Back to datasets

1import os2import datasets3from datasets import load_dataset4 5 6 7 8 9# 下载数据集10dataset = datasets.load_dataset("ccdv/govreport-summarization")11 12 13# 指定保存路径14save_directory = "/data/yyk/experiment/datasets/long-context-icl/Integrate_Code/Summarization"15 16# 保存数据集17dataset.save_to_disk(save_directory)18 19# 打印数据集信息20print(f"数据集已保存到 {save_directory}")