mlfoundations/MINT-1T-HTML
🍃 MINT-1T:Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens 🍃 MINT-1T is an open-source Multimodal INTerleaved dataset with 1 trillion text tokens and 3.4 billion images, a 10x scale-up from existing open-source datasets. Additionally, we include previously untapped sources such as PDFs and ArXiv papers. 🍃 MINT-1T is designed to facilitate research in multimodal pretraining. 🍃 MINT-1T is created by a team from the University of Washington in… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/MINT-1T-HTML.
Update README.md
Update README.md
make shards smaller (#8)
Update README.md (#9)
cast-metadata-as-str (#7)
Update README.md
Update README.md
Update README.md
Update README.md
fix-data-format (#6)
Update README.md
Delete CC-MAIN-* directories (#5)
Update README.md
Update README.md
Update README.md
Update README.md
Update README.md
Add file data_v1_1/shard_018_part_003.parquet
Add file data_v1_1/shard_018_part_002.parquet
Add file data_v1_1/shard_018_part_001.parquet
Add file data_v1_1/shard_018_part_004.parquet
Add file data_v1_1/shard_018_part_005.parquet
Add file data_v1_1/shard_017_part_003.parquet
Add file data_v1_1/shard_017_part_002.parquet
Add file data_v1_1/shard_017_part_004.parquet
Add file data_v1_1/shard_017_part_001.parquet
Add file data_v1_1/shard_017_part_005.parquet
Add file data_v1_0/shard_060_part_001.parquet
Add file data_v1_0/shard_060_part_002.parquet
Add file data_v1_0/shard_060_part_009.parquet
Add file data_v1_0/shard_060_part_005.parquet
Add file data_v1_0/shard_060_part_003.parquet
Add file data_v1_0/shard_060_part_008.parquet
Add file data_v1_0/shard_060_part_006.parquet
Add file data_v1_0/shard_060_part_004.parquet
Add file data_v1_0/shard_060_part_007.parquet
Add file data_v1_0/shard_059_part_001.parquet
Add file data_v1_0/shard_059_part_003.parquet
Add file data_v1_0/shard_059_part_002.parquet
Add file data_v1_0/shard_059_part_007.parquet
Add file data_v1_0/shard_059_part_005.parquet
Add file data_v1_0/shard_059_part_006.parquet
Add file data_v1_0/shard_059_part_008.parquet
Add file data_v1_0/shard_059_part_004.parquet
Add file data_v1_0/shard_059_part_009.parquet
Add file data_v1_0/shard_058_part_005.parquet
Add file data_v1_0/shard_058_part_001.parquet
Add file data_v1_0/shard_058_part_002.parquet
Add file data_v1_0/shard_058_part_003.parquet
Add file data_v1_0/shard_058_part_004.parquet