Datasetpublic

mlfoundations/MINT-1T-HTML

🍃 MINT-1T:Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens 🍃 MINT-1T is an open-source Multimodal INTerleaved dataset with 1 trillion text tokens and 3.4 billion images, a 10x scale-up from existing open-source datasets. Additionally, we include previously untapped sources such as PDFs and ArXiv papers. 🍃 MINT-1T is designed to facilitate research in multimodal pretraining. 🍃 MINT-1T is created by a team from the University of Washington in… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/MINT-1T-HTML.

sourceHugging Facecc-by-4.0updated 2y agoView on Hugging Face

98likes698kdownloads

root.gitattributes

.gitattributes56 linesDownload Raw Back to root

1*.7z filter=lfs diff=lfs merge=lfs -text2*.arrow filter=lfs diff=lfs merge=lfs -text3*.bin filter=lfs diff=lfs merge=lfs -text4*.bz2 filter=lfs diff=lfs merge=lfs -text5*.ckpt filter=lfs diff=lfs merge=lfs -text6*.ftz filter=lfs diff=lfs merge=lfs -text7*.gz filter=lfs diff=lfs merge=lfs -text8*.h5 filter=lfs diff=lfs merge=lfs -text9*.joblib filter=lfs diff=lfs merge=lfs -text10*.lfs.* filter=lfs diff=lfs merge=lfs -text11*.lz4 filter=lfs diff=lfs merge=lfs -text12*.mlmodel filter=lfs diff=lfs merge=lfs -text13*.model filter=lfs diff=lfs merge=lfs -text14*.msgpack filter=lfs diff=lfs merge=lfs -text15*.npy filter=lfs diff=lfs merge=lfs -text16*.npz filter=lfs diff=lfs merge=lfs -text17*.onnx filter=lfs diff=lfs merge=lfs -text18*.ot filter=lfs diff=lfs merge=lfs -text19*.parquet filter=lfs diff=lfs merge=lfs -text20*.pb filter=lfs diff=lfs merge=lfs -text21*.pickle filter=lfs diff=lfs merge=lfs -text22*.pkl filter=lfs diff=lfs merge=lfs -text23*.pt filter=lfs diff=lfs merge=lfs -text24*.pth filter=lfs diff=lfs merge=lfs -text25*.rar filter=lfs diff=lfs merge=lfs -text26*.safetensors filter=lfs diff=lfs merge=lfs -text27saved_model/**/* filter=lfs diff=lfs merge=lfs -text28*.tar.* filter=lfs diff=lfs merge=lfs -text29*.tar filter=lfs diff=lfs merge=lfs -text30*.tflite filter=lfs diff=lfs merge=lfs -text31*.tgz filter=lfs diff=lfs merge=lfs -text32*.wasm filter=lfs diff=lfs merge=lfs -text33*.xz filter=lfs diff=lfs merge=lfs -text34*.zip filter=lfs diff=lfs merge=lfs -text35*.zst filter=lfs diff=lfs merge=lfs -text36*tfevents* filter=lfs diff=lfs merge=lfs -text37# Audio files - uncompressed38*.pcm filter=lfs diff=lfs merge=lfs -text39*.sam filter=lfs diff=lfs merge=lfs -text40*.raw filter=lfs diff=lfs merge=lfs -text41# Audio files - compressed42*.aac filter=lfs diff=lfs merge=lfs -text43*.flac filter=lfs diff=lfs merge=lfs -text44*.mp3 filter=lfs diff=lfs merge=lfs -text45*.ogg filter=lfs diff=lfs merge=lfs -text46*.wav filter=lfs diff=lfs merge=lfs -text47# Image files - uncompressed48*.bmp filter=lfs diff=lfs merge=lfs -text49*.gif filter=lfs diff=lfs merge=lfs -text50*.png filter=lfs diff=lfs merge=lfs -text51*.tiff filter=lfs diff=lfs merge=lfs -text52# Image files - compressed53*.jpg filter=lfs diff=lfs merge=lfs -text54*.jpeg filter=lfs diff=lfs merge=lfs -text55*.webp filter=lfs diff=lfs merge=lfs -text56