Datasetpublic

m-a-p/IN-Scientific

📥 IN-Scientific IN-Scientific: An Open Multimodal Interleaved Dataset for Scientific Knowledge Representation This project is a subproject of the 📌PIN project, focusing on the development of the largest scientific document multimodal dataset, which integrates both text and images. 📑: https://arxiv.org/abs/2406.13923 🤗: https://huggingface.co/datasets/m-a-p/PIN-14M Dataset statistics Source Content Images (#) Content Images (Size GB) Documents (#)… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/IN-Scientific.

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

0likes32downloads

Dataset Card

📥 IN-Scientific

IN-Scientific: An Open Multimodal Interleaved Dataset for Scientific Knowledge Representation

This project is a subproject of the 📌PIN project, focusing on the development of the largest scientific document multimodal dataset, which integrates both text and images.

📑: https://arxiv.org/abs/2406.13923

🤗: https://huggingface.co/datasets/m-a-p/PIN-14M

Dataset statistics

Source	Content Images (#)	Content Images (Size GB)	Documents (#)	Documents (Size GB)	Llama3 Tokens (#)
IN-Arxiv	3.947 M	507.99	0.715 M	37.38	11.752 B
IN-PMC	19.711 M	2154.95	5.677 M	219.00	57.500 B
Total	23.658 M	2662.94	6.392 M	256.38	69.252 B

Examples

IN-Arxiv

json

{
    "id": "1407.4558",
    "meta": {
        "language": "en",
        "source": "Arxiv",
        "date_download": "2024-12-03"
    },
    "quality_signals": {
        "doc_length": 55245,
        "num_imgs": 1,
        "llama3_tokens_count": 22007
    },
    "content_image": [
        "content_image/1407.4558/x1.png"
    ],
    "md": "# Div First-Order System LL* (FOSLL*) for Second-Order Elliptic Partial Differential Equations \u2020\n[FOOTNOTE:\u2020][ENDFOOTNOTE]\n\nZhiqiang Cai\n\n Department of Mathematics ..."
}

IN-PMC

json

{
    "id": "PMC3231073",
    "meta": {
        "date_download": "2024-12-14",
        "language": "en",
        "source": "PMC"
    },
    "quality_signals": {
        "doc_length": 20785,
        "llama3_tokens_count": 4364,
        "num_imgs": 7
    },
    "content_image": [
        "content_image/PMC3231073/sensors-10-10663-v2f1.jpg",
        "content_image/PMC3231073/sensors-10-10663-v2f2.jpg",
        "content_image/PMC3231073/sensors-10-10663-v2f3.jpg",
        "content_image/PMC3231073/sensors-10-10663-v2f4.jpg",
        "content_image/PMC3231073/sensors-10-10663-v2f5.jpg",
        "content_image/PMC3231073/sensors-10-10663-v2f6.jpg",
        "content_image/PMC3231073/sensors-10-10663-v2f7.jpg"
    ],
    "md": "# The Use of Helmholtz Resonance for Measuring the Volume of Liquids and Solids\n\n## Abstract\n\nAn experimental investigation was undertaken to ascertain the potential of using Helmholtz resonance for volume determination and the factors that may influence accuracy. The uses for a rapid non-interference volume measurement system range from agricultural produce and mineral sampling through to liquid fill measurements. By weighing the sample the density can also measured indirectly..."
}