arnosimons/wikipedia-physics-corpus
Dataset Card for the Wikipedia-Physics Corpus Wikipedia-Physics Corpus contains 102,409 paragraphs extracted from 6,642 key physics-related Wikipedia articles as well as word-sense labels for 1,186 occurrences of "Planck" across 885 paragraphs. This table details the main columns of the dataset: Column Description text Full text of the paragraph length Number of unicode characters in the paragraph page_title Titel of the Wikipedia article url URL of the… See the full description on the dataset page: https://huggingface.co/datasets/arnosimons/wikipedia-physics-corpus.
Dataset Card for the Wikipedia-Physics Corpus
Wikipedia-Physics Corpus contains 102,409 paragraphs extracted from 6,642 key physics-related Wikipedia articles as well as word-sense labels for 1,186 occurrences of "Planck" across 885 paragraphs.
This table details the main columns of the dataset:
The primary purpose of the Wikipedia-Physics Corpus lies in its use for analyzing the meaning of concepts in physics. For further insights into the corpus and the underlying research project (<a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >Network Epistemology in Practice</a>) please refer to this paper:
- <a target="_blank" rel="noopener noreferrer" href="https://arxiv.org/abs/2411.14073">Simons, A. (2024). Meaning at the planck scale? Contextualized word embeddings for doing history, philosophy, and sociology of science. arXiv:2411.14073.</a>
Construction
Wikipedia articles were selected using the <a target="_blank" rel="noopener noreferrer" href="https://petscan.wmcloud.org/">PetScan</a> tool, which generated a list of all pages categorized under "physics" or its immediate subcategories. Markup was removed and minimal cleaning applied to produce plain text paragraphs, and while references were removed, all formulas were retained.
Additionally, 1,186 occurrences of the term "Planck" were labeled across 885 paragraphs containing the term. These occurrences were identified using a case-insensitive regular expression to capture variations like "PLANCK" and "Planck(2015)"" while excluding irrelevant forms like "planckian". The labels denote distinct meanings of the target term, such as PERSON, CONSTANT, UNITS, LAW, MPS, MISSION, and FOKKER.
Details
- Developer: <a target="_blank" rel="noopener noreferrer" href="https://www.tu.berlin/en/hps-mod-sci/arno-simons">Arno Simons</a>
- Funded by: The European Union under Grant agreement ID: <a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >101044932</a>
- Language: English