JoaoCoelho/scientific_papers_citation_scores

Dataset Summary

This dataset comprises an array of scientific papers, each paper is associated with a series of scores. These scores quantify the number of citations each paper has received. The data regarding the papers and their citations were sourced from OpenCitations, a comprehensive and accessible online database of scholarly citations (available at https://opencitations.net/).

How are these scores calculated?

Imagine a tree where papers are nodes and citations are edges. For instance, if paper A is cited by both papers B and C, it gets a score of 2 at tree depth 1 (scoredepth1). If paper D cites B, but C receives no citations, paper A's score at tree depth 2 (scoredepth2) is 1. These scores (scoredepth1, scoredepth2, ..., scoredepthN) are recorded for each paper, indicating the number of citations at various tree depths, up to depth N. This method offers a detailed view of a paper's influence through its citation network.

Why do I need this information?

Exploring the citation tree of scientific papers can be extremely useful if you decide to fine-tune an LLM and teach it to produce novel scientific concepts that are likely to be winners in the scientific community. It is expected that papers that produce new and valid scientific concepts are cited by many other papers and these subsequent papers have themselves also a big amount of "descendants". In other words, it may be possible to teach an LLM what a projection of a "world model" looks like since only truthful and consistent ideas will be verified by researchers and cited by future papers. The deeper the citation tree goes for a certain paper, the higher likelyhood that it has novel, true and consistent ideas in it.

Dataset Structure

The dataset is obtained from the ArXiv repository. The dataset comprises rows, each represented by an object with specific attributes. Here's a breakdown of an example row:

{
  "omid": "omid:br/061603088748",
  "pub_date": "2018-09",
  "title": "Development of different methods and their efficiencies for the estimation of diffusion coefficients following the diffusion couple technique",
  "doi": "10.1016/j.actamat.2018.04.051",
  "link": "https://arxiv.org/abs/1809.09687",
  "pdf": "https://arxiv.org/pdf/1809.09687",
  "e-print": "https://arxiv.org/e-print/1809.09687",
  "score-depth-1": 61,
  "score-depth-2": 62,
  "score-depth-3": 76,
  "score-depth-4": 62,
  "score-depth-5": 76
}

Accessing the Dataset

The most recent version of the dataset is named scores-max=10000-date=15_12_2023.parquet. This file is accessible under the "Files and versions" section. The 'max=10000' part in the filename signifies that the highest possible score is 10,000.

Suggested Approach for Fine-Tuning/Reward Model Training

For training a Reinforcement Learning (RL) model, consider the following methodology:

• —Select a subset of papers meeting specific criteria. For instance, papers with a depth-5 score greater than 9000 could be considered as 'Successful'.
• —Tip: select papers only with similar ages. Very recent papers may have fewer scores since there wasn't enough time to cite them.

• —Identify papers with a depth-1 score of 0, labeling these as 'Not Successful'.

• —Utilize these two sets to apply RL techniques on a specific Language Learning Model (LLM), thereby fine-tuning the model based on the defined success criteria.

This approach helps in categorizing papers based on their perceived impact and using this categorization to refine the learning model's performance.

Citation Information

@misc {joão_coelho_2023,
	author       = { {João Coelho} },
	title        = { scientific_papers_citation_scores (Revision b7910b4) },
	year         = 2023,
	url          = { https://huggingface.co/datasets/JoaoCoelho/scientific_papers_citation_scores },
	doi          = { 10.57967/hf/1489 },
	publisher    = { Hugging Face }
}