boblaros/GraphRAG

Local GraphRAG Research Artifacts

This repository contains the data and output artifacts produced for my master's thesis research on knowledge graph construction in local GraphRAG systems. It is an evidence package for the study, rather than the code used to run the experiment.

The collection includes the study corpus, model extraction outputs, final knowledge graphs, retrieved evidence, generated answers, and evaluation results. Together, these files make it possible to inspect the experiment from source documents through to question-level outcomes.

The complete experiment harness, including the code, configuration, and commands used to produce these artifacts, is available on GitHub.

What was compared

The experiment used 12 local language models to build graphs under three cumulative conditions:

Native -> ER -> ER+RR

Native is the original LightRAG graph. ER applies Entity Resolution to that graph. ER+RR then applies Relation Recovery to the ER result. Because the conditions form a sequence, this collection does not include a separate Native+RR condition or measure RR as an independent intervention.

What is included

• —120 MultiHop-RAG questions: 30 inference, 30 comparison, 30 temporal, and 30 unanswerable.
• —155 documents: 125 gold documents and 30 hard negatives.
• —12 builder models and 36 graph conditions.
• —Raw model calls and normalized extraction records.
• —Final graph nodes and edges for every model and condition.
• —36 retrieval outputs, 36 generated-answer outputs, and 4,320 question-level evaluation records.
• —Paired comparisons between the cumulative conditions and exploratory cross-model analyses.

<details> <summary>Builder model identifiers</summary>

gemma3_270m, gemma3_1b, gemma3_4b, gemma3_12b, qwen3_0_6b, qwen3_1_7b, qwen3_4b, qwen3_8b, qwen35_0_8b, qwen35_2b, qwen35_4b, and qwen35_9b.

</details>

Finding the files

Files are grouped first by builder model and then, where applicable, by graph condition. The top-level manifest.json provides the role, record count, source, size, and checksum of every published payload file.

Loading the data

Each JSONL file has one consistent schema and can be loaded separately with 🤗 Datasets. For example:

from datasets import load_dataset

questions = load_dataset(
    "json",
    data_files="corpus/questions.jsonl",
    split="train",
)

metrics = load_dataset(
    "json",
    data_files="evaluation/question_metrics/qwen35_2b/native_lightrag.jsonl",
    split="train",
)

The different artifact families use different schemas. Question-level files can be joined on question_id. Model, run, and condition fields preserve lineage between extraction, graphs, retrieval, answers, and evaluation.

Integrity

On Linux:

sha256sum -c checksums.sha256

On macOS:

shasum -a 256 -c checksums.sha256

Study design and provenance

The corpus is a fixed, question-type-balanced subset derived from MultiHop-RAG. Extraction was performed once per builder, after which ER and RR reused the saved extraction records. Retrieval and answering used the same questions and fixed downstream models in all three conditions. The records and manifest preserve model identities, hashes, schema versions, and run lineage.

Mutable LightRAG workspaces, vector stores, logs, caches, and model weights are not included because they are execution infrastructure rather than the research artifacts intended for analysis and reuse. See the experiment harness linked above for the implementation and reproduction workflow.

Limitations

The subset is balanced by question type rather than sampled to reproduce the full source distribution. Model outputs may contain extraction, retrieval, or answering errors and should not be treated as verified facts. Graph structure alone does not establish semantic correctness or downstream answer quality. The corpus contains public news text and may mention identifiable people and organizations present in the source articles.

License

The source MultiHop-RAG data and this derived artifact collection are provided under the Open Data Commons Attribution License (ODC-By) v1.0. Attribute Yixuan Tang and Yi Yang and retain the license and provenance notices when redistributing the data. Generated artifacts also retain model and source lineage for research attribution.

ODC-By governs the database and attribution terms for this package. Underlying article content may remain subject to rights held by its original publishers; users should review the relevant source terms for uses beyond research and reproducibility.

Citation

Please cite the accompanying thesis:

@mastersthesis{kutivadze2026graphrag,
  author = {Georgii Kutivadze},
  title = {Knowledge Graph Construction Quality in Local GraphRAG: An Empirical Study of Model Capacity, Graph Post-Processing, and Downstream Performance},
  school = {Università Cattolica del Sacro Cuore},
  year = {2026},
  type = {Master's thesis}
}

Please also cite the source dataset and graph framework:

@misc{tang2024multihoprag,
  title = {MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries},
  author = {Yixuan Tang and Yi Yang},
  year = {2024},
  eprint = {2401.15391},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL}
}

@article{guo2024lightrag,
  title = {LightRAG: Simple and Fast Retrieval-Augmented Generation},
  author = {Zirui Guo and Lianghao Xia and Yanhua Yu and Tu Ao and Chao Huang},
  year = {2024},
  eprint = {2410.05779},
  archivePrefix = {arXiv},
  primaryClass = {cs.IR}
}