Nitin1211/dbpedia-hindi-predicate-linking-gold

DBpedia Hindi — Predicate Linking Gold Set

8,029 Hindi predicate → DBpedia property mappings, used to fine-tune F2LLM-1.7B for predicate normalization, as part of the DBpedia Hindi Chapter (Google Summer of Code 2026).

Format

{
  "predicate": "<Hindi relation text>",
  "top_50_candidates": ["<dbo: URI>", "..."],
  "gold_dbo": "dbo:propertyName",
  "raw_llm_response": "ANSWER: dbo:propertyName"
}

How This Was Built

Two-stage pipeline across 8,029 unique Hindi predicates:

1. 1.Retrieval — F2LLM-8B. For each predicate, retrieved the top-50 candidate DBpedia properties by embedding similarity against the full ontology.
2. 2.Disambiguation — GPT-OSS-120B. The retrieved candidates, with sentence context, were passed to GPT-OSS-120B to select the correct property, or confirm none genuinely fits.

Results

• —5,855 predicates (72.9%) received a confirmed real DBO mapping
• —2,174 predicates (27.1%) initially returned no match
• —A later recovery pass, searching ranks 51–100 instead of stopping at 50, recovered 1,249 of those (57.5%) — raising total coverage to 88.4%

Used For

Fine-tuning F2LLM-1.7B via QLoRA. Held-out evaluation (585 examples): precision@1 = 37.6%, precision@40 = 89.9%.

Part of a Larger Pipeline

Full code and documentation: https://github.com/singhhnitin/neural-extraction-framework/tree/gsoc26h-development/GSoC26_H