DSSGxMunich/bplan_keyword_extraction

Dataset Card for Keyword Extraction

Dataset Description

Homepage: DSSGx Munich organization page.

Repository: GitHub.

Dataset Summary

This folder contains the exact keyword extraction and agent information extraction datasets.

Dataset Structure

Folder structure

• —exact_search
• —baunvo_keywords.csv -> appearance of BauNVO keywords in each document.
• —hochwasser_keywords.csv -> appearance of hochwasser-related keywords in each document.

• —knowledge_extraction_agent
• —fh.json -> length of firsthöhe detected by agent and result from fuzzy keyword search.
• —gfz.json -> Geschossflächenzahl detected by agent and result from fuzzy keyword search.
• —grz.json -> Grundflächenzahl detected by agent and result from fuzzy keyword search.
• —max_h.json -> Maximale gebäudehöhe detected by agent and result from fuzzy keyword search.
• —min_h.json -> Minimale gebäudehöhe detected by agent and result from fuzzy keyword search.
• —th.json -> Traufhöhe detected by agent and result from fuzzy keyword search.

Data Fields

• —baunvo_keywords.csv:
• —filename: name of PDF file that was extracted.
• —columns baunvo-XX and 13b: names of the categories that were searched for, and keywords that appeared matching that category.

• —hochwasser_keywords.csv:
• —filename: name of PDF file that was extracted.
• —contextualised_keyword: paragraph context in which the exact keyword appears.
• —actual_keyword: actual keyword searched for.
• —category: category of hochwasser keyword(hq100, hqhaufig, hqextrem)

• —All the files in knowledge_extraction_agent are .json files which contain the following structure:
• —id: id of document extracted.
• —keyword_input: fuzzy keyword input for the value extraction (context paragraph).
• —keywordagentresponse: result of the agent.
• —keywordextractedvalue: extracted value from agent.
• —validation: validation of result.

Dataset Creation

Initial Data Collection and Normalization

This is the result of the keyword extraction from the document_texts.csv file. The exact keyword extraction was done by selecting a set of relevant keywords and searching for them in the text. Meanwhile, the agent keyword extraction is the result of searching for certain keywords using fuzzy search to get the context surrounding them, and extracting relevant values with GPT.

Considerations for Using the Data

Discussion of Biases

The results of this keyword and agent results were NOT validated manually. Therefore, this is why we provide the contextual paragraph of the values: the information should be double-checked by professionals.