Programmer-RD-AI/sfia-9-chunks

sfia-9-chunks Dataset Overview The sfia-9-chunks dataset is a derived dataset from sfia-9-scraped. It uses sentence embeddings and hierarchical clustering to split each SFIA-9 document into coherent semantic chunks. This chunking facilitates more efficient downstream tasks like semantic search, question answering, and topic modeling. Chunking Methodology We employ the following procedure to generate chunks: from sentence_transformers import… See the full description on the dataset page: https://huggingface.co/datasets/Programmer-RD-AI/sfia-9-chunks.