AdamLucek/quickb-kb
quickb-kb Generated using QuicKB, a tool developed by Adam Lucek. QuicKB optimizes document retrieval by creating fine-tuned knowledge bases through an end-to-end pipeline that handles document chunking, training data generation, and embedding model optimization. Chunking Configuration Chunker: RecursiveTokenChunker Parameters: chunk_size: 400 chunk_overlap: 0 length_type: 'character' separators: ['\n\n', '\n', '.', '?', '!', ' ', ''] keep_separator: True… See the full description on the dataset page: https://huggingface.co/datasets/AdamLucek/quickb-kb.
013
1---2language:3- en4pretty_name: "quickb-kb"5tags:6- quickb7- text-chunking8- 1K<n<10K9task_categories:10- text-generation11- text-retrieval12task_ids:13- document-retrieval 14library_name: quickb15---16 17# quickb-kb18 19Generated using [QuicKB](https://github.com/AdamLucek/quickb), a tool developed by [Adam Lucek](https://huggingface.co/AdamLucek).20 21QuicKB optimizes document retrieval by creating fine-tuned knowledge bases through an end-to-end pipeline that handles document chunking, training data generation, and embedding model optimization.22 23### Chunking Configuration24- **Chunker**: RecursiveTokenChunker25- **Parameters**:26 - **chunk_size**: `400`27 - **chunk_overlap**: `0`28 - **length_type**: `'character'`29 - **separators**: `['\n\n', '\n', '.', '?', '!', ' ', '']`30 - **keep_separator**: `True`31 - **is_separator_regex**: `False`32 33### Dataset Statistics34- Total chunks: 2,80735- Average chunk size: 50.7 words36- Source files: 1037 38 39 40### Dataset Structure41This dataset contains the following fields:42 43- `text`: The content of each text chunk44- `source`: The source file path for the chunk45- `id`: Unique identifier for each chunk