CoolFace
Datasetpublic

Svngoku/mande-ancient-treasures-de-grunne-van-dyke-2016

mande-ancient-treasures-de-grunne-van-dyke-2016 Dataset created with PDF2Dataset -- OCR + structure-aware chunking pipeline. Dataset Summary Metric Value Total chunks 268 Avg chars/chunk 722 Avg images/chunk 0.14 Source files 1 Duplicates removed 0 Quality filtered 6 Schema Column Type Description chunk_id string Unique identifier: filename_chunk_N text string Raw markdown chunk with image refs text_clean… See the full description on the dataset page: https://huggingface.co/datasets/Svngoku/mande-ancient-treasures-de-grunne-van-dyke-2016.

sourceHugging Facemitupdated 3mo agoView on Hugging Face
0likes8downloads
Dataset Card

mande-ancient-treasures-de-grunne-van-dyke-2016

Dataset created with PDF2Dataset -- OCR + structure-aware chunking pipeline.

Dataset Summary

MetricValue
Total chunks268
Avg chars/chunk722
Avg images/chunk0.14
Source files1
Duplicates removed0
Quality filtered6

Schema

ColumnTypeDescription
chunk_idstringUnique identifier: filename_chunk_N
textstringRaw markdown chunk with image refs
text_cleanstringCleaned text without markdown formatting
chapterstringH1 header active at chunk position
sectionstringH2 header active at chunk position
subsectionstringH3+ header active at chunk position
imageslist[Image]Rendered images extracted from chunk (viewable in Dataset Viewer)
image_refslist[string]Image reference IDs in chunk text
num_imagesintNumber of images in chunk
has_imagesboolWhether chunk contains images
source_filenamestringOriginal source file name
start_indexintCharacter offset in source document
char_countintCharacter count of chunk text

Source Files

FileChunks
mande-ancient-treasures-de-grunne-van-dyke-2016-.pdf268

Document Structure

Chapters found in the source documents:

  • —04
  • —06
  • —08
  • —10
  • —13
  • —15
  • —18
  • —19
  • —23
  • —24
  • —28
  • —Preface
  • —Préface

Pipeline

This dataset was processed through the PDF2Dataset pipeline:

  1. 1.OCR -- Mistral OCR extracts text and images from PDF/image files
  2. 2.Chunking -- Structure-aware recursive splitting preserves document hierarchy
  3. 3.Validation -- Schema validation ensures every chunk has required fields
  4. 4.Deduplication -- Content-hash based dedup removes identical chunks
  5. 5.Quality Filtering -- Removes empty, too-short, or malformed chunks