Datasetpublic

lhoestq/resumes-raw-pdf-for-ocr

Extracted lists of pages from PDF resumes and the PDF texts. Created using this code: import io import PIL.Image from datasets import load_dataset def render(pdf): images = [] for page in pdf.pages: buffer = io.BytesIO() page.to_image(height=840).save(buffer) images.append(PIL.Image.open(buffer)) return images def extract_text(pdf): return "\n".join(page.extract_text() for page in pdf.pages) ds = load_dataset("d4rk3r/resumes-raw-pdf", split="train")… See the full description on the dataset page: https://huggingface.co/datasets/lhoestq/resumes-raw-pdf-for-ocr.

sourceHugging Faceupdated 1y agoView on Hugging Face

3likes63downloads

Dataset Card

Extracted lists of pages from PDF resumes and the PDF texts.

Created using this code:

python

import io
import PIL.Image
from datasets import load_dataset

def render(pdf):
    images = []
    for page in pdf.pages:
        buffer = io.BytesIO()
        page.to_image(height=840).save(buffer)
        images.append(PIL.Image.open(buffer))
    return images

def extract_text(pdf):
    return "\n".join(page.extract_text() for page in pdf.pages)

ds = load_dataset("d4rk3r/resumes-raw-pdf", split="train")
ds = ds.map(lambda x: {
    "images": render(x["pdf"]),
    "text": extract_text(x["pdf"])
}, remove_columns=["pdf"])
ds = ds.filter(lambda x: len(x["text"].strip()) > 0)
ds.push_to_hub("lhoestq/resumes-raw-pdf-for-ocr")