datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
govdocs1-by-extension
govdocs1 Dataset: By File Extension
[!NOTE]
PDFs from govdocs1 are at this repo in "raw" file form - no simple "mostly correct" way to convert to text
Markdown-parsed versions of documents in govdocs1 with light filtering.
Usage
Load specific file formats (e.g., .doc files) parsed to markdown with pandoc:
from datasets import load_dataset
# Replace "doc" with desired config name
dataset = load_dataset("BEE-spoke-data/govdocs1-by-extension", "doc")… See the full description on the dataset page: https://huggingface.co/datasets/BEE-spoke-data/govdocs1-by-extension.govdocs1-txt-raw
Dataset Card for "govdocs1-txt-raw"
Somewhere to put the raw txt files before filtering them
Source info/page: https://digitalcorpora.org/corpora/file-corpora/files/
@inproceedings{garfinkel2009bringing,
title={Bringing Science to Digital Forensics with Standardized Forensic Corpora},
author={Garfinkel, Simson and Farrell, Paul and Roussev, Vassil and Dinolt, George},
booktitle={Digital Forensic Research Workshop (DFRWS) 2009},
year={2009},
address={Montreal, Canada}… See the full description on the dataset page: https://huggingface.co/datasets/BEE-spoke-data/govdocs1-txt-raw.