Datasetpublicgated

hiimivantang-org/chunked-enwiki-ns0-20250301-enterprise-html

20250301.jsonl contains preprocessed text chunks from the wikipedia 20250301 html dump. dead_letter_queue.jsonl contains articles that couldn't be processed due to formatitng or missing tags. en_redirection_map.pkl can be deserialize to avoid building redirection_map everytime preprocessing_wikipedia_html_dump runs.

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes1downloads

discussions and pull requests

Conversations for this repository live on Hugging Face.

CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.

Open discussions on Hugging Face