JulianKrgd/wikipedia-en-julian

Wikipedia English - JULIAN Training Dataset

This dataset contains cleaned English Wikipedia articles used to train the JULIAN-100M language model.

Dataset Description

• —Language: English
• —Source: Wikipedia dumps (latest available)
• —Size: ~3.5 billion tokens (~9.8GB JSONL, ~2-3GB Parquet)
• —Format: Cleaned articles with title, text, and URL
• —License: Creative Commons Attribution-ShareAlike 3.0

Dataset Structure

Data Fields

• —title (string): Article title
• —text (string): Full article text (cleaned and formatted)
• —url (string): Original Wikipedia URL
• —language (string): Language code ("en")

Data Example

{
  "title": "Artificial Intelligence",
  "text": "Artificial intelligence (AI) is intelligence demonstrated by machines...",
  "url": "https://en.wikipedia.org/wiki/Artificial_Intelligence",
  "language": "en"
}

Data Collection

Source

Downloaded from Wikimedia dumps (English Wikipedia).

Processing Pipeline

1. 1.Download: Latest Wikipedia XML dump
2. 2.Extraction: Parse XML, extract article text
3. 3.Cleaning:
4. 4.Remove Wiki markup and templates
5. 5.Remove infoboxes and navigation elements
6. 6.Clean HTML entities and special characters
7. 7.Remove very short articles (<50 characters)
8. 8.Remove duplicate content
9. 9.Filtering:
10. 10.Keep only main namespace articles
11. 11.Remove disambiguation and redirect pages
12. 12.Filter low-quality content
13. 13.Formatting: Convert to JSONL with structured fields

Statistics

Usage

Loading with Datasets Library

from datasets import load_dataset

# Load full dataset
dataset = load_dataset("juliankerignard/wikipedia-en-julian", split="train")

# Stream for large datasets
dataset = load_dataset("juliankerignard/wikipedia-en-julian", split="train", streaming=True)

# Example: Get first article
print(dataset[0]['title'])
print(dataset[0]['text'][:200])

Training Example

from datasets import load_dataset
import sentencepiece as spm

# Load dataset
dataset = load_dataset("juliankerignard/wikipedia-en-julian", split="train", streaming=True)

# Load tokenizer
tokenizer = spm.SentencePieceProcessor()
tokenizer.Load("julian_24k.model")

# Tokenize and prepare for training
def tokenize_function(examples):
    return {"input_ids": tokenizer.EncodeAsIds(examples["text"])}

tokenized_dataset = dataset.map(tokenize_function, batched=True)

Limitations and Bias

Limitations

1. 1.Wikipedia Bias: Reflects Wikipedia's editorial policies and contributor demographics
2. 2.Coverage Gaps: Some topics are over-represented (technology, Western culture), others under-represented
3. 3.Temporal Snapshot: Knowledge is frozen at the time of the dump
4. 4.Style Homogeneity: Encyclopedia writing style, not conversational or creative writing

Potential Biases

• —Geographic: English Wikipedia has more coverage of English-speaking countries
• —Demographic: Reflects Wikipedia editor demographics (primarily male, Western)
• —Topic: Technology and pop culture are over-represented vs. non-Western topics
• —Recency: Recent events have more coverage than historical topics

Ethical Considerations

• —Contains encyclopedic content, which may include sensitive topics
• —Not suitable for training models to be used in high-stakes decision making
• —Users should be aware of Wikipedia's known biases when using this dataset
• —Recommended for research and educational purposes

License

This dataset is derived from Wikipedia content, which is licensed under:

• —Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)
• —GNU Free Documentation License (GFDL)

See Wikipedia's copyright policy for details.

Citation

If you use this dataset, please cite:

@misc{julian_wikipedia_en_2025,
  title={Wikipedia English - JULIAN Training Dataset},
  author={Julian Kerignard},
  year={2025},
  howpublished={\\url{https://huggingface.co/datasets/juliankerignard/wikipedia-en-julian}},
  note={Derived from English Wikipedia dumps}
}

Also cite the original Wikipedia content:

@misc{wikipedia_en,
  author = "{Wikipedia contributors}",
  title = "English Wikipedia",
  year = "2025",
  howpublished = {\\url{https://en.wikipedia.org/}},
  note = "[Online; accessed DATE]"
}

Related Resources

• —Model: JULIAN-100M - Trained on this dataset
• —French Dataset: wikipedia-fr-julian
• —Tokenizer: Included in JULIAN-100M model repository

Contact

• —Author: Julian Kerignard
• —HuggingFace: https://huggingface.co/juliankerignard

Note: This is a research dataset created for training the JULIAN-100M language model. For the latest Wikipedia content, please visit wikipedia.org.