faidrap/DCLM_German

DCLM German Dataset This dataset contains German language data processed for LLM pretraining, filtered using FastText language detection. Usage from datasets import load_dataset # Load the entire dataset dataset = load_dataset("faidrap/DCLM_German") # Stream for large datasets (recommended) dataset = load_dataset("faidrap/DCLM_German", streaming=True) # Access the data for example in dataset['train']: print(example['text'][:100]) # Print first 100 chars… See the full description on the dataset page: https://huggingface.co/datasets/faidrap/DCLM_German.