LakoreAI/multilingual-classification-0001

Multilingual Text Classification Dataset

This dataset is designed for multilingual text classification tasks. It includes labeled text samples across 8 languages, making it ideal for training and evaluating models on cross-lingual transfer, language identification, and multilingual understanding.

Dataset Overview

Total Download Size: 2.6 MB Total Dataset Size: 3.8 MB Task Type: Text Classification

Data Fields

Language Labels

Intended Uses

• —Multilingual language classification
• —Cross-lingual and zero-shot evaluation
• —Benchmarking multilingual embeddings (e.g., mBERT, XLM-R, LaBSE)
• —Studying language similarity and confusion patterns

Usage Example

You can easily load the dataset using the Hugging Face datasets library:

from datasets import load_dataset

dataset = load_dataset("8Opt/multilingual-classification-0001")

example = dataset["train"][0]
print(example)

Output:

{
  "text": "Das ist ein Beispielsatz.",
  "lang": "deu",
  "label": 0
}

Label mapping:

label2idx = {
  'deu': 0,
  'zho': 1,
  'amh': 2,
  'hin': 3,
  'arb': 4,
  'hau': 5,
  'tur': 6,
  'urd': 7,
  'spa': 8,
  'fas': 9,
  'eng': 10,
  'nep': 11
}

Configurations

Configuration name: default

Each split is stored under data/:

data/
 ├── train-*
 ├── validation-*
 └── test-*

Citation

If you use this dataset in your work, please cite it as:

@dataset{8Opt,
  title={Multilingual Text Classification Dataset},
  author={8Opt},
  year={2025},
  url={https://huggingface.co/datasets/8Opt/multilingual-classification-0001}
}