Pangeanic/Cantonese-English-Machine-Translation-Corpus-text

PangeanicYueEn - Cantonese English Parallel Corpus

PangeanicYueEn is a large-scale Cantonese-English parallel corpus designed for machine translation, multilingual large language model (LLM) training, cross-lingual NLP research, retrieval-augmented generation (RAG), bilingual embeddings, instruction tuning, and multilingual AI systems.

This release contains 150,000 Cantonese-English sentence pairs sampled from a larger corpus of approximately 8.05 million parallel sentence pairs. For the complete dataset, please contact Pangeanic

The dataset focuses specifically on written Cantonese (Yue Chinese) paired with English translations, providing valuable training data for one of the most underrepresented Chinese language varieties in modern AI systems.

Why Cantonese Matters for AI

Most multilingual AI systems are heavily trained on:

• —English
• —Mandarin Chinese
• —Spanish
• —French
• —German

However, Cantonese (Yue Chinese) remains significantly underrepresented despite being spoken by tens of millions of people worldwide across:

• —Hong Kong
• —Macau
• —Guangdong
• —Southeast Asia
• —North America
• —Europe

As demand grows for multilingual AI, conversational assistants, translation systems, and regionally adapted LLMs, high-quality Cantonese datasets become increasingly important.

PangeanicYueEn helps address this gap by providing large-scale Cantonese-English parallel text suitable for modern AI development.

Dataset Overview

Dataset Structure

The dataset contains two columns:

Example:

Example Usage

from datasets import load_dataset

dataset = load_dataset("Pangeanic/Cantonese-English-Machine-Translation-Corpus-text")

print(dataset["train"][0])

AI and LLM Use Cases

This dataset is suitable for:

• —Cantonese machine translation
• —English-Cantonese translation models
• —multilingual LLM training
• —continued pretraining
• —supervised fine-tuning (SFT)
• —retrieval-augmented generation (RAG)
• —bilingual embedding models
• —cross-lingual retrieval
• —multilingual search systems
• —conversational AI
• —chatbot development
• —language model evaluation
• —synthetic data generation
• —multilingual instruction tuning
• —low-resource language research

Why This Dataset Is Useful

Large-Scale Cantonese Coverage

Cantonese remains one of the most underserved major languages in open AI training datasets.

This corpus provides substantial Cantonese-language coverage for researchers and developers working on multilingual AI systems.

Parallel Data for Translation

Aligned sentence pairs make the dataset suitable for:

• —neural machine translation
• —sequence-to-sequence models
• —encoder-decoder architectures
• —multilingual transformer training

LLM Adaptation

The dataset can be used for:

• —multilingual LLM adaptation
• —Cantonese language enhancement
• —bilingual alignment
• —continued pretraining
• —instruction tuning

Cross-Lingual Applications

Developers can use PangeanicYueEn for:

• —multilingual search
• —semantic retrieval
• —bilingual embeddings
• —knowledge transfer
• —multilingual RAG systems

Recommended Applications

Machine Translation

Train Cantonese ↔ English translation systems.

Multilingual Large Language Models

Improve Cantonese understanding and generation capabilities.

Conversational AI

Build Cantonese-capable assistants and chatbots.

Retrieval-Augmented Generation (RAG)

Enhance retrieval performance for Cantonese and English content.

Cross-Lingual Retrieval

Develop semantic search systems that operate across languages.

Dataset Characteristics

• —Cantonese (Yue Chinese)
• —Traditional Chinese script
• —English translations
• —Sentence-level alignment
• —Large-scale bilingual data
• —AI-ready format
• —Suitable for modern NLP pipelines

Intended Research Areas

This dataset supports research in:

• —machine translation
• —multilingual NLP
• —low-resource languages
• —bilingual embeddings
• —cross-lingual transfer learning
• —language model adaptation
• —multilingual retrieval
• —multilingual RAG
• —conversational AI
• —Cantonese language technologies

Limitations

• —This release is a sample of a larger corpus.
• —The dataset does not include document-level metadata.
• —The dataset is provided at sentence level.
• —Domain distribution may vary across the corpus.

Users should evaluate suitability for their specific use cases and conduct additional filtering if required.

Citation

@dataset{pangeanic_cantonese_english_mt_2026,
  author = {{Pangeanic}},
  title = {Cantonese-English Machine Translation Corpus},
  year = {2026},
  publisher = {Pangeanic},
  url = {https://huggingface.co/datasets/Pangeanic/Cantonese-English-Machine-Translation-Corpus-text}
}

Keywords

Cantonese dataset, Yue dataset, Cantonese English corpus, Cantonese machine translation, bilingual dataset, parallel corpus, multilingual NLP, multilingual LLM, Cantonese AI, Chinese English translation, Cantonese chatbot dataset, Cantonese language model, machine translation corpus, bilingual AI training data, retrieval augmented generation, multilingual embeddings, low-resource language dataset, cross-lingual NLP, Cantonese NLP, Yue Chinese dataset.