datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
malaysian-youtube
Malaysian Youtube
Malaysian and Singaporean youtube channels, total up to 60k audio files with total 18.7k hours.
URLs data at https://github.com/mesolitica/malaya-speech/tree/master/data/youtube/data
Notebooks at https://github.com/mesolitica/malaya-speech/tree/master/data/youtube
How to load the data efficiently?
import pandas as pd
import json
from datasets import Audio
from torch.utils.data import DataLoader, Dataset
chunks = 30
sr = 16000
class Train(Dataset):… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/malaysian-youtube.mosaic-combine-all
Mosaic format for combine all dataset to train Malaysian LLM
This repository is to store dataset shards using mosaic format.
prepared at https://github.com/malaysia-ai/dedup-text-dataset/blob/main/pretrain-llm/combine-all.ipynb
using tokenizer https://huggingface.co/malaysia-ai/bpe-tokenizer
4096 context length.
how-to
git clone,
git lfs clone https://huggingface.co/datasets/malaysia-ai/mosaic-combine-all
load it,
from streaming import LocalDataset
import numpy… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/mosaic-combine-all.Malaysian-Emilia
Malaysian Emilia
An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Malaysian and Singaporean Speech Generation, where originally from Emilia.
We are improving Malaysian Emilia due to https://github.com/open-mmlab/Amphion/issues/436, check out mesolitica/Malaysian-Emilia-v2
Dataset
Clone and Extract
We upload as split zip files so you can clone and extract distributedly,
huggingface-cli download --repo-type dataset \
--include… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-Emilia.mosaic-starcoder-filtered
Mosaic format for filtered starcoder dataset to train Malaysian LLM
This repository is to store dataset shards using mosaic format.
prepared at https://github.com/malaysia-ai/dedup-text-dataset/blob/main/pretrain-llm/combine-starcoder.ipynb
using tokenizer https://huggingface.co/malaysia-ai/bpe-tokenizer
4096 context length.
how-to
git clone,
git lfs clone https://huggingface.co/datasets/malaysia-ai/mosaic-starcoder-filtered
load it,
from streaming import… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/mosaic-starcoder-filtered.crawl-my-websitemosaic-dedup-text-dataset
Mosaic format for dedup text dataset to train Malaysian LLM
This repository is to store dataset shards using mosaic format.
prepared at https://github.com/malaysia-ai/dedup-text-dataset/blob/main/pretrain-llm/combine-dedup-text-dataset-4096.ipynb
using tokenizer https://huggingface.co/malaysia-ai/bpe-tokenizer
4096 context length.
how-to
git clone,
git lfs clone https://huggingface.co/datasets/malaysia-ai/mosaic-dedup-text-dataset
load it,
from streaming import… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/mosaic-dedup-text-dataset.Malaysian-Emilia-annotated
Malaysian Emilia Annotated
Annotate Malaysian-Emilia using Data-Speech pipeline.
Malaysian Youtube
Originally from malaysia-ai/crawl-youtube
Total 3168.8 hours.
Gender prediction, filtered-24k_processed_24k_gender.zip
Language prediction, filtered-24k_processed_language.zip
Force alignment.
Post cleaned to 24k and 44k sampling rates,
24k, filtered-24k_processed_24k.zip
44k, filtered-24k_processed_44k.zip
Synthetic description… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-Emilia-annotated.Malaysian-STT-Whisper
Malaysian STT Whisper format
Heavy postprocessing and post-translation to improve pseudolabeled Whisper Large V3. Also include word level timestamp.
Postprocessing
Check repetitive trigrams.
Verify Voice Activity using Silero-VAD.
Verify scores using Force Alignment.
Post-translation
We use mesolitica/nanot5-base-malaysian-translation-v2.1.
Dataset involved
Malaysian context v2
Singaporean context
Indonesian context
Mandarin audio
Tamil audio… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-STT-Whisper.Malaysia-Textbook
Malaysia-Textbook-PDF
Gather Malaysia textbooks in PDF format, after that use Qwen/Qwen3-235B-A22B-Instruct-2507 to convert PDF image to text.
mosaic-dedup-text-dataset-filtered
Mosaic format for filtered dedup text dataset to train Malaysian LLM
This repository is to store dataset shards using mosaic format.
prepared at https://github.com/malaysia-ai/dedup-text-dataset/blob/main/pretrain-llm/combine-dedup-text-dataset-filtered-4096.ipynb
using tokenizer https://huggingface.co/malaysia-ai/bpe-tokenizer
4096 context length.
how-to
git clone,
git lfs clone https://huggingface.co/datasets/malaysia-ai/mosaic-dedup-text-dataset
load it… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/mosaic-dedup-text-dataset-filtered.malaysia-parliament-billsmosaic-madlad-400-ms
Mosaic format for extra dataset to train Malaysian LLM
This repository is to store dataset shards using mosaic format.
prepared at https://github.com/malaysia-ai/dedup-text-dataset/blob/main/pretrain-llm/combine-madlad-400-ms.ipynb
using tokenizer https://huggingface.co/malaysia-ai/bpe-tokenizer
4096 context length.
how-to
git clone,
git lfs clone https://huggingface.co/datasets/malaysia-ai/mosaic-madlad-400-ms
load it,
from streaming import LocalDataset
import… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/mosaic-madlad-400-ms.malaysian-dialects-youtube
Malaysian dialects Youtube
Entire videos from https://www.youtube.com using 'malay dialects' keyword.
With total 398634 audio files, total 68607.6 hours.
how to download
huggingface-cli download --repo-type dataset \
--include '*.z*' \
--local-dir './' \
malaysia-ai/malaysian-dialects-youtube
https://gist.githubusercontent.com/huseinzol05/2e26de4f3b29d99e993b349864ab6c10/raw/9b2251f3ff958770215d70c8d82d311f82791b78/unzip.py
python3 unzip.py
Source code
Source… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/malaysian-dialects-youtube.Malaysian-STT
Malaysian-STT
Prepare streaming and whole mode Speech-to-Text Malaysian context dataset, suitable to train streaming LLM base or Encoder-Decoder such as Whisper.
Merged 30 seconds chunk into one audio file, can up to 10 minutes.
Segmentize based on silent at least 0.3 seconds.
Reject low score based on force alignment.
Reject timestamp anomaly based on force alignment.
Dataset involved
Dialects
IMDA
Malaysian context
Malaysia Parliament
Science context… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/Malaysian-STT.pretrain-text-dataset
Dataset Introduction
This dataset is a collection of malaysian texts in the Malay, English, Chinese, and Tamil languages, gathered by Malaysia AI volunteers through web crawling of malaysian websites.
The dataset amounts to approximately 250 GB of text data, and has undergone deduplication process.
Project Link
To learn more about this project, https://github.com/users/huseinzol05/projects/1/views/1
We no longer update the project.
Github Repository
Our data… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/pretrain-text-dataset.pseudolabel-malaysian-youtube-whisper-large-v3
Pseudolabel Malaysian Youtube videos using Whisper Large V3
Original dataset at https://huggingface.co/datasets/malaysia-ai/crawl-youtube, distributed pseudolabelled using 4x A100s
script at https://github.com/mesolitica/malaysian-dataset/tree/master/speech-to-text-semisupervised/pseudolabel-whisper
Each audio is 30 seconds.
Each audio saved in 16k sample rate.
Malaysian-SFT
Malaysian SFT
It contains dataset from the following sets:
mesolitica/chatgpt4-malaysian-general-qa
mesolitica/llama3-70b-social-media-qa
mesolitica/llama3-70b-qa
mesolitica/llama3-70b-non-bumi-qa
mesolitica/peribahasa-instructions
mesolitica/google-translate-camel-ai
mesolitica/synthetic-jawi-conversation
mesolitica/jawi-code-instructions
mesolitica/chatgpt4-code-instruct
mesolitica/malaysian-ultrachat
mesolitica/malay-dialect-instructions… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-SFT.Malaysian-TTS-v2
Malaysian TTS v2
Generate Malay and localize English for TTS dataset, currently only support 2 speakers, husein and idayu, where total audio is 4642.77 hours.
How to prepare the dataset
huggingface-cli download \
mesolitica/Malaysian-TTS-v2 \
--include "all-*.zip" \
--repo-type "dataset" \
--local-dir './'
huggingface-cli download \
mesolitica/STT-Normalizer \
--include "*husein*.zip" \
--exclude "*force*" \
--repo-type "dataset" \
--local-dir './'… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-TTS-v2.Malaysian-TTS
TTS
Malaysian Synthetic TTS dataset.
Generate using each Malaysian-F5-TTS-v2.
Each generation verified using esammahdi/ctc-forced-aligner.
Post-filter pitch using interactiveaudiolab/penn.
Speaker
Husein, 300 hours.
Shafiqah Idayu, 292 hours.
Anwar Ibrahim, 269 hours.
KP RTM Suhaimi Malay, 306 hours.
KP RTM Suhaimi Chinese, 192 hours.
Clean version
We trimmed start and end silents, and compressed at processed
Dataset uploaded as HuggingFace datasets… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-TTS.Multilingual-TTS
Multilingual-TTS
A large multilingual corpus for pretraining TTS/STT models, gathered and normalized from 230+ public sources. ~191k hours of audio across 150+ languages, organized into 1,544 dataset configs and tokenized to 34.5B NeuCodec speech tokens (50 Hz, single-codebook) over 111.1M clips.
Each config is one source dataset normalized to rows of {audio_filename, text, speaker}:
audio_filename — clip path inside that config's <config>_audio.zip (mono MP3).
text —… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/Multilingual-TTS.pseudolabel-malaysian-youtube-whisper-large-v3-timestamp
Pseudolabel Malaysian Youtube using Whisper Large V3 including Timestamp
how to prepare the dataset
wget https://huggingface.co/datasets/mesolitica/pseudolabel-malaysian-youtube-whisper-large-v3-timestamp/resolve/main/prepared-pseudolabel.jsonl
huggingface-cli download --repo-type dataset \
--include 'output-audio-*.zip' \
--local-dir './' \
--max-workers 20 \
mesolitica/pseudolabel-malaysian-youtube-whisper-large-v3-timestamp
wget… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/pseudolabel-malaysian-youtube-whisper-large-v3-timestamp.google-image-malaysian-vehicle-dedup
Google Image Malaysian Vehicle Dedup
Original dataset https://huggingface.co/datasets/malaysia-ai/crawl-google-image-malaysian-vehicle
Source code at https://github.com/mesolitica/malaysian-dataset/tree/master/vlm/dedup-malaysian-vehicle
Dedup 70% similar
dedup-0.7.jsonl, total deduped 97598 images,
{'filename': 'train-00075-of-00165-c0ebcc169b1f62d2.parquet',
'keyword': '2021 Honda City 1.5 E',
'no': 2,
'selected_indices': [696,
702,
705,
707,
712,
716… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/google-image-malaysian-vehicle-dedup.pseudolabel-dialects-youtube-whisper-large-v3
malaysia-ai/pseudolabel-dialects-youtube-whisper-large-v3
Pseudolabel malaysia-ai/malaysian-dialects-youtube using openai/whisper-large-v3
How to prepare the dataset
huggingface-cli download --repo-type dataset \
--include '*.zip' \
--local-dir './' \
--max-workers 20 \
malaysia-ai/pseudolabel-dialects-youtube-whisper-large-v3
wget https://gist.githubusercontent.com/huseinzol05/2e26de4f3b29d99e993b349864ab6c10/raw/9b2251f3ff958770215d70c8d82d311f82791b78/unzip.py
python3… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/pseudolabel-dialects-youtube-whisper-large-v3.Multilingual-TTS-language
Multilingual-TTS-language
malaysia-ai/Multilingual-TTS
with two extra columns:
column
description
audio_filename, text, speaker
unchanged from malaysia-ai/Multilingual-TTS
language
language detected from the text (transcript) column of every row
post-normalized
text after rule-based punctuation / capitalization normalization
All original columns and the file/folder layout are preserved: 1552 subsets /
1561 parquet files, 121,842,371 rows (August 2026). Every… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/Multilingual-TTS-language.fineweb-filter-malaysian-context
HuggingFaceFW/fineweb filter Malaysian context
What is it?
We filter the original 🍷 FineWeb dataset that consists more than 15T tokens on simple Malaysian keywords.
Total tokens for the filtered dataset is 174102784199 tokens, 174B tokens.
How we do it?
We filter rows using {'malay', 'malaysia', 'melayu', 'bursa', 'ringgit'} keywords on r5.16xlarge EC2 instance for 7 days.
We calculate total tokens using tiktoken.encoding_for_model("gpt2") on c7a.24xlarge EC2… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/fineweb-filter-malaysian-context.malaysian-cartoons-youtube
Crawl Youtube Malaysian cartoons
With total 45544 audio files, total 20766 hours.
how to download
huggingface-cli download --repo-type dataset \
--include '*.zip' \
--local-dir './' \
malaysia-ai/malaysian-cartoons-youtube
wget https://gist.githubusercontent.com/huseinzol05/2e26de4f3b29d99e993b349864ab6c10/raw/9b2251f3ff958770215d70c8d82d311f82791b78/unzip.py
python3 unzip.py
Source code
Source code at… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/malaysian-cartoons-youtube.Malaysian-Emilia
Malaysian Emilia
Gather Malaysian Emilia from,
https://huggingface.co/datasets/mesolitica/Malaysian-Emilia-v2
https://huggingface.co/datasets/Scicom-intl/Malaysian-Chinese-Emilia
https://huggingface.co/datasets/mesolitica/Malaysian-Emilia#malaysian-dialect
And do,
Trim silent.
Permutation for Voice Conversion include post-filtering during permutation.
Convert to Neucodec speech tokens.
Malaysian-Chinese-Emilia
Malaysian-Chinese-Emilia
Use https://github.com/mesolitica/Emilia to pseudo-label Malaysian Chinese audio.
Total rows: 605169
Total hours: 1857.611445057867 hours
Permutation for Voice Conversion
Also we already calculated speaker permutation to prepare for voice conversion.
common_voice_22_0
Common Voice Corpus 22.0
Originally from https://huggingface.co/datasets/fsicoli/common_voice_22_0, we mirror using multiple zip files also trimmed the silents.
How to prepare the dataset
huggingface-cli download --repo-type dataset \
--include '*.zip' \
--local-dir './' \
--max-workers 20 \
malaysia-ai/common_voice_22_0
wget https://gist.githubusercontent.com/huseinzol05/2e26de4f3b29d99e993b349864ab6c10/raw/9b2251f3ff958770215d70c8d82d311f82791b78/unzip.py
python3… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/common_voice_22_0.mosaic-instructions
Mosaic format for instructions dataset to train Malaysian LLM
This repository is to store dataset shards using mosaic format.
prepared at https://github.com/malaysia-ai/dedup-text-dataset/blob/main/pretrain-llm/combine-instructions.ipynb
using tokenizer https://huggingface.co/malaysia-ai/bpe-tokenizer
4096 context length.
how-to
git clone,
git lfs clone https://huggingface.co/datasets/malaysia-ai/mosaic-instructions
load it,
from streaming import LocalDataset… See the full description on the dataset page: https://huggingface.co/datasets/malaysia-ai/mosaic-instructions.