datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Malay-Dialect-Instructions
Malay dialect instruction including coding
Negeri Sembilan
QA
public transport QA,
Coding
CUDA coding,
Kedah
QA
infra QA,
Coding
Rust coding,
Kelantan
QA
Najib Razak QA,
Coding
Go coding,
Perak
QA
Anwar Ibrahim QA,
Coding
SQL coding,
Pahang
QA
Pendatang asing QA,
Coding
Typescript coding,
Terengganu… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malay-Dialect-Instructions.Malaysian-Ultrachat
Ultrachat like using Malaysian context
Prepare multiturn dialogue between user and assistant for malaysian context,
Astroawani, https://huggingface.co/datasets/malaysia-ai/crawl-astroawani, ultrachat-astroawani-malay.jsonl, 60198 rows, 477 MB.
Crossref melayu papers, https://huggingface.co/datasets/mesolitica/crawl-my-website/resolve/main/melayu-pdf.jsonl, ultrachat-crossref-melayu-malay.jsonl, 9959 rows, 187 MB
Epenerbitan… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Malaysian-Ultrachat.outmind-dataset
OutMind Unified Dataset (Mesosfer-AI)
The official high-quality, multimodal-balanced training dataset curated and cleaned for OutMind, an educational and hackable Small Language Model (SLM) framework developed by Mesosfer-AI.
🌟 Dataset Overview
OutMind dataset is engineered to provide a balanced foundation across Indonesian, English, Chinese, Mathematical Reasoning, and Python Code, organized into three strictly separated tiers:
outmind-dataset/
├── pretrain/… See the full description on the dataset page: https://huggingface.co/datasets/Mesosfer/outmind-dataset.synthetic-jawi-conversation
Synthetic Jawi Conversation
Originally from https://huggingface.co/datasets/mesolitica/mixtral-malaysian-general-qa, we just convert it into Jawi.
Notebooks at https://github.com/mesolitica/malaysian-dataset/tree/master/chatbot/jawi-conversation
synthetic-jawi-conversation.jsonl, 509375 rows, 13 GB.
Example data
[{'role': 'context',
'content': 'din Venue: DK7, Block J, 3rd Floor, Faculty of Medicine 14 21.06.2023 9.00am – 12.00pm (SDL): to turn in Reflection… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/synthetic-jawi-conversation.