hishab
Datasets
All datasets matching “hishab”titulm-bangla-corpus
TituLM Bangla Corpus
This dataset is associated with the paper TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking
TituLM Bangla Corpus is one of the largest Bangla clean corpus prepared for pretraining, continual pretraining or fine-tuning Large Language Model(LLM) for improving Bangla text generation capability.
This dataset contains diverse sources and categories of Bangla text. The largest part of this dataset contains filtered common crawled datasets. As we saw… See the full description on the dataset page: https://huggingface.co/datasets/hishab/titulm-bangla-corpus.titulm-bangla-mmlu
Titulm Bangla MMLU
Read the paper for details: https://arxiv.org/abs/2502.11187
Citation
@misc{nahin2025titullmsfamilybanglallms,
title={TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking},
author={Shahriar Kabir Nahin and Rabindra Nath Nandi and Sagor Sarker and Quazi Sarwar Muhtaseem and Md Kowsher and Apu Chandraw Shill and Md Ibrahim and Mehadi Hasan Menon and Tareq Al Muntasir and Firoj Alam},
year={2025},
eprint={2502.11187}… See the full description on the dataset page: https://huggingface.co/datasets/hishab/titulm-bangla-mmlu.bangla-mmlu
Data Summary
We curated multiple-choice questions from various open-source educational websites and textbooks, inspired by the original MMLU dataset (Hendrycks et al., 2020). The dataset includes multiple-choice questions from different Bangladeshi exams, such as job exams, the Bangladesh Civil Service Exam, and undergraduate admission exams. In Figure 7, we report category wise distributions.
Bangla MMLU Dataset Overview
The Bangla MMLU dataset consists of a total of 116,503… See the full description on the dataset page: https://huggingface.co/datasets/hishab/bangla-mmlu.commonsenseqa-bn
Dataset Summary
This is the Bangla translated version of the CommonsenseQA dataset. The dataset was translated using a new method called Expressive Semantic Translation (EST). This method combines both Google Machine Translation and LLM-based rewriting of the translation to enhance the expressiveness and semantic accuracy of the translated content.
Dataset Structure
Data instances
Defaults
An example of a 'train' looks as follows:
{… See the full description on the dataset page: https://huggingface.co/datasets/hishab/commonsenseqa-bn.openbookqa-bn
Data Summary
This is the Bangla-translated version of the OpenBookQA dataset. The dataset was translated using a new method called Expressive Semantic Translation (EST), which combines Google Machine Translation with LLM-based rewriting modifications. This method enhances the semantic accuracy and expressiveness of the translated content. OpenBookQA focuses on advanced question-answering, requiring multi-step reasoning, additional common and commonsense knowledge, and rich text… See the full description on the dataset page: https://huggingface.co/datasets/hishab/openbookqa-bn.piqa-bn
Dataset Summary
This is the translated version of the PIQA LLM evaluation dataset. The dataset was translated using a new method called Expressive Semantic Translation (EST), which combines Google Translation with LLM-based rewriting. PIQA introduces the task of physical commonsense reasoning and provides a corresponding benchmark for understanding physical interactions in everyday situations. It focuses on atypical solutions to practical problems, inspired by instructional guides… See the full description on the dataset page: https://huggingface.co/datasets/hishab/piqa-bn.