multilingual
multilingual-speech-commands-15lang
Multilingual Speech Commands Dataset (15 Languages, Augmented)
This dataset contains augmented speech command samples in 15 languages, derived from multiple public datasets. Only commands that overlap with the Google Speech Commands (GSC) vocabulary are included, making the dataset suitable for multilingual keyword spotting tasks aligned with GSC-style classification.
Audio samples have been augmented using standard audio techniques to improve model robustness (e.g., time-shifting… See the full description on the dataset page: https://huggingface.co/datasets/artur-muratov/multilingual-speech-commands-15lang.SWE-bench_Multilingual
SWE-bench Multilingual
Dataset Summary
SWE-bench Multilingual is a dataset that tests systems' ability to resolve real-world GitHub issues across a broad range of programming languages. The original SWE-bench is Python-only; this dataset extends the same task format to 9 languages drawn from 41 popular repositories.
The dataset collects 300 test Issue-Pull Request pairs. Evaluation is performed by unit test verification, using post-PR behavior as the reference solution.
The… See the full description on the dataset page: https://huggingface.co/datasets/SWE-bench/SWE-bench_Multilingual.MultilingualMultiModalClassification
Additional Information
To load the dataset,
import datasets
ds = datasets.load_dataset("AmazonScience/MultilingualMultiModalClassification", data_dir="wiki-doc-ar-merged")
print(ds)
DatasetDict({
train: Dataset({
features: ['image', 'filename', 'words', 'ocr_bboxes', 'label'],
num_rows: 8129
})
validation: Dataset({
features: ['image', 'filename', 'words', 'ocr_bboxes', 'label'],
num_rows: 1742
})
test: Dataset({
features:… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/MultilingualMultiModalClassification.multilingual_librispeech
Dataset Card for MultiLingual LibriSpeech
Dataset Summary
This is a streamable version of the Multilingual LibriSpeech (MLS) dataset.
The data archives were restructured from the original ones from OpenSLR to make it easier to stream.
MLS dataset is a large multilingual corpus suitable for speech research. The dataset is derived from read audiobooks from LibriVox and consists of
8 languages - English, German, Dutch, Spanish, French, Italian, Portuguese, Polish.… See the full description on the dataset page: https://huggingface.co/datasets/facebook/multilingual_librispeech.OCR-Synthetic-Multilingual-v1
OCR-Synthetic-Multilingual-v1
Overview
Large-scale synthetically generated OCR training dataset for multilingual text detection and recognition. The data was produced using a heavily modified and extended version of SynthDoG (Synthetic Document Generator), originally introduced in the Donut project by Kim et al.
This dataset was used to train Nemotron OCR v2, a state-of-the-art multilingual OCR model that is part of the NVIDIA NeMo Retriever collection.
Languages… See the full description on the dataset page: https://huggingface.co/datasets/Nishant2414/OCR-Synthetic-Multilingual-v1.OCR-Synthetic-Multilingual-v1
OCR-Synthetic-Multilingual-v1
Overview
Large-scale synthetically generated OCR training dataset for multilingual text detection and recognition. The data was produced using a heavily modified and extended version of SynthDoG (Synthetic Document Generator), originally introduced in the Donut project by Kim et al.
This dataset was used to train Nemotron OCR v2, a state-of-the-art multilingual OCR model that is part of the NVIDIA NeMo Retriever collection.
Languages… See the full description on the dataset page: https://huggingface.co/datasets/Arturito1/OCR-Synthetic-Multilingual-v1.