datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
parallel-image-text-dataset-builder
parallel-image-text-dataset-builder (sample)
A small representative sample from the
parallel-image-text-dataset-builder
pipeline: it ingests image-text pairs, removes near-duplicates with
perceptual-hash (dhash) LSH-style bucketing, filters weak pairs by CLIP
image-text similarity, and writes fixed-size WebDataset-style tar shards.
Contents
shard-00002.tar - one WebDataset-style shard (536 samples). Each sample is
two members sharing a key: {key}.jpg (image) and… See the full description on the dataset page: https://huggingface.co/datasets/narinzar/parallel-image-text-dataset-builder.Chinese-English-Parallel-Translation-Corpus-Chinese-Source-Text-English-Translat
Chinese-English Parallel Translation Corpus (Chinese Source Text & English Translation)
A Chinese–English parallel corpus resource for translation and cross-lingual alignment applications, providing one-to-one bilingual text pairs: Chinese source texts aligned with their corresponding English translations. The data covers common writing styles and domains, making it suitable for parallel alignment, translation modeling, and cross-lingual representation learning.
It supports… See the full description on the dataset page: https://huggingface.co/datasets/shangzx/Chinese-English-Parallel-Translation-Corpus-Chinese-Source-Text-English-Translat.JParaCrawl-Filtered-English-Japanese-Parallel-Corpus-textkashmiri_parallel_Diacratic_to_Non_diacratic_Text_dataset
THE DATASET CAN BE USED AS:
1. 🎯 Diacritics Restoration Models
Train sequence-to-sequence models to automatically add diacritics to plain Kashmiri text.
Input: کاشر زبان (plain text)Output: کٲشُر زَبان (with diacritics)
Model architectures:
Transformer-based (BERT, T5, mT5)
LSTM/GRU sequence-to-sequence
Character-level neural networks
Training approach:
# Example: Fine-tuning mT5 for diacritization
from transformers import MT5ForConditionalGeneration
model =… See the full description on the dataset page: https://huggingface.co/datasets/Omarrran/kashmiri_parallel_Diacratic_to_Non_diacratic_Text_dataset.