hezar
Datasets
All datasets matching “hezar”sentiment-dksfThe Sentiment DKSF (Digikala/Snappfood comments) is a dataset for sentiment analysis.
persian-license-plate-v1
Dataset is downloaded from here which was provided at Amirkabir University of Technology.
The dataset is labeled by the authors.
Experimental results show that the fine-tuned model works well in Persian License Plate.
Usage
You can download the dataset easily using HF datasets package in Python:
!pip install datasets
from datasets import load_dataset
dataset = load_dataset("hezarai/persian-license-plate-v1", split="train") # Other splits: validation, test
print(dataset[0])
lscp-pos-500kThis is a 500 thousand sample version of the original LSCP dataset that only contains the text and part-of-speech tags and is used for sequence labeling.
Citation
@InProceedings{abdikhojasteh:2020:LREC,
author = {Abdi Khojasteh, Hadi and Ansari, Ebrahim and Bohlouli, Mahdi},
title = {LSCP: Enhanced Large Scale Colloquial Persian Language Understanding},
booktitle = {Proceedings of the Twelfth International Conference on Language Resources and Evaluation (LREC 2020)}… See the full description on the dataset page: https://huggingface.co/datasets/hezarai/lscp-pos-500k.flickr30k-faThe Flickr30K dataset filtered and translated to Persian.
This dataset was originally made by Sajjad Ayoubi and uploaded to Kaggle at https://www.kaggle.com/datasets/sajjadayobi360/flickrfa.
This repo contains the exact dataset split to train/test using a custom sampling criteria and can be directly loaded using HuggingFace datasets or right from Hezar.
Usage
Hugging Face Datasets
pip install datasets
from datasets import load_dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/hezarai/flickr30k-fa.xlsum-faThe Persian portion of the XLSum dataset.
Citation
@inproceedings{hasan-etal-2021-xl,
title = "{XL}-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages",
author = "Hasan, Tahmid and
Bhattacharjee, Abhik and
Islam, Md. Saiful and
Mubasshir, Kazi and
Li, Yuan-Fang and
Kang, Yong-Bin and
Rahman, M. Sohel and
Shahriyar, Rifat",
booktitle = "Findings of the Association for Computational Linguistics:… See the full description on the dataset page: https://huggingface.co/datasets/hezarai/xlsum-fa.common-voice-13-faThe Persian portion of the original CommonVoice 13 dataset at https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0
Load
# Using HF Datasets
from datasets import load_dataset
dataset = load_dataset("hezarai/common-voice-13-fa", split="train")
# Using Hezar
from hezar.data import Dataset
dataset = Dataset.load("hezarai/common-voice-13-fa", split="train")