NLPclass/mt5-title-generation

1---2library_name: transformers3license: mit4language:5- fa6tags:7- title-generation8- nlp9- transformers10- persian11- farsi12- text-generation13- mt514pipeline_tag: text-generation15---16 17# Title Generation for Persian using Transformers18 19## Model Details20 21**Model Description:**22This model is a fine-tuned version of `mt5-small` on a custom Persian dataset for the task of title generation. The model was trained for 4 epochs on a dataset containing 25,000 rows of Persian text, using an NVIDIA P100 GPU. It is designed to generate titles for Persian text, making it useful for applications such as summarizing articles, generating headlines, and creating titles for various text inputs.23 24**Intended Use:**25The model is intended for generating titles for Persian text. It can be used in applications such as summarizing articles, generating headlines, or creating titles for various text inputs.26 27**Model Architecture:**28- **Model Type:** Transformers-based text generation29- **Language:** Persian (fa)30- **Base Model:** `mt5-small`31 32## Training Data33 34**Dataset:**35The model was fine-tuned on a custom Persian dataset specifically curated for the task of title generation. The dataset includes 25,000 rows of Persian texts along with their corresponding titles.36 37**Data Preprocessing:**38- Text normalization and cleaning were performed to ensure consistency.39- Tokenization was done using the mT5 tokenizer.40 41## Training Procedure42 43**Training Configuration:**44- **Number of Epochs:** 445- **Batch Size:** 846- **Learning Rate:** 1e-547- **Optimizer:** AdamW48 49**Training Environment:**50- **Hardware:** NVIDIA P100 GPU51- **Training Time:** Approximately 4 hours52 53## How To Use54 55You can use this model with the `transformers` library as follows:56 57```python58from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline59 60# Load the tokenizer and model61tokenizer = AutoTokenizer.from_pretrained("NLPclass/mt5-title-generation")62model = AutoModelForSeq2SeqLM.from_pretrained("NLPclass/mt5-title-generation")63 64# Example text in Persian65input_text = "به گزارش ایمنا، در دیدار سوپر جام فوتبال روسیه زنیت سن‌پترزبورگ قهرمان رقابتهای لیگ و جام حذفی این کشور در حضور عده‌ای معدود از تماشاگران به دیدار لوکوموتیو مسکو نایب قهرمان لیگ روسیه رفت"66inputs = tokenizer(input_text, return_tensors="pt", max_length=512, truncation=True)67outputs = model.generate(inputs.input_ids, max_length=50, num_beams=5, early_stopping=True)68 69# Decode the generated title70generated_title = tokenizer.decode(outputs[0], skip_special_tokens=True)71print(generated_title)72 73 74# Create a text generation pipeline75title_generation_pipeline = pipeline("text-generation", model="NLPclass/mt5-title-generation")76generated_title = title_generation_pipeline(input_text, max_length=50, num_beams=5, early_stopping=True)77print(generated_title)78 79```80```bibtex81@misc{NLPclass,82  author = {NLPclass},83  title = {Title Generation for Persian using Transformers},84  year = {2024},85  publisher = {Hugging Face},86  howpublished = {\url{https://huggingface.co/NLPclass/mt5-title-generation}},87}88```