amir1381sa/persian-paraphrase-bert

Persian Paraphrase Identification with ParsBERT

A Persian Natural Language Processing model for identifying whether two Persian sentences have the same meaning (paraphrase) or not.

Task

This model performs binary sequence classification on pairs of Persian texts.

• —Class 0: Non-Paraphrase
• —Class 1: Paraphrase

The model predicts whether the two sentences convey the same meaning.

Base Model

The base pretrained model used in this project is:

HooshvareLab/bert-base-parsbert-uncased

The model was fine-tuned for Persian paraphrase identification using a binary classification head.

Dataset

The model was trained and evaluated on the PARSINLU QQP dataset.

Dataset source:

https://github.com/persiannlp/parsinlu

The dataset contains pairs of Persian questions/sentences with paraphrase labels.

Model Architecture

Base architecture: BERT / ParsBERT

Task: Sequence Classification

Number of classes: 2

Labels

Fine-Tuning Strategy

Part of the pretrained BERT model was frozen during fine-tuning.

Frozen

• —Embedding layer
• —Encoder layers 0–7

Trainable

• —Encoder layers 8–11
• —Classification head

This strategy allows the upper transformer layers and classification head to adapt to the paraphrase identification task while keeping the lower-level pretrained representations fixed.

Training Configuration

Evaluation Metrics

The following metrics were used:

• —Accuracy
• —Precision
• —Recall
• —F1 Score

Test Results

Parameter Statistics

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "amir1381sa/persian-paraphrase-bert"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text1 = "امروز هوا خیلی خوب است."
text2 = "امروز وضعیت آب و هوا بسیار مناسب است."

inputs = tokenizer(
    text1,
    text2,
    return_tensors="pt",
    truncation=True,
    max_length=128
)

with torch.no_grad():
    outputs = model(**inputs)

prediction = torch.argmax(outputs.logits, dim=-1).item()

print(model.config.id2label[prediction])

Output

The model returns one of the following labels:

Non-Paraphrase
Paraphrase

Repository Contents

This repository contains:

• —Fine-tuned ParsBERT model
• —Tokenizer
• —Model configuration
• —Test evaluation results
• —Training logs
• —Experiment configuration

Project Summary

This project demonstrates fine-tuning of a pretrained Persian BERT model for the task of Persian Text Paraphrase Identification.

The pretrained model was adapted to a binary classification task using the PARSINLU QQP dataset.

Reproducibility

Experiments were performed using:

• —Python
• —PyTorch
• —Hugging Face Transformers
• —Hugging Face Datasets
• —Scikit-learn

Random seed:

42