Msoldier-ai/parsbert-qa

🔗 Links & Resources

• —GitHub Repository: https://github.com/MmdDevAi/qa-parsbert
• —Hugging Face Model Card: https://huggingface.co/Msoldier-ai/parsbert-qa
• —Interactive Web Demo (Hugging Face Spaces): https://huggingface.co/spaces/Msoldier-ai/parsbert-qa-and-samsum

📚 Intelligent Document Question Answering & RAG System

An end-to-end Question Answering (QA) and Retrieval-Augmented Generation (RAG) system designed for accurate, context-aware information extraction from complex documents. This repository covers both extractive QA architectures (DistilBERT, RoBERTa) and modern RAG pipelines using LangChain.

🌟 Highlights & Key Features

• —Extractive QA Baseline: Fast inference and high token-level precision using fine-tuned DistilBERT and RoBERTa models.
• —Retrieval-Augmented Generation (RAG): Integrated retrieval pipeline powered by LangChain for contextual grounding and hallucination reduction.
• —Document Ingestion & Chunking: Efficient parsing and semantic chunking for dense text and technical documentation.
• —Evaluation Framework: Benchmarked against standard QA metrics (Exact Match and F1-Score).

🏗️ Architecture & Model Progression

1. 1.Extractive Pipeline (DistilBERT / RoBERTa):
2. 2.Identifies the start and end logits directly from the source passage.
3. 3.Ideal for low-latency, strictly factual question answering where answers are verbatim in the context.

1. 1.RAG Pipeline (LangChain Integration):
2. 2.Uses vector embeddings and similarity search to retrieve relevant document chunks dynamically.
3. 3.Synthesizes answers over large multi-page documents without exceeding model context limits.

📊 Quantitative Evaluation & Results

The system was evaluated on the PQuAD (Persian Question Answering Dataset) benchmark using standard reading comprehension metrics (Exact Match and F1-Score).

Benchmark Performance

Detailed Breakdown (HasAns vs. NoAns)

To evaluate real-world robustness against unanswerable or ambiguous queries, performance is broken down by answer availability:

Error Categorization Analysis

Error patterns on the validation set fall into three primary failure modes:

1. 1.Span Boundary Mismatch: The model locates the correct sentence but includes extraneous context tokens or truncates dependent clauses.
2. 2.False Negatives: The model predicts no answer when a subtle or implicit answer exists within the text.
3. 3.False Positives: The model predicts an entity or phrase from the context when the question actually lacks sufficient evidence to be answered.