Datasetpublic

zhaosuifeng/FinRAGBench-V

FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain 🤗 Code 📄 Paper Overview FinRAGBench-V is a comprehensive benchmark for visual retrieval-augmented generation (RAG) in finance, addressing the challenge that most existing financial RAG research focuses predominantly on text while overlooking rich visual content in financial documents. By integrating multimodal data and providing visual citation, FinRAGBench-V ensures… See the full description on the dataset page: https://huggingface.co/datasets/zhaosuifeng/FinRAGBench-V.

sourceHugging Faceapache-2.0updated 11mo agoView on Hugging Face

12likes1kdownloads

Dataset Card

<h1 align="center">FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain</h1> 🤗 <a href="https://github.com/zhaosuifeng/FinRAGBench-V/tree/main">Code</a> 📄 <a href="https://arxiv.org/abs/2505.17471">Paper</a>

Overview

FinRAGBench-V is a comprehensive benchmark for visual retrieval-augmented generation (RAG) in finance, addressing the challenge that most existing financial RAG research focuses predominantly on text while overlooking rich visual content in financial documents. By integrating multimodal data and providing visual citation, FinRAGBench-V ensures traceability and supports robust evaluation of Multimodal Large Language Models (MLLMs).

📂 Dataset Structure

The dataset is organized into the following components:

—`pdf_for_QA/` Contains the collected PDF documents (Chinese and English), serving as raw source files.

—`corpus/` Each PDF was split page by page into images. These page-level images form the retrieval corpus.

—`queries/` QA dataset in JSON format. Each entry includes:
—`query-id`: Unique identifier, based on PDF name + page(s) + suffix. (You can ignore the format; official `qrels` provide mapping to `corpus-id`.)
—`query`: Natural language question.
—`answer`: Ground-truth answer.
—`category`: Task category (see Task Categories).
—`answer_type`: Either long or short.
—`from_pages`: Source page indices in the original PDF.

—`qrels/` Relevance judgments (.tsv files) mapping between query-id and corpus-id for retrieval evaluation.

—`citation_labels/` The 200 sampled images with human annotated citations in relative coordinates.

🏷️ Task Categories

The dataset covers diverse financial multimodal reasoning tasks, including:

Task Category	Description	Example
Text Inference	Information extraction and summarization from text	Deriving insights or identifying financial data
Chart-Information Extraction	Extracting metrics from charts	Percentage of a sector in a pie chart
Chart-Numerical Calculation	Numerical reasoning over charts	Computing interest rate changes
Chart Time-Sensitive	Time-based analysis of charts	Identifying events, trends, peaks, or troughs
Table-Numerical Calculation	Quantitative reasoning over table data	Summing costs, computing changes
Table-Compare and Sort	Ranking and comparison tasks	Highest/lowest values across entities
Multipage	Cross-page reasoning (Chart-Multipage, Table-Multipage, Text-Multipage)	truncated tables, multi-chart synthesis

⚡ Key Features

—Multilingual (Chinese & English) financial domain data.
—Multimodal (text, tables, charts).
—Large retrieval corpus with QA pairs.
—Fine-grained task categories for multimodal reasoning evaluation.

📑 Citation

If you use this dataset, please cite:

bibtex

@article{zhao2025finragbench,
  title={FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain},
  author={Zhao, Suifeng and Jin, Zhuoran and Li, Sujian and Gao, Jun},
  journal={arXiv preprint arXiv:2505.17471},
  year={2025}
}