Fatema142/BB-FinQA-X

BB-FinQA-X

BB-FinQA-X is a 500-item, expert-grounded question-answering benchmark built from the Bangladesh Bank Annual Report, FY2024–25, the central bank of Bangladesh's official yearly report on macroeconomic conditions, monetary policy, banking-sector supervision, and financial markets.

Every question is paired with a literal, page-cited evidence quote from the source report, drawn from narrative text, statistical tables, and charts alike. This makes the dataset suitable for evaluating text-only, table-QA, chart-QA, and multimodal financial reasoning systems.

Source material

Bangladesh Bank publishes its Annual Report as public information; this dataset's questions and evidence are derived from the report's nine substantive chapters, listed below with their scope and item counts.

Official source: Bangladesh Bank Annual Report 2024–2025 (PDF), published by Bangladesh Bank under its Annual Report publications section.

Data fields

Each item is a flat JSON object with the following fields:

Example

{
  "id": "chapter_5-q11",
  "number": 11,
  "chapter": "Chapter 5",
  "chapter_title": "Banking Sector Performance, Regulation and Bank Supervision",
  "question": "Compare the composition of aggregate industry assets between June 2024 and March 2025 in terms of the loans and advances share and the government bills and bonds share.",
  "answer": "The loans and advances share was essentially unchanged — 63.6 percent (BDT 17,112.67 billion) in June 2024 versus 63.5 percent (BDT 17,114.88 billion, or 63.51 percent per paragraph 5.06) in March 2025 — and the government bills and bonds share held at 17.7 percent (BDT 4,749.23 billion in June 2024 and BDT 4,763.46 billion in March 2025).",
  "evidence": "Para 5.06: \"The aggregate banking sector assets consisted of BDT 17,114.88 billion in loans and advances (63.51 percent of total assets) ... BDT 4,763.46 billion as investments in government bills and bonds (treasury securities) ... (Chart 5.01).\" Chart 5.01: June 2024 — Loans & Advances 17112.67 (63.6%), Govt. bills & bond 4749.23 (17.7%); March 2025 — Loans & Advances 17114.88 (63.5%), Govt. bills & bond 4763.46 (17.7%).",
  "query_type": "Comparison",
  "presentation_format": "Text + Chart",
  "difficulty": "Medium",
  "source_page": "33-34"
}

Two independent taxonomies

BB-FinQA-X labels every item along two orthogonal axes. They answer different questions about each item and should not be conflated:

• —presentation_format — how the supporting evidence is presented on the page (its modality)
• —query_type — what kind of reasoning the question demands of a system

A single item always has exactly one value from each axis — e.g. an item can be presentation_format: "Table Only" and query_type: "Numerical Calculation" at the same time, because the two labels describe different things about it.

Taxonomy 1 — presentation_format (data handling format)

Describes the modality of the evidence a system must read and combine to answer the question.

This axis is what makes the dataset useful for benchmarking document/table/chart QA systems specifically: a model that only handles plain text will be unable to answer the 250 items whose presentation_format includes a table or chart.

Taxonomy 2 — query_type (data category)

Describes the reasoning operation the question requires, independent of where the evidence lives.

This axis is what makes the dataset useful for benchmarking reasoning depth, independent of modality — e.g. comparing how a model's accuracy on Fact Extraction degrades on Multi-hop Reasoning items that require chaining evidence across sources.

Difficulty (a third, coarser label)

difficulty (Easy / Medium / Hard) is a holistic difficulty rating assigned per item and is correlated with, but not derived purely from, the two taxonomies above (e.g. a Table Only item can still be Hard if it requires spotting a subtle row among many similar ones).

Intended uses

• —Evaluating retrieval-augmented generation (RAG) and long-context QA systems on real-world, multimodal financial documents.
• —Benchmarking table-QA and chart-QA models on central-bank statistical tables and charts, using presentation_format to select modality-specific subsets.
• —Studying multi-hop numerical and comparative reasoning over financial narratives, using query_type to select reasoning-specific subsets.
• —Evaluating faithfulness/grounding (i.e., whether a system's answer is actually supported by its cited evidence) using the evidence field as ground truth.

Licensing

This dataset (questions, answers, and evidence annotations) is released under the MIT License. The underlying source document, the Bangladesh Bank Annual Report FY2024–25, is a public publication of Bangladesh Bank; users should consult Bangladesh Bank's own terms for the source report itself.

Citation

If you use this dataset, please cite:

@misc{faria2026clairfinadversarialmultiagentframework,
      title={CLAIR-Fin: An Adversarial Multi-Agent Framework for Claim-Level Verification and Adaptive Debate in Cross-Modal Financial QA}, 
      author={Fatema Tuj Johora Faria and Mukaffi Bin Moin and Jubayer Al Mahmud and M. F. Mridha and Md. Alam Hossain},
      year={2026},
      eprint={2608.13706},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.13706}, 
}

Contact

Maintained by Fatema Tuj Johora Faria (fatema.faria142@gmail.com) and Mukaffi Bin Moin (mukaffi28@gmail.com). For questions, corrections, or issues, please open a discussion on the dataset's Hugging Face repository.