NextGenWhu/FinCDM-Fin-KQA

FinCDM-FinEval-KQA

Repository: NextGenWhu/FinCDM-CPA-KQA

📖 Overview

FinCDM-CPA-KQA is a specialized dataset for financial knowledge-based question answering, derived from the research presented in:

"From Scores to Skills: A Cognitive Diagnosis Framework for Evaluating Financial Large Language Models"

This dataset is designed to evaluate large language models (LLMs) on their ability to perform financial knowledge reasoning, compliance assessment, and knowledge-based question answering. It serves as a robust benchmark for assessing how well models understand and apply financial domain knowledge, making it valuable for both research and practical applications in finance.

📊 Dataset Description

The dataset is provided in JSON format and consists of multiple-choice questions tailored to financial knowledge. Each entry includes the following fields:

• —`id`: Unique identifier for the question.
• —`query`: The question text, including multiple-choice options.
• —`answer`: The correct option (e.g., A, B, C, or D).
• —`choices`: A list of option labels (e.g., ["A", "B", "C", "D"]).
• —`gold`: The 0-based index of the correct answer in the choices list.
• —`text`: The correct option with a brief explanation.

🚀 Use Cases

• —Benchmarking LLMs: Evaluate the financial knowledge reasoning capabilities of large language models.
• —Training QA Systems: Develop and fine-tune question-answering systems for financial applications.
• —Compliance and Auditing: Support tasks related to financial compliance, risk assessment, and auditing.

📝 Citation

If you use this dataset in your research or applications, please cite the following paper:

@article{FinCDM2025,
  title={From Scores to Skills: A Cognitive Diagnosis Framework for Evaluating Financial Large Language Models},
  author={},
  journal={arXiv preprint arXiv:2508.13491},
  year={2025},
  url={https://arxiv.org/abs/2508.13491}
}