CoolFace
Datasetpublic

3amthoughts/4am_FinQNA-Filtered

4am_FinQNA-Filtered Dataset Summary This dataset is a refined, JSONL-formatted version of the FinQNA dataset, specifically optimized for high-performance financial QA tasks. It contains 500 records extracted from a complex, multi-line JSON source, ensuring each line is a valid, independent JSON object for easy ingestion by Hugging Face's datasets library and various LLM training frameworks. Benchmark Comparison Benchmark Category Alignment… See the full description on the dataset page: https://huggingface.co/datasets/3amthoughts/4am_FinQNA-Filtered.

sourceHugging Facemitupdated 4mo agoView on Hugging Face
1likes19downloads
Dataset Card

4am_FinQNA-Filtered

Dataset Summary

This dataset is a refined, JSONL-formatted version of the FinQNA dataset, specifically optimized for high-performance financial QA tasks. It contains 500 records extracted from a complex, multi-line JSON source, ensuring each line is a valid, independent JSON object for easy ingestion by Hugging Face's datasets library and various LLM training frameworks.

Benchmark Comparison

Benchmark CategoryAlignment ScoreReasoning Requirements
FinQA / ConvFinQA87.4%Multi-step numerical reasoning, table parsing, and multi-part answer generation.
FinanceBench86.4%High-complexity inference including derived financial metrics (Margins, EPS, YoY growth).

Key Complexity Metrics

  • —Mathematical Density: ~87% of records require calculating multiple discrete values or performing sequential arithmetic operations.
  • —Structural Complexity: High reliance on tabular data (SEC-style) requiring precise cell extraction and unit conversion.
  • —Data Integrity: Estimated at 96.0%, with a theoretical LLM reasoning ceiling of 98.5% for models capable of complex derived logic.

Usage for Evaluation

This dataset serves as a high-signal 'stress test' for Large Language Models. Due to the multi-part nature of the answers (e.g., 1.8; 3.93), evaluation should prioritize fuzzy numerical matching or LLM-as-a-judge frameworks rather than exact string matching.

Dataset Structure

Each record represents a financial query and its associated context:

  • —id: Unique identifier for the financial record.
  • —pre_text: Textual context found before the financial tables (e.g., analyst remarks).
  • —post_text: Textual context found after the financial tables.
  • —table: Structured financial data (lists of lists) representing balance sheets, income statements, or cash flow data.
  • —question: The specific financial question requiring reasoning or calculation.
  • —answer: The target answer derived from the provided context.

Technical Processing

The dataset underwent a robust conversion process to ensure data integrity:

  1. 1.Extraction: Handled multi-line JSON structures and concatenated objects.
  2. 2.Cleaning: Filtered out malformed JSON markers and separators.
  3. 3.Format: Converted to .jsonl (JSON Lines) to support streaming and efficient batch processing.

Usage

You can load this dataset using the Hugging Face library:

python
from datasets import load_dataset

dataset = load_dataset("3amthoughts/4am_FinQNA-Filtered")
print(dataset['train'][0])

Applications

  • —Training Large Language Models (LLMs) for financial reasoning.
  • —Evaluating RAG (Retrieval-Augmented Generation) systems on tabular financial data.
  • —Benchmarking numerical reasoning in a business context.