thinkwee/DDRBench_10K

DDRBench: Deep Data Research Benchmark

**📊 Leaderboard & Demo** | **📄 Paper (Arxiv)**

Overview

DDRBench (Deep Data Research Benchmark) is a comprehensive evaluation framework designed to assess the capabilities of Large Language Model (LLM) agents in performing complex, multi-turn data research and reasoning tasks. Unlike traditional Q&A benchmarks, DDRBench focuses on scenarios requiring deep interaction with structured databases, tool usage, and long-context reasoning.

This dataset repository specifically hosts the 10-K Financial Database, a core component of the DDRBench suite. It contains structured financial data extracted from SEC 10-K filings, enabling agents to answer intricate financial questions that mimic real-world analyst workflows.

Dataset Structure

The dataset is organized into multiple configurations (subsets), representing different tables from the underlying SQLite database:

• —`financial_facts`: The primary table containing over 5 million financial metrics (US-GAAP, IFRS) with values, units, and fiscal periods.
• —`companies`: Registry of companies with CIK, names, and SIC codes.
• —`filings`: Metadata for the SEC filings source documents.
• —`company_addresses` & `company_tickers`: Geographic and market identification data.
• —`table_documentation` & `column_documentation`: Meta-information describing the database schema to the agents.

Usage

Data Inspection

Load specific tables using the datasets library:

from datasets import load_dataset

# Load the main financial facts table
financial_facts = load_dataset("thinkwee/DDRBench_10K", "financial_facts")

# Load company information
companies = load_dataset("thinkwee/DDRBench_10K", "companies")

For agent trajectories and evaluation logs, please refer to the DDRBench Trajectory Dataset.

Run Deep Data Research

Please use the database file under `/raw` path and refer to https://github.com/thinkwee/DDR_Bench for running the agent.