Datasetpublic

AgenticFinLab/PortBench-QA

PortBench QA Dataset Dataset Description 6,269 structured question-answer pairs probing correlation-based financial reasoning for multi-asset portfolio management, generated from the PortBench Market Base Dataset. Task Templates Template Task Complexity Pairs T1 Return prediction — direction for next N days 1 (single asset) 1,000 T2 Risk assessment — VaR at given confidence level 1 1,000 T3 Position sizing — given max drawdown… See the full description on the dataset page: https://huggingface.co/datasets/AgenticFinLab/PortBench-QA.

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

3likes139downloads

Dataset Card

![Paper](https://arxiv.org/abs/2605.27887) ![Code](https://github.com/AgenticFinLab/portbench) ![Homepage](https://portbench.github.io/)

PortBench QA Dataset

Dataset Description

6,269 structured question-answer pairs probing correlation-based financial reasoning for multi-asset portfolio management, generated from the PortBench Market Base Dataset.

Task Templates

Template	Task	Complexity	Pairs
T1	Return prediction — direction for next N days	1 (single asset)	1,000
T2	Risk assessment — VaR at given confidence level	1	1,000
T3	Position sizing — given max drawdown constraint	1	1,000
T4	Pairwise allocation — minimize variance for 2 assets	2 (pairwise)	1,000
T5	Multi-asset optimization — maximize Sharpe for 3+ assets	3 (multi-asset)	1,000
T6	Rebalancing decision — threshold-based trigger	3	778
T7	Regime detection — identify bull/bear/sideways + adjust allocation	4 (full portfolio)	491

Splits

Split	Period	Pairs
Train	2015-01-02 – 2022-12-31	1,941
Val	2023-01-01 – 2024-12-31	2,700
Test	2025-01-01 – 2025-12-31	1,628

Data Fields

Each JSONL record contains:

Field	Type	Description
`id`	string	Unique identifier (`{template}_{class}_{date}_{seq}`)
`template`	string	Task template (T1–T7)
`complexity`	int	Difficulty level (1–4)
`split`	string	train / val / test
`market_regime`	string	bull / bear / sideways / crisis
`asset_class`	string	Target asset class
`assets`	list[string]	Ticker symbols involved
`decision_date`	string	Point-in-time decision date (YYYY-MM-DD)
`context_summary`	string	Market context window (prices, macro, correlations, news)
`question`	string	The question with all necessary numerical context
`answer`	string	Ground-truth answer
`answer_numeric`	float	Numeric ground-truth (for scoring)
`explanation`	string	Step-by-step explanation of the answer
`metadata`	object	Additional fields (future_return, horizon, volatility, text coverage, etc.)

Example

json

{
  "id": "T1_all_20251226_0002",
  "template": "T1",
  "complexity": 1,
  "split": "test",
  "market_regime": "sideways",
  "assets": ["DBB"],
  "decision_date": "2025-12-26",
  "question": "Asset: DBB\nHistorical prices (past 60 trading days): start=20.40, end=22.01, cumulative_return=+7.9%, annualized_volatility=14.0%\n...\nPredict whether the return of DBB over the next 21 trading days will be: positive (>+1%), negative (<-1%), or flat (within ±1%).",
  "answer": "flat",
  "answer_numeric": 0.0,
  "explanation": "The actual 21-day forward return for DBB starting 2025-12-26 was +0.00%, which classifies as 'flat'."
}

Text Coverage

85.3% of QA pairs include associated news text in the context window (avg 3,997 chars).

Market Regime Distribution

QA pairs are stratified by market regime (bull/bear/sideways/crisis) to enable per-regime performance decomposition.

Intended Use

—Evaluating LLM financial reasoning capabilities across four difficulty levels
—Benchmarking correlation-based multi-asset decision-making
—Comparing static knowledge (QA accuracy) with dynamic pipeline performance (CEPS)

Point-in-Time (PiT) Constraint

All questions use only information available at or before the decision_date. Ground-truth answers are computed from realized future data that is never included in the question or context.

Citation

bibtex

@article{zhao2026portbench,
  title={PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management},
  author={Zhao, Yuxuan and Chen, Sijia and Su, Ningxin},
  journal={arXiv preprint arXiv:2605.27887},
  year={2026}
}