Datasetpublic

yuxbao/ask-accounting-rag-retrieval-golden-set

Ask Accounting RAG Retrieval Golden Test Set This private dataset evaluates citation retrieval for the Ask Accounting RAG service. Each case contains a question, the evidence that must be retrieved, the factual claims that a correct answer must support, and a citation rule. It is a regression test set, not model-training data. Cases remain in candidate status until a domain owner has reviewed the question, evidence, and expected claims. Only approved cases should gate a release.… See the full description on the dataset page: https://huggingface.co/datasets/yuxbao/ask-accounting-rag-retrieval-golden-set.

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes10downloads

Dataset Card

Ask Accounting RAG Retrieval Golden Test Set

This private dataset evaluates citation retrieval for the Ask Accounting RAG service. Each case contains a question, the evidence that must be retrieved, the factual claims that a correct answer must support, and a citation rule.

It is a regression test set, not model-training data. Cases remain in candidate status until a domain owner has reviewed the question, evidence, and expected claims. Only approved cases should gate a release.

Coverage

The dataset contains 25 cases, with five cases for each UI source group:

Source group	Cases	Retrieval collection
R3 Docupedia	5	`confluence_R3_qa_pairs`
R3 BR	5	`confluence_BR_qa_pairs`
R3 others	5	`confluence_R3_qa_pairs`
S4 Docupedia	5	`confluence_S4_qa_pairs`
CD SOCOS	5	`confluence_CD_qa_pairs`

Data Fields

Field	Description
`id`	Stable case identifier.
`source_group`	Ask Accounting source-group filter used for the run.
`question`	User-style evaluation question.
`required_evidence`	Page URLs or Qdrant document titles that must be retrieved.
`supporting_evidence`	Helpful evidence that is not required for a pass.
`expected_claims`	Facts the answer must support using the required evidence.
`citation_rule`	Pass condition for citation retrieval.
`status`	Review state. `candidate` is not release-gating until approved.

CD SOCOS evidence uses collection and document_title because the current SOCOS ingestion payload does not retain document deep links. R3 BR similarly uses collection/title evidence because its indexed corpus is not currently represented by a sys_data_source record.

Loading

python

from datasets import load_dataset

dataset = load_dataset(
    "yuxbao/ask-accounting-rag-retrieval-golden-set",
    split="test",
)

Evaluation Protocol

1.Select the case's source_group in Ask Accounting.
2.Submit the question using the normal RAG request path.
3.Collect returned citations and compare them with required_evidence.
4.Mark the retrieval as passed only when the citation_rule is satisfied.
5.Separately evaluate the generated answer against expected_claims.
6.Persist the run result, application version, and corpus sync timestamp in an internal results store. Do not write production user chats or credentials into this dataset.

Governance

This repository is private because it contains internal document references and business-process facts. Do not upload document bodies, user conversation data, access tokens, passwords, or other credentials.