todor-cmd/memdoc
memdoc A dataset for evaluating QA agents that retrieve from both conversational memory and a document corpus, while controlling for variance that comes from the question and its evidence themselves. Each item is a question with 2-4 gold-evidence chunks partitioned across the two stores. The same questions and evidence can be presented as memory-only, document-only, or split across both. Because the question text and the underlying facts stay fixed, differences in agent… See the full description on the dataset page: https://huggingface.co/datasets/todor-cmd/memdoc.
memdoc
A dataset for evaluating QA agents that retrieve from both conversational memory and a document corpus, while controlling for variance that comes from the question and its evidence themselves.
Each item is a question with 2-4 gold-evidence chunks partitioned across the two stores. The same questions and evidence can be presented as memory-only, document-only, or split across both. Because the question text and the underlying facts stay fixed, differences in agent behaviour can be attributed to where evidence lives and how the agent uses the two stores, rather than to a changing question mix.
The snapshot has 198 answerable questions, three persona memory corpora, the MultiHop-RAG document collection, and an evidence-id → URL map. Questions and the document corpus are derived from MultiHop-RAG (ODC-BY). Memory sessions were generated for this work.