openbenchmarks/OB-LegalQA

OB LegalQA 95 questions on 7 redlined contract PDFs. They evaluate document parsers, not models: can a parser give a downstream agent what it needs to answer a question about a heavily negotiated contract? Columns column meaning id stable item id question the question put to the agent answer gold answer, from the position the parties agreed answer_detail why that is the answer task_type which redline pattern the question turns on documents… See the full description on the dataset page: https://huggingface.co/datasets/openbenchmarks/OB-LegalQA.