yourbench-testing/harry-potter-quiz-test
Harry Potter Quiz Test This dataset was generated using YourBench (v1.0.0), an open-source framework for generating domain-specific benchmarks from document collections. Pipeline Steps ingestion: Read raw source documents, convert them to normalized markdown and save for downstream steps summarization: Perform hierarchical summarization: chunk-level LLM summaries followed by combine-stage reduction chunking: Split texts into token-based single-hop and multi-hop… See the full description on the dataset page: https://huggingface.co/datasets/yourbench-testing/harry-potter-quiz-test.
Harry Potter Quiz Test
This dataset was generated using YourBench (v1.0.0), an open-source framework for generating domain-specific benchmarks from document collections.
Pipeline Steps
- ingestion: Read raw source documents, convert them to normalized markdown and save for downstream steps
- summarization: Perform hierarchical summarization: chunk-level LLM summaries followed by combine-stage reduction
- chunking: Split texts into token-based single-hop and multi-hop chunks
- single_shot_question_generation: Generate standalone question-answer pairs per chunk using LLM
Reproducibility
To reproduce this dataset, use YourBench v1.0.0 with the following configuration:
hf_configuration:
hf_dataset_name: harry-potter-quiz-test
hf_organization: $HF_ORGANISATION
hf_token: $HF_TOKEN
push_to_hub: true
model_list:
- model_name: Qwen/Qwen3-VL-8B-Instruct
base_url: http://us-east-1.sumuk.org:9100/stargate.neon-pentatonic.ts.net/30000/v1/
api_key: $API_KEY
max_concurrent_requests: 8
pipeline:
ingestion:
source_documents_dir: example/harry_potter_quizz/data
output_dir: example/harry_potter_quizz/processed
supported_file_extensions:
- .md
- .txt
- .pdf
summarization:
summarization_system_prompt: custom_summarization_system_prompt.md
num_hierarchy_levels: 2
recursive_doc_size_limit: 100
recursive_chunk_size_limit: 10
max_tokens_final: 4000
chunking:
l_max_tokens: 1024
token_overlap: 256
single_shot_question_generation:
question_mode: multi-choice
chunk_sampling:
enable: false
num_samples: 100
strategy: random
random_seed: 42
num_single_shot_samples: 100
chunk_sample_budget: 5
question_budget_per_chunk: 2
max_tokens: 4000
prepare_lighteval:
single_shot_subset: single_shot_questions
multi_hop_subset: multi_hop_questions
cross_doc_subset: cross_document_questions
chunked_subset: chunked
summarized_subset: summarized
output_subset: prepared_lighteval
debug: true
(This dataset card was automatically generated by YourBench)