yolo1213811/research-ideation-arena-dataset

Research Ideation

This repository contains the public data payload for the Research Ideation benchmark, presented in the paper Ideation Arena: Evaluating LLM Generated Research Ideas with Battle-style Human Expert Assessment. The code is available at https://github.com/foss12138/Research-Ideation-Arena.

Files

• —final_ideation_results_with_response.json
• —Final pairwise ideation evaluation records with released responses.
• —queries.json
• —Query definitions used by the benchmark.

final_ideation_results_with_response.json

Each item contains:

• —query_text: serialized query context used for ideation
• —agent_a_name: display name of response A
• —agent_b_name: display name of response B
• —response_a: released text for response A
• —response_b: released text for response B
• —dimension_results: pairwise judgments for benchmark dimensions

Example structure:

{
  "query_text": "...",
  "agent_a_name": "...",
  "agent_b_name": "...",
  "response_a": "...",
  "response_b": "...",
  "dimension_results": {
    "D0": "A",
    "D1": "Tie",
    "D2": "B",
    "D3": "A",
    "D4": "Tie"
  }
}

queries.json

Each item contains:

• —query_id
• —query

Example structure:

{
  "query_id": "research_ideation_1",
  "query": "..."
}

Notes

• —This release is a sanitized public version.
• —The released final results file does not include internal metadata fields from the original evaluation pipeline.
• —dimension_results stores pairwise labels for benchmark dimensions. Labels use A, B, Tie, and in some settings may include BothBad.

License

This dataset is released under CC BY-NC 4.0.