rajatagarwal457/gdelt-forecast-freeform

GDELT-Forecast Free-form

924 free-form forecasting questions (named entities, numbers, dates, short narrative answers) generated from clusters of news articles in the GDELT 2.0 corpus (Aug 2025 – Apr 2026). Each question is paired with the original seed-event articles, top-5 retrieved evidence articles dated strictly before the question creation date, and a verified ground-truth answer.

Intended use

Training and evaluating LLM-based forecasting models on non-binary questions in a strict forecasting posture. Companion to `gdelt-forecast-binary` (1,215 yes/no questions from the same corpus).

This split is harder than the binary one in two ways:

• —Open answer space: the model must generate the right name / number / date / phrase rather than choose between two options.
• —Judging cost: exact-match doesn't work for most rows — you need a fuzzy judge (e.g., GPT-4o) to score predictions.

Distribution by answer type

How it was built

Same five-stage pipeline as `gdelt-forecast-binary`. At Phase 3 (question generation), each cluster was assigned a non-yes/no answer type by GPT-4o based on what kind of question best fit the underlying event. See the binary dataset card for full pipeline details.

Schema

Identical to gdelt-forecast-binary. See that card's "Schema" section. Notable differences:

• —answer_type ∈ {name, number, free_form, date}
• —ground_truth_answer is a free-form string
• —required_type documents what the generation prompt asked for; it usually matches answer_type but the model occasionally produced a different valid type

Statistics

• —Total questions: 924
• —Cluster size: 3 → 351, 4 → 199, 5 → 374
• —Sources agree: 922 / 924
• —Date range (question_start_date): late Aug 2025 → mid Mar 2026

Strict forecasting posture

Same as the binary split. Respect the `question_start_date` cutoff when building your own retrieval.

Judging non-binary answers

Exact-match is too strict for most rows. We recommend the OpenForecaster-style GPT-4o judge prompt:

Your task is to judge whether the given response to a question matches a given ground truth answer or not. For numeric answers, the relative error must be ≤ 1 % to count as a match. For names and free-form answers, the response can be more specific or paraphrase the ground truth, but must cover the same information.

Implementation reference: local_reward.py in the Anthral forecast-research repo.

Known limitations

Same as the binary split. Additionally: answer phrasing variance is high. The same fact may be expressed many valid ways; do not rely on exact-match scoring.

Citation

@misc{anthral2026gdeltforecast,
  title  = {GDELT-Forecast: Strict-cutoff forecasting benchmarks from GDELT news clusters},
  author = {Rajat Agarwal and Anthral Labs},
  year   = {2026},
  url    = {https://huggingface.co/datasets/rajatagarwal457/gdelt-forecast-freeform},
}

License

CC-BY-4.0, matching upstream GDELT 2.0.