strategy-scope/si_et_al-ideation-gpt41mini-20260414_223535
si_et_al-ideation-gpt41mini-20260414_223535 Benchmark: si_et_al Generated: 2026-04-14T22:46:55.858516 Parameters model: gpt-4.1-mini-2025-04-14 judge_model: anthropic/claude-sonnet-4-5-20250929 strategy: none n_ideas_per_run: 5 n_runs: 3 n_topics: 3 rag: False Overall Averages Topics: 3 Total ideas: 41 Generation model: gpt-4.1-mini-2025-04-14 Judge model: anthropic/claude-sonnet-4-5-20250929 Runs per topic: 3 Ideas per run: 5 Evaluation… See the full description on the dataset page: https://huggingface.co/datasets/strategy-scope/si_et_al-ideation-gpt41mini-20260414_223535.
sietal-ideation-gpt41mini-20260414_223535
Benchmark: sietal Generated: 2026-04-14T22:46:55.858516
Parameters
- model: gpt-4.1-mini-2025-04-14
- judge_model: anthropic/claude-sonnet-4-5-20250929
- strategy: none
- n_ideas_per_run: 5
- n_runs: 3
- n_topics: 3
- rag: False
Overall Averages
- Topics: 3
- Total ideas: 41
- Generation model: gpt-4.1-mini-2025-04-14
- Judge model: anthropic/claude-sonnet-4-5-20250929
- Runs per topic: 3
- Ideas per run: 5
Evaluation protocol: port of Si et al. ai_researcher/src/idea_direct_score.py (single overall 1-10 score with conference-acceptance anchors; anchors: 7=accept, 8=top 50% accepted, 9=top 15%, 10=seminal).
Per-Topic Results
Columns
- topic: NLP topic (bias, coding, factuality, math, multilingual, safety, uncertainty)
- runs: list of per-run generation records (prompt, raw_response, ideas, usage, seed)
- ideas: pooled idea texts across all runs
- n_ideas: total ideas after pooling
- evaluation: overall 1-10 score (codebase port) + cosine + Jaccard diversity + optional tournament ranking