CoolFace
Datasetpublic

reasoning-degeneration-dev/test-updated-extractor-v2

test-updated-extractor-v2 LLM-based math span extraction with canonicalization Dataset Info Rows: 1 Columns: 26 Columns Column Type Description question Value('string') No description provided metadata Value('string') No description provided task_source Value('string') No description provided formatted_prompt List({'content': Value('string'), 'role': Value('string')}) No description provided responses_by_sample… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/test-updated-extractor-v2.

sourceHugging Facemitupdated 9mo agoView on Hugging Face
0likes10downloads
Dataset Card

test-updated-extractor-v2

LLM-based math span extraction with canonicalization

Dataset Info

  • —Rows: 1
  • —Columns: 26

Columns

ColumnTypeDescription
questionValue('string')No description provided
metadataValue('string')No description provided
task_sourceValue('string')No description provided
formatted_promptList({'content': Value('string'), 'role': Value('string')})No description provided
responsesbysampleList(List(Value('string')))No description provided
tokensbysampleList(Value('int64'))No description provided
numroundsby_sampleList(Value('int64'))No description provided
continuationpromptsby_sampleList(List(Value('string')))No description provided
continuationtextby_sampleList(List(Value('string')))No description provided
strategy_typeValue('string')No description provided
terminationreasonby_sampleList(Value('string'))No description provided
correctroundby_sampleList(Value('null'))No description provided
smartterminationenabledValue('bool')No description provided
retrycountby_sampleList(Value('int64'))No description provided
answersbysampleList(List(List(Value('string'))))No description provided
evalsbysampleList(List(List(Value('bool'))))No description provided
evalmetadataby_sampleList(List(List({'answerblock': Value('string'), 'error': Value('null'), 'finalanswer': Value('float64'), 'is_correct': Value('bool')})))No description provided
annotationsbysampleList(List(Value('null')))No description provided
arithmeticattemptsby_sampleList(List({'canonicalform': Value('string'), 'canonicalkey': Value('string'), 'endchar': Value('int64'), 'endcharinround': Value('null'), 'hasfreevariables': Value('bool'), 'islegal': Value('null'), 'ismultioperand': Value('bool'), 'isvalidarithmetic': Value('bool'), 'matchtype': Value('null'), 'offsetfound': Value('bool'), 'operanda': Value('float64'), 'operandastr': Value('null'), 'operandb': Value('float64'), 'operandbstr': Value('null'), 'operator': Value('string'), 'operatorraw': Value('null'), 'result': Value('float64'), 'resultstr': Value('null'), 'roundidx': Value('int64'), 'separatorraw': Value('null'), 'spantext': Value('string'), 'startchar': Value('int64'), 'startcharinround': Value('null')}))Extracted math operations with spans (compatible with stage_02a)
canonicalizedequationsby_sampleList(List(Value('string')))LLM-generated canonical equation forms
extractionstatsby_sampleList({'hasfreevariables': Value('int64'), 'invalidarithmetic': Value('int64'), 'mergedtextlength': Value('int64'), 'multioperand': Value('int64'), 'offsetsfound': Value('int64'), 'totalattemptscreated': Value('int64'), 'totalspansfromllm': Value('int64'), 'unevaluable': Value('int64'), 'uniquecanonicalkeys': Value('int64'), 'valid_arithmetic': Value('int64')})Statistics about LLM extraction per sample
expressionattemptsby_sampleList(List(Value('null')))No description provided
subgoalsbysampleList(List(Value('null')))No description provided
failuremarkersby_sampleList(List(Value('null')))No description provided
subgoalwindowsby_sampleList(List(Value('null')))No description provided
branchesbysampleList(List(Value('null')))No description provided

Generation Parameters

json
{
  "script_name": "stage_02_alternative_llm_extractor.py",
  "model": "gpt-4o-mini",
  "hyperparameters": {
    "chunking_mode": "paragraph"
  },
  "input_datasets": [
    "reasoning-degeneration-dev/EAS-Qwen3-30B-A3B-Thinking-2507-cd8arg-iterative"
  ],
  "description": "LLM-based math span extraction with canonicalization"
}

Experiment Documentation

For complete experiment details, see https://github.com/reasoning-degeneration/ReasoningDegeneration/tree/main/experiments/degen_test_1_countdown/documentation

Usage

python
from datasets import load_dataset

dataset = load_dataset("reasoning-degeneration-dev/test-updated-extractor-v2", split="train")
print(f"Loaded {len(dataset)} rows")

This dataset is tracked in [reasoning-degeneration-dev/PROJECT-MANIFEST](https://huggingface.co/datasets/reasoning-degeneration-dev/PROJECT-MANIFEST)