Datasetpublic

reasoning-degeneration-dev/t1-wingdings-countdown-together_ai-moonshotai-kimi-k2-thinking-61281244

t1-wingdings-countdown-together_ai-moonshotai-kimi-k2-thinking-61281244 Wingdings compliance evaluation on countdown arithmetic. Model must reason using only symbolic characters (arrows, checkmarks, boxes, etc.) while solving arithmetic countdown problems. Results Metric Value pass@1 0.3000 Compliance (mean) 0.3583 Compliance (min) 0.2491 Compliance (max) 0.4841 Total problems 10 Details Parameter Value Model… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/t1-wingdings-countdown-together_ai-moonshotai-kimi-k2-thinking-61281244.

sourceHugging Facemitupdated 7mo agoView on Hugging Face

0likes16downloads

Dataset Card

t1-wingdings-countdown-together_ai-moonshotai-kimi-k2-thinking-61281244

Wingdings compliance evaluation on countdown arithmetic.

Model must reason using only symbolic characters (arrows, checkmarks, boxes, etc.) while solving arithmetic countdown problems.

Results

Metric	Value
pass@1	0.3000
Compliance (mean)	0.3583
Compliance (min)	0.2491
Compliance (max)	0.4841
Total problems	10

Details

Parameter	Value
Model	`together_ai/moonshotai/Kimi-K2-Thinking`
Thinking	True
Temperature	0.7
Max tokens	32768
Num examples	10
Num args	8

Usage

python

from datasets import load_dataset
ds = load_dataset("reasoning-degeneration-dev/t1-wingdings-countdown-together_ai-moonshotai-kimi-k2-thinking-61281244", split="train")
print(f"Mean compliance: {sum(ds['wingdings_compliance']) / len(ds):.4f}")

Tracked in [reasoning-degeneration-dev/PROJECT-MANIFEST](https://huggingface.co/datasets/reasoning-degeneration-dev/PROJECT-MANIFEST)