Datasetpublic

reasoning-degeneration-dev/t1-wingdings-musr-murder-together_ai-moonshotai-kimi-k2-thinking-8c1b04d5

t1-wingdings-musr-murder-together_ai-moonshotai-kimi-k2-thinking-8c1b04d5 Wingdings compliance evaluation on MuSR murder mysteries. Model must reason using only symbolic characters while solving murder mystery problems (means, motive, opportunity). Results Metric Value pass@1 0.9000 Compliance (mean) 0.1603 Compliance (min) 0.0979 Compliance (max) 0.3724 Total problems 10 Details Parameter Value Model… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/t1-wingdings-musr-murder-together_ai-moonshotai-kimi-k2-thinking-8c1b04d5.

sourceHugging Facemitupdated 7mo agoView on Hugging Face

0likes12downloads

Dataset Card

t1-wingdings-musr-murder-together_ai-moonshotai-kimi-k2-thinking-8c1b04d5

Wingdings compliance evaluation on MuSR murder mysteries.

Model must reason using only symbolic characters while solving murder mystery problems (means, motive, opportunity).

Results

Metric	Value
pass@1	0.9000
Compliance (mean)	0.1603
Compliance (min)	0.0979
Compliance (max)	0.3724
Total problems	10

Details

Parameter	Value
Model	`together_ai/moonshotai/Kimi-K2-Thinking`
Thinking	True
Temperature	0.7
Max tokens	32768
Samples	1

Usage

python

from datasets import load_dataset
ds = load_dataset("reasoning-degeneration-dev/t1-wingdings-musr-murder-together_ai-moonshotai-kimi-k2-thinking-8c1b04d5", split="train")
print(f"Mean compliance: {sum(ds['wingdings_compliance']) / len(ds):.4f}")

Tracked in [reasoning-degeneration-dev/PROJECT-MANIFEST](https://huggingface.co/datasets/reasoning-degeneration-dev/PROJECT-MANIFEST)