pranaysuyash/laya-formatting-fragility

Laya Formatting Fragility — a perturbation suite for typed decision models Small decision models ("decide, don't chat" — Laya, AgentJev and friends) return typed answers with probabilities in milliseconds. Their answers can be sensitive to formatting: option key names, option order, and state phrasing change results even when the situation and the gold answer are identical. Vendor benchmarks don't measure this. This dataset does. Every row pair differs in exactly one formatting… See the full description on the dataset page: https://huggingface.co/datasets/pranaysuyash/laya-formatting-fragility.