Datasetpublic

yinita/ps4mas-castle-scale

PS4MAS CASTLE scale Current CASTLE / scale experiment catalog This generated section is authoritative. Older tables above are historical. Each experiments/<name>/ contains unified episodes.parquet, meta.json, and an unmodified summary.json when available. Raw full-hop traces.jsonl and run_config.json are separate files; scores are never injected into raw traces. Partial snapshots have immutable content-derived names. Existing experiments are skipped unless… See the full description on the dataset page: https://huggingface.co/datasets/yinita/ps4mas-castle-scale.

sourceHugging Facemitupdated 7d agoView on Hugging Face

0likes431downloads

Dataset Card

PS4MAS CASTLE scale

Current CASTLE / scale experiment catalog

This generated section is authoritative. Older tables above are historical.

Each experiments/<name>/ contains unified episodes.parquet, meta.json, and an unmodified summary.json when available. Raw full-hop traces.jsonl and run_config.json are separate files; scores are never injected into raw traces.

Partial snapshots have immutable content-derived names. Existing experiments are skipped unless --overwrite-existing is explicit. Routine uploads use additions only. Explicitly authorized, content-audited invalid-data cleanups are recorded separately under audits/.

Experiment	Model	Episode rows	Run	Judge	Full hops	Dataset
0911/scale_castle_final/qwen35_27b/central_b3	scaleqwen3527bcentralb3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/central_bmax	scaleqwen3527bcentralbmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/central_bmax__partial_6b2afb2d7ebc8939	scaleqwen3527bcentralbmax	772	[ ]	[ ]	772/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/central_bmax__partial_c14451927bbc21d7	scaleqwen3527bcentralbmax	716	[ ]	[ ]	716/800	CASTLE legacy/unverified
0911/scale_castle_final/qwen35_27b/debate_b3	scaleqwen3527bdebateb3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/debate_b3__partial_b57f73d9fd37a1cc	scaleqwen3527bdebateb3	411	[ ]	[ ]	411/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/debate_bmax	scaleqwen3527bdebatebmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/debate_bmax__partial_7d32fb75d8a00c46	scaleqwen3527bdebatebmax	161	[ ]	[ ]	161/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/debate_bmax__partial_df272f3d2c83b2b1	scaleqwen3527bdebatebmax	320	[ ]	[ ]	320/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/discuss_decide_b3	scaleqwen3527bdiscussdecide_b3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/discuss_decide_b3__partial_71e4d43b7c088730	scaleqwen3527bdiscussdecide_b3	295	[ ]	[ ]	295/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/discuss_decide_bmax	scaleqwen3527bdiscussdecide_bmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/discuss_decide_bmax__partial_65b077c1ee55651e	scaleqwen3527bdiscussdecide_bmax	383	[ ]	[ ]	383/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/plan_execute_b3	scaleqwen3527bplanexecute_b3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_27b/plan_execute_bmax	scaleqwen3527bplanexecute_bmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/central_b3	scaleqwen359bcentralb3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/central_bmax	scaleqwen359bcentralbmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/debate_b3	scaleqwen359bdebateb3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/debate_bmax	scaleqwen359bdebatebmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/discuss_decide_b3	scaleqwen359bdiscussdecide_b3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/discuss_decide_bmax	scaleqwen359bdiscussdecide_bmax	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/discuss_decide_bmax__partial_d1030fedf8da5107	scaleqwen359bdiscussdecide_bmax	661	[ ]	[ ]	661/800	CASTLE legacy/unverified
0911/scale_castle_final/qwen35_9b/plan_execute_b3	scaleqwen359bplanexecute_b3	800	[x]	[ ]	800/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/plan_execute_b3__partial_31bb233131167c17	scaleqwen359bplanexecute_b3	291	[ ]	[ ]	291/800	official CASTLE-200
0911/scale_castle_final/qwen35_9b/plan_execute_bmax	scaleqwen359bplanexecute_bmax	800	[x]	[ ]	800/800	official CASTLE-200

Run = all intended scenario/config pairs produced a record; blank answers remain flagged in meta. Judge = every intended pair has valid non-mock scores. Mock placeholders never count as completed judging. Score-only historical results are archived as snapshots and do not prove runtime profile correctness. Full-hop coverage is a separate requirement. No missing historical hops are regenerated automatically.