CoolFace
Datasetpublic

reasoning-degeneration-dev/t1-musr-prompt-enhancement-together_ai-meta-llama-meta-llama-3-1-8b

t1-musr-prompt-enhancement-together_ai-meta-llama-meta-llama-3-1-8b MuSR Prompt Enhancement via Knowledge Synthesis + Compliance Tracking. Performance Eval Set pass@1 Original (base) 0.7450 Original (enhanced prompt) 0.7300 Heldout (base prompt) 0.7050 Heldout (enhanced prompt) 0.7050 Strategies Natural strategy: Means‑Motive‑Opportunity Heuristic Enhanced strategy: Means-Motive-Opportunity Matrix Synthesized Facts… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/t1-musr-prompt-enhancement-together_ai-meta-llama-meta-llama-3-1-8b.

sourceHugging Facemitupdated 7mo agoView on Hugging Face
0likes11downloads
Dataset Card

t1-musr-prompt-enhancement-together_ai-meta-llama-meta-llama-3-1-8b

MuSR Prompt Enhancement via Knowledge Synthesis + Compliance Tracking.

Performance

Eval Setpass@1
Original (base)0.7450
Original (enhanced prompt)0.7300
Heldout (base prompt)0.7050
Heldout (enhanced prompt)0.7050

Strategies

Natural strategy: Means‑Motive‑Opportunity Heuristic Enhanced strategy: Means-Motive-Opportunity Matrix

Synthesized Facts

  1. 1.Always list and evaluate means, motive, and opportunity for every suspect—note which of the three is strong, weak, or missing rather than assuming equality.
  2. 2.Verify temporal feasibility: cross-check timestamps, CCTV, receipts, and witness times to confirm a suspect could physically be at the scene when the crime occurred.
  3. 3.Treat direct physical or documentary links (forensic matches, receipts, weapon serial numbers) as stronger evidence than vague or circumstantial associations (works nearby, general knowledge).
  4. 4.Distinguish access to a type of weapon from possession of the exact weapon—prioritize evidence tying a particular item to a suspect (unique markings, purchase record, possession).
  5. 5.Cross-check and corroborate alibis with independent records; flag evasions and contradictions but seek external confirmation before using them as decisive evidence.
  6. 6.Interpret evasive or emotionally defensive behavior as a prompt to investigate further, not as a standalone proof of guilt—always seek corroborating facts.
  7. 7.Quantify motive strength: separate emotional motives (anger, jealousy) from material motives (financial gain, career survival) and treat clearer, verifiable gains as more substantial.
  8. 8.Look for signs of deliberate cover-up—aliases, altered records, cleaned evidence, missing items—and treat them as strong indicators of consciousness of guilt if corroborated.
  9. 9.Compare suspects directly in a pairwise way on each evidential axis instead of evaluating each in isolation; a relative comparison often reveals the stronger candidate.
  10. 10.Use negative evidence (airtight alibi, lack of access) to rule out suspects; absence of opportunity can be more exculpatory than motive is incriminating.
  11. 11.Avoid over-weighting mere presence at or near the scene: presence gives opportunity but must be combined with means and motive and supported by corroboration.
  12. 12.Apply Occam’s razor carefully: prefer the simplest explanation that fits the evidence, but remain open to more complex scenarios when multiple independent clues point toward them.
  13. 13.When suspect statements conflict, document precise timeline inconsistencies and prioritize physical timestamps (camera, transaction logs) over recollection.
  14. 14.Prioritize independent, third-party evidence (surveillance, receipts, lab tests) above single-witness testimony when there is a conflict between sources.
  15. 15.Report confidence as degrees rather than absolutes; state which elements are speculative and which are well-supported to avoid premature conclusions.
  16. 16.Identify unique capabilities (specialized training, tools, or skills) that make a suspect especially capable of using the method of killing and weight those accordingly.
  17. 17.Treat routine access (job in same building) as weaker than possession of keys, spare keys, or documented unsupervised entry—document the difference explicitly.
  18. 18.Avoid anchoring on the first plausible suspect; actively seek disconfirming evidence and re-evaluate the leading hypothesis as new facts emerge.
  19. 19.Consider the possibility of accomplices, staged scenes, or third-party manipulation when evidence is internally inconsistent or when both suspects have weak but similar motives.
  20. 20.When multiple suspects satisfy similar elements, prefer the suspect backed by the most corroborated, independent evidence across different categories (forensic, documentary, testimonial).

Details

ParameterValue
Modeltogether_ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
Synthesis modelgpt-5-mini
Judge modelopenai/gpt-4o-mini
Questions per set50
Samples per question4
Facts synthesized20

Generated by `musr_prompt_enhancement/run_experiment.py` on 2026-02-23 23:11