mhmsadegh/rectom-causal-reasoning-crossencoder-bce-v2-709items

RecToM Causal-Reasoning Cross-Encoder v2 (BCE, 709-item training pool)

Same BCE cross-encoder recipe as rectom-causal-reasoning-crossencoder-bce (the {'MATCH@1': 0.064, 'Hit@1': 0.064, 'Hit@3': 0.218, 'Hit@5': 0.305, 'MRR': 0.179} 266-item model), retrained on a 709-item leakage-safe Case Bank (docs129 large-scale expansion) to test whether scaling causal-path supervision improves reasoning-aware retrieval over frozen semantic (BGE) retrieval.

Group-disjoint 5-fold CV results (bare-dialogueid grouping, fixing a grouping-key bug found in docs129): { "crossencoderfrozen": { "n": 709, "MATCH@1": 0.06770098730606489, "Hit@1": 0.06770098730606489, "Hit@3": 0.17489421720733428, "Hit@5": 0.2679830747531735, "MRR": 0.16755665214761337, "pertaskhit3": { "belief": 0.49019607843137253, "desire": 0.18110236220472442, "judgement": 0.10483870967741936, "prediction": 0.09375, "intention": 0.11764705882352941 }, "pertaskn": { "belief": 102, "desire": 127, "judgement": 248, "prediction": 96, "intention": 136 } }, "crossencoderfinetuned": { "n": 709, "MATCH@1": 0.08180535966149506, "Hit@1": 0.08180535966149506, "Hit@3": 0.2059238363892807, "Hit@5": 0.2947813822284908, "MRR": 0.1802352172741932, "pertaskhit3": { "belief": 0.3333333333333333, "desire": 0.14960629921259844, "judgement": 0.22580645161290322, "prediction": 0.07291666666666667, "intention": 0.22058823529411764 }, "pertask_n": { "belief": 102, "desire": 127, "judgement": 248, "prediction": 96, "intention": 136 } } }

McNemar (fine-tuned vs frozen): {'ftonlywins': 100, 'frozenonlywins': 78, 'ndiscordant': 178, 'pvalue': np.float64(0.11523195131755465)} Bootstrap 95% CI (Hit@3 delta): {'pointdelta': 0.03102961918194641, 'ci95lo': -0.004231311706629048, 'ci95_hi': 0.06770098730606489}

No causal path, gold answer, Direct/Causal outcome, or Benefit/Harm label is ever used as model input -- supervision-only, via the pair label.