Apppublic

SabaPivot/repro-differentiable-conformal-training-for-llm-reasoning-factuality

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes

App README

Native released-data reproduction

The exact official DCF implementation at commit 0b4d5487a9868a18c4f9aa5b3d96cdccc705ca97 runs on all 50 released MATH reasoning graphs (503 claim nodes). The audit also recomputes ten 14,600-decision agreement rows, the MATH/FELM headline metrics, five released calibration suites, two prediction suites, an end-to-end gradient, and an ancestor-removal destructive control.

Claim 1 is directly falsified as a composite reliability claim: its retention gain is real, but released coverage misses the 97% target. Claims 2–6 are verified within their exact source scope. No paper-scale result was invented, substituted, or repaired with a nearby result.

bash

python3 -m pip install -r requirements.txt
PYTHONDONTWRITEBYTECODE=1 PYTHONHASHSEED=0 PYTHONWARNINGS=error python3 native_release_audit.py --output-dir /tmp/dcf-native
PYTHONDONTWRITEBYTECODE=1 PYTHONHASHSEED=0 PYTHONWARNINGS=error python3 validate_evidence.py
PYTHONDONTWRITEBYTECODE=1 python3 verify_manifest.py

The semantic-quality-gate v4 forecast is 12 verified points locally. Judge points count only after a persisted exact-current-SHA verdict.