danny2507/ragu-webq-mistral7b-uq-results
RAGU WebQ Mistral-7B uncertainty results Reproducibility artifacts for a WebQuestions uncertainty-quantification comparison using the frozen RAGU 400-example seed-10 subset, top-5 Contriever-MSMARCO contexts, and mistralai/Mistral-7B-Instruct-v0.3. Contents results/: per-example JSONL outputs and summary JSON files for PPL, regular entropy, RAGU semantic entropy, this project's claim-level semantic uncertainty, token uncertainty, and p(True). code/: standalone… See the full description on the dataset page: https://huggingface.co/datasets/danny2507/ragu-webq-mistral7b-uq-results.
RAGU WebQ Mistral-7B uncertainty results
Reproducibility artifacts for a WebQuestions uncertainty-quantification comparison using the frozen RAGU 400-example seed-10 subset, top-5 Contriever-MSMARCO contexts, and mistralai/Mistral-7B-Instruct-v0.3.
Contents
results/: per-example JSONL outputs and summary JSON files for PPL, regular entropy, RAGU semantic entropy, this project's claim-level semantic uncertainty, token uncertainty, and p(True).code/: standalone scripts used to generate and score the artifacts.
Protocol
All UQ scores consume the same saved greedy answer and ten sampled answers. The primary label in these artifacts is RAGU raw Acc (normalized gold-answer containment), not the paper's Qwen-based AccLM. The results therefore support an internally paired comparison, but are not a direct reproduction of the paper's AccLM Table-1 values.
The p(True) run uses regenerated seed-10 training demonstrations with raw-Acc labels. It should be treated as a raw-Acc reproduction rather than the paper's exact AccLM-supervised p(True) configuration.