proofnet
Datasets
All datasets matching “proofnet”proofnetA dataset that evaluates formally proving and autoformalizing undergraduate mathematics.ProofNetSharp
ProofNet#
ProofNet# is a Lean 4 port of the ProofNet benchmark including fixes.
A comparison with previous Lean 4 ports can be found at:
https://proofnet4-fix.streamlit.app/.
This benchmark is compatible with all Lean versions between v4.7.0 and v4.16.0-rc2.
Original Dataset Summary
ProofNet is a benchmark for autoformalization and formal proving of undergraduate-level mathematics. The ProofNet benchmarks consists of 371 examples, each consisting of a formal theorem… See the full description on the dataset page: https://huggingface.co/datasets/PAug/ProofNetSharp.minif2f_proofnet_lean_workbook_deepseek_prover_solsminif2f_proofnet_deepseek_prover_solsProofNetVerif
ProofNetVerif
ProofNetVerif is a benchmark to evaluate both reference-based and reference-free metrics for statement autoformalization introduced in
Improving Autoformalization using Type Checking. This benchmark is compatible with Lean v4.8.0.
Tasks
Reference-based metric evaluation:
Input: lean4_formalization, lean4_prediction
Output: correct
Reference-free metric evaluation:
Input: nl_statement, lean4_prediction
Output: correct
Note: Developing an accurate… See the full description on the dataset page: https://huggingface.co/datasets/PAug/ProofNetVerif.proofnet-lean4