omrifahn/kfac-memorization-code

K-FAC Memorization Suppression Reproduction of "From Memorization to Reasoning in the Spectrum of Loss Curvature" with extended experiments on modified importance formulas. Overview This project implements K-FAC (Kronecker-Factored Approximate Curvature) based weight editing to suppress verbatim memorization in language models while preserving general capabilities. Key insight: The Fisher Information Matrix, approximated by K-FAC, reveals directions in weight… See the full description on the dataset page: https://huggingface.co/datasets/omrifahn/kfac-memorization-code.