LLaMA2
Datasets
All datasets matching “LLaMA2”Llama-2-7b-KronQ-HG
Llama-2-7b — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Llama-2-7b, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. H_G is the per-sublayer sampled-Fisher gradient covariance (labels drawn from the model distribution) (E[g gᵀ] over the layer output), distinct from the standard input-side Hessian H_X (which GPTQ/GPTAQ build online during calibration).
Publishing this lets you… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Llama-2-7b-KronQ-HG.llama2_7b_chat-boolq-results
Dataset Card for "llama2_7b_chat-boolq-results"
More Information needed
fineweb-edu_default_Llama2_Tokenizer
fineweb-edu_default_Llama2_Tokenizer
The original fineweb-edu_default_Llama2_Tokenizer.tar.gz archive (≈1.9T on Ubuntu) was split into smaller 40 GB chunks for easier upload to Hugging Face.
sudo apt install git-lfs
pip install -U huggingface_hub # `hf version`==1.1.4
tar cvf - fineweb-edu_default_Llama2_Tokenizer/ | pigz -p 16 > fineweb-edu_default_Llama2_Tokenizer.tar.gz
split -b 40G -d -a 3 fineweb-edu_default_Llama2_Tokenizer.tar.gz… See the full description on the dataset page: https://huggingface.co/datasets/jsun/fineweb-edu_default_Llama2_Tokenizer.Llama-2-70b-KronQ-HG
Llama-2-70b-hf — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Llama-2-70b-hf, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. Per-sublayer empirical-Fisher gradient covariance (E[g gᵀ] over the layer output), distinct from the input-side Hessian H_X.
Publishing this lets you reproduce KronQ quantization without the offline Fisher precompute step.
Contents (80 layers… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Llama-2-70b-KronQ-HG.Llama-2-13b-KronQ-HG
Llama-2-13b — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Llama-2-13b, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. H_G is the per-sublayer empirical-Fisher gradient covariance (E[g gᵀ] over the layer output), distinct from the standard input-side Hessian H_X (built online during calibration).
Publishing this lets you reproduce KronQ quantization without the offline Fisher… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Llama-2-13b-KronQ-HG.slimpajama_Llama2_Tokenizer
slimpajama_Llama2_Tokenizer
The original slimpajama_Llama2_Tokenizer.tar.gz archive (≈794 GB on Ubuntu) was split into smaller 40 GB chunks for easier upload to Hugging Face.
sudo apt install git-lfs
pip install -U huggingface_hub # `hf version`==1.1.4
tar cvf - slimpajama_Llama2_Tokenizer/ | pigz -p 16 > slimpajama_Llama2_Tokenizer.tar.gz
split -b 40G -d -a 3 slimpajama_Llama2_Tokenizer.tar.gz slimpajama_Llama2_Tokenizer/slimpajama_Llama2_Tokenizer_part_
# Upload files… See the full description on the dataset page: https://huggingface.co/datasets/jsun/slimpajama_Llama2_Tokenizer.