llama-2
Datasets
All datasets matching “llama-2”Llama-2-7b-KronQ-HG
Llama-2-7b — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Llama-2-7b, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. H_G is the per-sublayer sampled-Fisher gradient covariance (labels drawn from the model distribution) (E[g gᵀ] over the layer output), distinct from the standard input-side Hessian H_X (which GPTQ/GPTAQ build online during calibration).
Publishing this lets you… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Llama-2-7b-KronQ-HG.llama2_7b_chat-boolq-results
Dataset Card for "llama2_7b_chat-boolq-results"
More Information needed
fineweb-edu_default_Llama2_Tokenizer
fineweb-edu_default_Llama2_Tokenizer
The original fineweb-edu_default_Llama2_Tokenizer.tar.gz archive (≈1.9T on Ubuntu) was split into smaller 40 GB chunks for easier upload to Hugging Face.
sudo apt install git-lfs
pip install -U huggingface_hub # `hf version`==1.1.4
tar cvf - fineweb-edu_default_Llama2_Tokenizer/ | pigz -p 16 > fineweb-edu_default_Llama2_Tokenizer.tar.gz
split -b 40G -d -a 3 fineweb-edu_default_Llama2_Tokenizer.tar.gz… See the full description on the dataset page: https://huggingface.co/datasets/jsun/fineweb-edu_default_Llama2_Tokenizer.prepacked-fineweb-edu-llama2-32K-T2048
prepacked-fineweb-edu-llama2-32K-T2048
Pre-tokenized and BOS-aligned best-fit packed version of FineWeb-Edu for training with looped nanochat.
Tokenized with the Llama 2 tokenizer (32,000 base vocab + 8 special tokens = 32,008).
Stats
Train split
Source
karpathy/fineweb-edu-100b-shuffle (1,821 shards)
Total tokens
63.26B
Total docs
97.1M
Rows
30,873,598
Shards
2,059 (train-00000 to train-02058)
Rows per shard
~15,000… See the full description on the dataset page: https://huggingface.co/datasets/KristianS7/prepacked-fineweb-edu-llama2-32K-T2048.llama2_7b_chat-piqa-resultsLlama-2-70b-KronQ-HG
Llama-2-70b-hf — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Llama-2-70b-hf, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. Per-sublayer empirical-Fisher gradient covariance (E[g gᵀ] over the layer output), distinct from the input-side Hessian H_X.
Publishing this lets you reproduce KronQ quantization without the offline Fisher precompute step.
Contents (80 layers… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Llama-2-70b-KronQ-HG.