neuralmagic
neuralmagic_-_Llama-2-7b-ultrachat200k-ggufneuralmagic_-_SparseLlama-3-8B-pruned_50.2of4-ggufneuralmagic_-_Llama-2-7b-evolcodealpaca-ggufneuralmagic_-_Llama-2-7b-pruned50-retrained-ggufneuralmagic_-_Llama-2-7b-dolphin-open_platypus-pruned_70-ggufneuralmagic_-_Llama-2-7b-pruned70-retrained-ggufneuralmagic-SparseLlama-3-8B-pruned_50.2of4-GGUFneuralmagic-Sparse-Llama-3.1-8B-ultrachat_200k-2of4-GGUF
quantized-llama-3.1-leaderboard-v2-evals
Open LLM Leaderboard v2 Benchmark Results
This artifact contains all the data from evaluations of Neural Magic's quantized Llama-3.1 models.
These evaluations were produced with lm-evaluation-harness by running the following command:
lm_eval \
--model vllm \
--model_args pretrained="<model_path>",dtype=auto,add_bos_token=False,max_model_len=4096,tensor_parallel_size="<num_gpus>",gpu_memory_utilization=0.8,enable_chunked_prefill=True \
--apply_chat_template \… See the full description on the dataset page: https://huggingface.co/datasets/neuralmagic/quantized-llama-3.1-leaderboard-v2-evals.LLM_compression_calibration
LLM Compression Calibration dataset
This dataset is the default calibration dataset used by Neural Magic for one-shot compression of Large Language Models (LLMs).
Note: This dataset is the result of active research and subject to change without notice.
Dataset Details
Dataset Sources
The current version of this dataset is compiled from data from these datasets:
garage-bAInd/Open-Platypus: 10,000 samples
Data Fields
The dataset contains 2 data… See the full description on the dataset page: https://huggingface.co/datasets/neuralmagic/LLM_compression_calibration.mmlu_itmmlu_demmlu_frcalibration