CoolFace
Modelpublic

PhilLovesCats/LLaMA-1-7B-GGUF

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes71downloads
Model Card

LLaMA 1 7B - GGUF

  • —Model creator: Meta Llama
  • —The 'Provided files'-section layout, as well as the metadata layout are heavily inspired from [TheBloke](https://huggingface.co/TheBloke)
  • —The 'License'-section, aswell as the 'LICENSE'-file are burrowed by [huggyllama](https://huggingface.co/huggyllama)

Please tell me when i have violated your copyright, so that i can remove the belonging section!

Description

Due the fact that the old LLaMA (1) models are in a deprecated format and really hard to use with modern tools, i converted them using the transformers library's convert_llama_weights_to_hf.py script to convert them into a PyTorch model.

Further i used the convert_hf_to_gguf.py script from llama.cpp and to bring them into the well-known GGUF file format. Afterwards, i've build llama.cpp and ran llama-quantize.exe to downscale the model files.

Provided files

The following table only lists the quantized (.gguf) files (without the PyTorch (.bin) ones).

NameQuant methodBitsSizeMD5-HashUse case
llama-1-7b.TQ1_0.ggufTQ1_011.76 GB188467F7FDD001BBBA244071E5EBF05Every small, significant quality loss - not recommended for most purposes
llama-1-7b.TQ2_0.ggufTQ2_022.04 GB8C0A29B4068CD2016A5DF0924B6F8FCFvery small, significant quality loss - not recommended for most purposes
llama-1-7b.Q2_K.ggufQ2_K22.35 GB5D371F8ADE1ADE8FBFA6F101A4489241smallest, significant quality loss - not recommended for most purposes
llama-1-7b.Q3_K_S.ggufQ3KS32.74 GB301B3495B61D1B9115F5E7BBAB2270A8very small, high quality loss
llama-1-7b.Q3_K_M.ggufQ3KM33.07 GB3755C9BCC82E1B453CB87EA3A9E2C84Dvery small, high quality loss
llama-1-7b.Q3_K_L.ggufQ3KL33.34 GB356CAE2C39D0085EECA6566C10F59235small, substantial quality loss
llama-1-7b.Q4_0.ggufQ4_043.56 GBD741CAC6ADA6C8B8B8C493F536A54B12legacy; small, very high quality loss - prefer using Q3KM
llama-1-7b.Q4_K_S.ggufQ4KS43.59 GBA84887EFC1C6461B2166E16F6A468C17small, greater quality loss
llama-1-7b.Q4_K_M.ggufQ4KM43.79 GB04B6ECFAD36D9B57C756E0A13310341Emedium, balanced quality - recommended
llama-1-7b.Q5_0.ggufQ5_054.33 GB7AA2E9BBF47D2C2F74B71D87C6D81EC6legacy; medium, balanced quality - prefer using Q4KM
llama-1-7b.Q5_K_S.ggufQ5KS54.33 GBF6EE8274C8D2D63AA2DEC2F8E6856AE9large, low quality loss - recommended
llama-1-7b.Q5_K_M.ggufQ5KM54.45 GB2B7477EB0D86394831F39A8B9BFCD11Dlarge, very low quality loss - recommended
llama-1-7b.Q6_K.ggufQ6_K65.14 GB3B2A3C6329954A3BAD867C4A41AD578Avery large, extremely low quality loss
llama-1-7b.Q8_0.ggufQ8_086.66 GB0D757DD0FC2000D745FE5BE6DDD033B2very large, extremely low quality loss - not recommended
llama-1-7b.F16.ggufF161612.50 GBD866619C36114FF150C43B955B1F2729original, no quality loss - not recommended
llama-1-7b.BF16.ggufBF161612.50 GBB045E6BC11B19A2039CCA0709FE1C953original, no quality loss - not recommended

Notice: Hashes were calculated with Get-FileHash .\FILENAME -Algorithm MD5 on Microsoft Windows 11 64-bit (x86_x64).

License

This model is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form but either lost your copy of the weights or got some trouble converting them to the Transformers format.