CoolFace
Modelpublic

gaianet/Llama-3-8B-Japanese-Instruct-GGUF

sourceHugging Faceotherupdated 2y agoView on Hugging Face
2likes663downloads
Model Card

<!-- header start --> <!-- 200823 --> <div style="width: auto; margin-left: auto; margin-right: auto"> <img src="https://github.com/GaiaNet-AI/.github/assets/45785633/d6976adc-f97d-4f86-a648-0f2f5c8e7eee" style="width: 100%; min-width: 400px; display: block; margin: auto;"> </div> <hr style="margin-top: 1.0em; margin-bottom: 1.0em;"> <!-- header end -->

Llama-3-8B-Japanese-Instruct-GGUF

Original Model

haqishen/Llama-3-8B-Japanese-Instruct

Run with Gaianet

Prompt template:

prompt template: llama-3-chat

Context size:

chatctxsize: 4096

Run with GaiaNet:

  • —Quick start: https://docs.gaianet.ai/node-guide/quick-start
  • —Customize your node: https://docs.gaianet.ai/node-guide/customize

Quantized GGUF Models

NameQuant methodBitsSizeUse case
Llama-3-8B-Japanese-Instruct-Q2_K.ggufQ2_K23.18 GBsmallest, significant quality loss - not recommended for most purposes
Llama-3-8B-Japanese-Instruct-Q3_K_L.ggufQ3KL34.32 GBsmall, substantial quality loss
Llama-3-8B-Japanese-Instruct-Q3_K_M.ggufQ3KM34.02 GBvery small, high quality loss
Llama-3-8B-Japanese-Instruct-Q3_K_S.ggufQ3KS33.66 GBvery small, high quality loss
Llama-3-8B-Japanese-Instruct-Q4_0.ggufQ4_044.66 GBlegacy; small, very high quality loss - prefer using Q3KM
Llama-3-8B-Japanese-Instruct-Q4_K_M.ggufQ4KM44.92 GBmedium, balanced quality - recommended
Llama-3-8B-Japanese-Instruct-Q4_K_S.ggufQ4KS44.69 GBsmall, greater quality loss
Llama-3-8B-Japanese-Instruct-Q5_0.ggufQ5_055.6 GBlegacy; medium, balanced quality - prefer using Q4KM
Llama-3-8B-Japanese-Instruct-Q5_K_M.ggufQ5KM55.73 GBlarge, very low quality loss - recommended
Llama-3-8B-Japanese-Instruct-Q5_K_S.ggufQ5KS55.6 GBlarge, low quality loss - recommended
Llama-3-8B-Japanese-Instruct-Q6_K.ggufQ6_K66.6 GBvery large, extremely low quality loss
Llama-3-8B-Japanese-Instruct-Q8_0.ggufQ8_088.54 GBvery large, extremely low quality loss - not recommended
Llama-3-8B-Japanese-Instruct-f16.gguff161616.1 GB

Quantized with llama.cpp b2824.