tensorblock/hkcode_llama3_8b-GGUF
097
1---2license: mit3datasets:4- hyokwan/llama3data_hkcode5language:6- ko7- en8metrics:9- accuracy10library_name: transformers11pipeline_tag: text-generation12tags:13- education14- fintech15- TensorBlock16- GGUF17base_model: hyokwan/hkcode_llama3_8b18---19 20<div style="width: auto; margin-left: auto; margin-right: auto">21<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">22</div>23 24[](https://tensorblock.co)25[](https://twitter.com/tensorblock_aoi)26[](https://discord.gg/Ej5NmeHFf2)27[](https://github.com/TensorBlock)28[](https://t.me/TensorBlock)29 30 31## hyokwan/hkcode_llama3_8b - GGUF32 33This repo contains GGUF format model files for [hyokwan/hkcode_llama3_8b](https://huggingface.co/hyokwan/hkcode_llama3_8b).34 35The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).36 37## Our projects38<table border="1" cellspacing="0" cellpadding="10">39 <tr>40 <th colspan="2" style="font-size: 25px;">Forge</th>41 </tr>42 <tr>43 <th colspan="2">44 <img src="https://imgur.com/faI5UKh.jpeg" alt="Forge Project" width="900"/>45 </th>46 </tr>47 <tr>48 <th colspan="2">An OpenAI-compatible multi-provider routing layer.</th>49 </tr>50 <tr>51 <th colspan="2">52 <a href="https://github.com/TensorBlock/forge" target="_blank" style="53 display: inline-block;54 padding: 8px 16px;55 background-color: #FF7F50;56 color: white;57 text-decoration: none;58 border-radius: 6px;59 font-weight: bold;60 font-family: sans-serif;61 ">🚀 Try it now! 🚀</a>62 </th>63 </tr>64 65 <tr>66 <th style="font-size: 25px;">Awesome MCP Servers</th>67 <th style="font-size: 25px;">TensorBlock Studio</th>68 </tr>69 <tr>70 <th><img src="https://imgur.com/2Xov7B7.jpeg" alt="MCP Servers" width="450"/></th>71 <th><img src="https://imgur.com/pJcmF5u.jpeg" alt="Studio" width="450"/></th>72 </tr>73 <tr>74 <th>A comprehensive collection of Model Context Protocol (MCP) servers.</th>75 <th>A lightweight, open, and extensible multi-LLM interaction studio.</th>76 </tr>77 <tr>78 <th>79 <a href="https://github.com/TensorBlock/awesome-mcp-servers" target="_blank" style="80 display: inline-block;81 padding: 8px 16px;82 background-color: #FF7F50;83 color: white;84 text-decoration: none;85 border-radius: 6px;86 font-weight: bold;87 font-family: sans-serif;88 ">👀 See what we built 👀</a>89 </th>90 <th>91 <a href="https://github.com/TensorBlock/TensorBlock-Studio" target="_blank" style="92 display: inline-block;93 padding: 8px 16px;94 background-color: #FF7F50;95 color: white;96 text-decoration: none;97 border-radius: 6px;98 font-weight: bold;99 font-family: sans-serif;100 ">👀 See what we built 👀</a>101 </th>102 </tr>103</table>104## Prompt template105 106```107<|begin_of_text|><|start_header_id|>system<|end_header_id|>108 109{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>110 111{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>112```113 114## Model file specification115 116| Filename | Quant type | File Size | Description |117| -------- | ---------- | --------- | ----------- |118| [hkcode_llama3_8b-Q2_K.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q2_K.gguf) | Q2_K | 3.179 GB | smallest, significant quality loss - not recommended for most purposes |119| [hkcode_llama3_8b-Q3_K_S.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q3_K_S.gguf) | Q3_K_S | 3.664 GB | very small, high quality loss |120| [hkcode_llama3_8b-Q3_K_M.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q3_K_M.gguf) | Q3_K_M | 4.019 GB | very small, high quality loss |121| [hkcode_llama3_8b-Q3_K_L.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q3_K_L.gguf) | Q3_K_L | 4.322 GB | small, substantial quality loss |122| [hkcode_llama3_8b-Q4_0.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q4_0.gguf) | Q4_0 | 4.661 GB | legacy; small, very high quality loss - prefer using Q3_K_M |123| [hkcode_llama3_8b-Q4_K_S.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q4_K_S.gguf) | Q4_K_S | 4.693 GB | small, greater quality loss |124| [hkcode_llama3_8b-Q4_K_M.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q4_K_M.gguf) | Q4_K_M | 4.921 GB | medium, balanced quality - recommended |125| [hkcode_llama3_8b-Q5_0.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q5_0.gguf) | Q5_0 | 5.599 GB | legacy; medium, balanced quality - prefer using Q4_K_M |126| [hkcode_llama3_8b-Q5_K_S.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q5_K_S.gguf) | Q5_K_S | 5.599 GB | large, low quality loss - recommended |127| [hkcode_llama3_8b-Q5_K_M.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q5_K_M.gguf) | Q5_K_M | 5.733 GB | large, very low quality loss - recommended |128| [hkcode_llama3_8b-Q6_K.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q6_K.gguf) | Q6_K | 6.596 GB | very large, extremely low quality loss |129| [hkcode_llama3_8b-Q8_0.gguf](https://huggingface.co/tensorblock/hkcode_llama3_8b-GGUF/blob/main/hkcode_llama3_8b-Q8_0.gguf) | Q8_0 | 8.541 GB | very large, extremely low quality loss - not recommended |130 131 132## Downloading instruction133 134### Command line135 136Firstly, install Huggingface Client137 138```shell139pip install -U "huggingface_hub[cli]"140```141 142Then, downoad the individual model file the a local directory143 144```shell145huggingface-cli download tensorblock/hkcode_llama3_8b-GGUF --include "hkcode_llama3_8b-Q2_K.gguf" --local-dir MY_LOCAL_DIR146```147 148If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:149 150```shell151huggingface-cli download tensorblock/hkcode_llama3_8b-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'152```153