tensorblock/Mathoctopus_Parallel_7B-GGUF
0117
1---2license: apache-2.03datasets:4- Mathoctopus/GSM8KInstruct_Parallel5language:6- en7- es8- zh9- de10- ru11- th12- sw13- ja14- fr15- bn16tags:17- TensorBlock18- GGUF19base_model: Mathoctopus/Parallel_7B20---21 22<div style="width: auto; margin-left: auto; margin-right: auto">23<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">24</div>25 26[](https://tensorblock.co)27[](https://twitter.com/tensorblock_aoi)28[](https://discord.gg/Ej5NmeHFf2)29[](https://github.com/TensorBlock)30[](https://t.me/TensorBlock)31 32 33## Mathoctopus/Parallel_7B - GGUF34 35This repo contains GGUF format model files for [Mathoctopus/Parallel_7B](https://huggingface.co/Mathoctopus/Parallel_7B).36 37The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b5165](https://github.com/ggml-org/llama.cpp/commit/1d735c0b4fa0551c51c2f4ac888dd9a01f447985).38 39## Our projects40<table border="1" cellspacing="0" cellpadding="10">41 <tr>42 <th colspan="2" style="font-size: 25px;">Forge</th>43 </tr>44 <tr>45 <th colspan="2">46 <img src="https://imgur.com/faI5UKh.jpeg" alt="Forge Project" width="900"/>47 </th>48 </tr>49 <tr>50 <th colspan="2">An OpenAI-compatible multi-provider routing layer.</th>51 </tr>52 <tr>53 <th colspan="2">54 <a href="https://github.com/TensorBlock/forge" target="_blank" style="55 display: inline-block;56 padding: 8px 16px;57 background-color: #FF7F50;58 color: white;59 text-decoration: none;60 border-radius: 6px;61 font-weight: bold;62 font-family: sans-serif;63 ">🚀 Try it now! 🚀</a>64 </th>65 </tr>66 67 <tr>68 <th style="font-size: 25px;">Awesome MCP Servers</th>69 <th style="font-size: 25px;">TensorBlock Studio</th>70 </tr>71 <tr>72 <th><img src="https://imgur.com/2Xov7B7.jpeg" alt="MCP Servers" width="450"/></th>73 <th><img src="https://imgur.com/pJcmF5u.jpeg" alt="Studio" width="450"/></th>74 </tr>75 <tr>76 <th>A comprehensive collection of Model Context Protocol (MCP) servers.</th>77 <th>A lightweight, open, and extensible multi-LLM interaction studio.</th>78 </tr>79 <tr>80 <th>81 <a href="https://github.com/TensorBlock/awesome-mcp-servers" target="_blank" style="82 display: inline-block;83 padding: 8px 16px;84 background-color: #FF7F50;85 color: white;86 text-decoration: none;87 border-radius: 6px;88 font-weight: bold;89 font-family: sans-serif;90 ">👀 See what we built 👀</a>91 </th>92 <th>93 <a href="https://github.com/TensorBlock/TensorBlock-Studio" target="_blank" style="94 display: inline-block;95 padding: 8px 16px;96 background-color: #FF7F50;97 color: white;98 text-decoration: none;99 border-radius: 6px;100 font-weight: bold;101 font-family: sans-serif;102 ">👀 See what we built 👀</a>103 </th>104 </tr>105</table>106 107## Prompt template108 109```110 111```112 113## Model file specification114 115| Filename | Quant type | File Size | Description |116| -------- | ---------- | --------- | ----------- |117| [Parallel_7B-Q2_K.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q2_K.gguf) | Q2_K | 2.533 GB | smallest, significant quality loss - not recommended for most purposes |118| [Parallel_7B-Q3_K_S.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q3_K_S.gguf) | Q3_K_S | 2.948 GB | very small, high quality loss |119| [Parallel_7B-Q3_K_M.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q3_K_M.gguf) | Q3_K_M | 3.298 GB | very small, high quality loss |120| [Parallel_7B-Q3_K_L.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q3_K_L.gguf) | Q3_K_L | 3.597 GB | small, substantial quality loss |121| [Parallel_7B-Q4_0.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q4_0.gguf) | Q4_0 | 3.826 GB | legacy; small, very high quality loss - prefer using Q3_K_M |122| [Parallel_7B-Q4_K_S.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q4_K_S.gguf) | Q4_K_S | 3.857 GB | small, greater quality loss |123| [Parallel_7B-Q4_K_M.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q4_K_M.gguf) | Q4_K_M | 4.081 GB | medium, balanced quality - recommended |124| [Parallel_7B-Q5_0.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q5_0.gguf) | Q5_0 | 4.652 GB | legacy; medium, balanced quality - prefer using Q4_K_M |125| [Parallel_7B-Q5_K_S.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q5_K_S.gguf) | Q5_K_S | 4.652 GB | large, low quality loss - recommended |126| [Parallel_7B-Q5_K_M.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q5_K_M.gguf) | Q5_K_M | 4.783 GB | large, very low quality loss - recommended |127| [Parallel_7B-Q6_K.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q6_K.gguf) | Q6_K | 5.529 GB | very large, extremely low quality loss |128| [Parallel_7B-Q8_0.gguf](https://huggingface.co/tensorblock/Mathoctopus_Parallel_7B-GGUF/blob/main/Parallel_7B-Q8_0.gguf) | Q8_0 | 7.161 GB | very large, extremely low quality loss - not recommended |129 130 131## Downloading instruction132 133### Command line134 135Firstly, install Huggingface Client136 137```shell138pip install -U "huggingface_hub[cli]"139```140 141Then, downoad the individual model file the a local directory142 143```shell144huggingface-cli download tensorblock/Mathoctopus_Parallel_7B-GGUF --include "Parallel_7B-Q2_K.gguf" --local-dir MY_LOCAL_DIR145```146 147If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:148 149```shell150huggingface-cli download tensorblock/Mathoctopus_Parallel_7B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'151```152