QuantFactory/granite-3b-code-base-2k-GGUF
2917
1 2---3 4pipeline_tag: text-generation5inference: false6license: apache-2.07datasets:8- codeparrot/github-code-clean9- bigcode/starcoderdata10# - Stackexchange11# - CommonCrawl12- open-web-math/open-web-math13- math-ai/StackMathQA14# - Arxiv15# - Wikipedia16# - conceptofmind/FLAN_2022 # Original link is broken, we used IBM's filtered version17metrics:18- code_eval19library_name: transformers20tags:21- code22- granite23model-index:24- name: granite-3b-code-base-2k25 results:26 - task:27 type: text-generation28 dataset:29 type: mbpp30 name: MBPP31 metrics:32 - name: pass@133 type: pass@134 value: 36.035 veriefied: false 36 - task:37 type: text-generation38 dataset:39 type: evalplus/mbppplus 40 name: MBPP+41 metrics:42 - name: pass@143 type: pass@144 value: 45.145 veriefied: false46 - task:47 type: text-generation48 dataset:49 type: bigcode/humanevalpack 50 name: HumanEvalSynthesis(Python)51 metrics:52 - name: pass@153 type: pass@154 value: 36.655 veriefied: false56 - task:57 type: text-generation58 dataset:59 type: bigcode/humanevalpack 60 name: HumanEvalSynthesis(JavaScript)61 metrics:62 - name: pass@163 type: pass@164 value: 37.265 veriefied: false66 - task:67 type: text-generation68 dataset:69 type: bigcode/humanevalpack 70 name: HumanEvalSynthesis(Java)71 metrics:72 - name: pass@173 type: pass@174 value: 40.975 veriefied: false76 - task:77 type: text-generation78 dataset:79 type: bigcode/humanevalpack 80 name: HumanEvalSynthesis(Go)81 metrics:82 - name: pass@183 type: pass@184 value: 26.285 veriefied: false86 - task:87 type: text-generation88 dataset:89 type: bigcode/humanevalpack 90 name: HumanEvalSynthesis(C++)91 metrics:92 - name: pass@193 type: pass@194 value: 35.495 veriefied: false96 - task:97 type: text-generation98 dataset:99 type: bigcode/humanevalpack 100 name: HumanEvalSynthesis(Rust)101 metrics:102 - name: pass@1103 type: pass@1104 value: 22.0105 veriefied: false106 - task:107 type: text-generation108 dataset:109 type: bigcode/humanevalpack 110 name: HumanEvalExplain(Python)111 metrics:112 - name: pass@1113 type: pass@1114 value: 25.0115 veriefied: false116 - task:117 type: text-generation118 dataset:119 type: bigcode/humanevalpack 120 name: HumanEvalExplain(JavaScript)121 metrics:122 - name: pass@1123 type: pass@1124 value: 18.9125 veriefied: false126 - task:127 type: text-generation128 dataset:129 type: bigcode/humanevalpack 130 name: HumanEvalExplain(Java)131 metrics:132 - name: pass@1133 type: pass@1134 value: 29.9135 veriefied: false136 - task:137 type: text-generation138 dataset:139 type: bigcode/humanevalpack 140 name: HumanEvalExplain(Go)141 metrics:142 - name: pass@1143 type: pass@1144 value: 17.1145 veriefied: false146 - task:147 type: text-generation148 dataset:149 type: bigcode/humanevalpack 150 name: HumanEvalExplain(C++)151 metrics:152 - name: pass@1153 type: pass@1154 value: 26.8155 veriefied: false156 - task:157 type: text-generation158 dataset:159 type: bigcode/humanevalpack 160 name: HumanEvalExplain(Rust)161 metrics:162 - name: pass@1163 type: pass@1164 value: 14.0165 veriefied: false166 - task:167 type: text-generation168 dataset:169 type: bigcode/humanevalpack 170 name: HumanEvalFix(Python)171 metrics:172 - name: pass@1173 type: pass@1174 value: 18.3175 veriefied: false176 - task:177 type: text-generation178 dataset:179 type: bigcode/humanevalpack 180 name: HumanEvalFix(JavaScript)181 metrics:182 - name: pass@1183 type: pass@1184 value: 23.2185 veriefied: false186 - task:187 type: text-generation188 dataset:189 type: bigcode/humanevalpack 190 name: HumanEvalFix(Java)191 metrics:192 - name: pass@1193 type: pass@1194 value: 29.9195 veriefied: false196 - task:197 type: text-generation198 dataset:199 type: bigcode/humanevalpack 200 name: HumanEvalFix(Go)201 metrics:202 - name: pass@1203 type: pass@1204 value: 24.4205 veriefied: false206 - task:207 type: text-generation208 dataset:209 type: bigcode/humanevalpack 210 name: HumanEvalFix(C++)211 metrics:212 - name: pass@1213 type: pass@1214 value: 16.5 215 veriefied: false216 - task:217 type: text-generation218 dataset:219 type: bigcode/humanevalpack 220 name: HumanEvalFix(Rust)221 metrics:222 - name: pass@1223 type: pass@1224 value: 3.7225 veriefied: false226 227---228 229230 231# QuantFactory/granite-3b-code-base-2k-GGUF232This is quantized version of [ibm-granite/granite-3b-code-base-2k](https://huggingface.co/ibm-granite/granite-3b-code-base-2k) created using llama.cpp233 234# Original Model Card235 236 237238 239# Granite-3B-Code-Base-2K240 241## Model Summary242**Granite-3B-Code-Base-2K** is a decoder-only code model designed for code generative tasks (e.g., code generation, code explanation, code fixing, etc.). It is trained from scratch with a two-phase training strategy. In phase 1, our model is trained on 4 trillion tokens sourced from 116 programming languages, ensuring a comprehensive understanding of programming languages and syntax. In phase 2, our model is trained on 500 billion tokens with a carefully designed mixture of high-quality data from code and natural language domains to improve the models’ ability to reason and follow instructions.243 244- **Developers:** IBM Research245- **GitHub Repository:** [ibm-granite/granite-code-models](https://github.com/ibm-granite/granite-code-models)246- **Paper:** [Granite Code Models: A Family of Open Foundation Models for Code Intelligence](https://arxiv.org/abs/2405.04324)247- **Release Date**: May 6th, 2024248- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).249 250## Usage251### Intended use252Prominent enterprise use cases of LLMs in software engineering productivity include code generation, code explanation, code fixing, generating unit tests, generating documentation, addressing technical debt issues, vulnerability detection, code translation, and more. All Granite Code Base models, including the **3B parameter model**, are able to handle these tasks as they were trained on a large amount of code data from 116 programming languages. 253 254### Generation255This is a simple example of how to use **Granite-3B-Code-Base-2K** model.256 257```python258import torch259from transformers import AutoModelForCausalLM, AutoTokenizer260device = "cuda" # or "cpu"261model_path = "ibm-granite/granite-3b-code-base-2k"262tokenizer = AutoTokenizer.from_pretrained(model_path)263# drop device_map if running on CPU264model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)265model.eval()266# change input text as desired267input_text = "def generate():"268# tokenize the text269input_tokens = tokenizer(input_text, return_tensors="pt")270# transfer tokenized inputs to the device271for i in input_tokens:272 input_tokens[i] = input_tokens[i].to(device)273# generate output tokens274output = model.generate(**input_tokens)275# decode output tokens into text276output = tokenizer.batch_decode(output)277# loop over the batch to print, in this example the batch size is 1278for i in output:279 print(i)280```281 282## Training Data283- **Data Collection and Filtering:** Pretraining code data is sourced from a combination of publicly available datasets (e.g., [GitHub Code Clean](https://huggingface.co/datasets/codeparrot/github-code-clean), [Starcoder data](https://huggingface.co/datasets/bigcode/starcoderdata)), and additional public code repositories and issues from GitHub. We filter raw data to retain a list of 116 programming languages. After language filtering, we also filter out low-quality code. 284- **Exact and Fuzzy Deduplication:** We adopt an aggressive deduplication strategy that includes both exact and fuzzy deduplication to remove documents having (near) identical code content.285- **HAP, PII, Malware Filtering:** We apply a HAP content filter that reduces models' likelihood of generating hateful, abusive, or profane language. We also make sure to redact Personally Identifiable Information (PII) by replacing PII content (e.g., names, email addresses, keys, passwords) with corresponding tokens (e.g., ⟨NAME⟩, ⟨EMAIL⟩, ⟨KEY⟩, ⟨PASSWORD⟩). Moreover, we scan all datasets using [ClamAV](https://www.clamav.net/) to identify and remove instances of malware in the source code.286- **Natural Language Datasets:** In addition to collecting code data for model training, we curate several publicly available high-quality natural language datasets to improve models' proficiency in language understanding and mathematical reasoning. Unlike the code data, we do not deduplicate these datasets.287 288## Infrastructure289We train the Granite Code models using two of IBM's super computing clusters, namely Vela and Blue Vela, both outfitted with NVIDIA A100 and H100 GPUs respectively. These clusters provide a scalable and efficient infrastructure for training our models over thousands of GPUs.290 291## Ethical Considerations and Limitations292The use of Large Language Models involves risks and ethical considerations people must be aware of. Regarding code generation, caution is urged against complete reliance on specific code models for crucial decisions or impactful information as the generated code is not guaranteed to work as intended. **Granite-3B-Code-Base-2K** model is not the exception in this regard. Even though this model is suited for multiple code-related tasks, it has not undergone any safety alignment, there it may produce problematic outputs. Additionally, it remains uncertain whether smaller models might exhibit increased susceptibility to hallucination in generation scenarios by copying source code verbatim from the training dataset due to their reduced sizes and memorization capacities. This aspect is currently an active area of research, and we anticipate more rigorous exploration, comprehension, and mitigations in this domain. Regarding ethics, a latent risk associated with all Large Language Models is their malicious utilization. We urge the community to use **Granite-3B-Code-Base-2K** model with ethical intentions and in a responsible way. 293 294