TokenBender/llama2-7b-chat-hf-codeCherryPop-qLoRA-merged

Overview:

description:

This is a llama2 7B HF chat model fine-tuned on 122k code instructions. In my early experiments it seems to be doing very well.

additional_info:

It's a bottom of the barrel model 😂 but after quantization it can be valuable for sure. It definitely proves that a 7B can be useful for boilerplate code stuff though.

Plans:

next_steps: "I've a few things in mind and after that this will be more valuable."

tasks:

• —name: "I'll quantize these" timeline: "Possibly tonight or tomorrow in the day" result: "Then it can be run locally with 4G ram."
• —name: "I've used alpaca style instruction tuning" improvement: | I'll switch to llama2 style [INST]<<SYS>> style and see if it improves anything.
• —name: "HumanEval report and checking for any training data leaks"
• —attempt: "I'll try 8k context via RoPE enhancement" hypothesis: "Let's see if that degrades performance or not." commercialuse: | So far I think this can be used commercially but this is a adapter on Meta's llama2 with some gating issues so that is there. contactinfo: "If you find any issues or want to just holler at me, you can reach out to me - https://twitter.com/4evaBehindSOTA"

Library:

name: "peft"

Training procedure:

quantizationconfig: loadin8bit: False loadin4bit: True llmint8threshold: 6.0 llmint8skipmodules: None llmint8enablefp32cpuoffload: False llmint8hasfp16weight: False bnb4bitquanttype: "nf4" bnb4bitusedoublequant: False bnb4bitcompute_dtype: "float16"

Framework versions:

PEFT: "0.5.0.dev0"