Modelpublic

lewishamilton21/Qwen_1.5B_multilingual_Fine-Tuned_LLM

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

3likes10downloads

Model Card

Hugging Face Model Card (README.md)

`markdown

---
license: apache-2.0
tags:
- causal-lm
- text-generation
- chatbot
- qwen
- deepseek
- lora
- 4bit
- bitsandbytes
library_name: transformers
pipeline_tag: text-generation
quantized: true
base_model: Qwen/Qwen2.5-1.5B-Instruct
---

Qwen_1.5B_multilingual_Fine-Tuned_LLM — LoRA 4-bit Fine-Tuned Model

This is a conversational language model based on [Qwen/Qwen2.5-1.5B-Instruct](: https://huggingface.co/Gensyn/Qwen2.5-1.5B-Instruct) fine-tuned with [LoRA adapters](https://github.com/huggingface/peft) for efficient training and inference. The model is loaded using **4-bit quantization (NF4)** through [BitsAndBytes](https://github.com/TimDettmers/bitsandbytes), enabling memory-efficient inference on consumer-grade GPUs.

---

##  Model Details

- **Base model**: `: 'Qwen2.5-1.5B-Instruct`
- **Fine-tuning technique**: LoRA (Low-Rank Adaptation)
- **Quantization**: 4-bit NF4 via BitsAndBytes
- **Framework**: Hugging Face Transformers + PEFT
- **Pipeline**: `text-generation`

---

##  Intended Use

This model is designed for **multi-turn chatbot applications**, creative writing, instruction following, and general-purpose text generation tasks within responsible use guidelines.

---

##  Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

modelname = "lewishamilton21/Qwen1.5BmultilingualFine-Tuned_LLM"

bnbconfig = BitsAndBytesConfig( loadin4bit=True, bnb4bitusedoublequant=True, bnb4bitquanttype="nf4", bnb4bitcompute_dtype="float16" )

tokenizer = AutoTokenizer.frompretrained(modelname) model = AutoModelForCausalLM.frompretrained( modelname, quantizationconfig=bnbconfig, device_map="auto" )

inputs = tokenizer("Hello, how are you today?", returntensors="pt").to("cuda") outputs = model.generate(**inputs, maxnewtokens=100) print(tokenizer.decode(outputs[0], skipspecial_tokens=True))


---

##  Evaluation Metrics

| Metric            | Value (example) |
| :---------------- | :-------------- |
| Quantization Type | 4-bit NF4       |
| LoRA Rank         | 8 or 16         |
| Max Length Tested | 2048 tokens     |
| VRAM (A100 40GB)  | \~3.5 GB        |

*Custom benchmarks coming soon.*

---

##  Training & Fine-Tuning

Fine-tuned via LoRA adapters using PEFT. To reproduce:

from peft import getpeftmodel, LoraConfig, preparemodelforkbittraining from transformers import TrainingArguments, Trainer

Load model in 4bit

model = AutoModelForCausalLM.frompretrained( basemodelid, quantizationconfig=bnbconfig, devicemap="auto" )

model = preparemodelforkbittraining(model) model = getpeftmodel(model, LoraConfig(...))

Trainer setup

trainer = Trainer( model=model, args=TrainingArguments(...), train_dataset=dataset ) trainer.train()


---

##  License

Apache 2.0 — free for research and commercial use within the license terms.

---



##  Acknowledgements

* DeepSeek AI
* Hugging Face Transformers
* BitsAndBytes by Tim Dettmers
* Hugging Face PEFT

---