Modelpublic

ewideplus/gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes10downloads

Model Card

gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7

This is a LoRA adapter fine-tuned on unsloth/gemma-3-12b-it-bnb-4bit. The adapter can be loaded and used with the PEFT library or merged with the base model for standalone use.

🚀 Quick Start

Using with PEFT (Recommended)

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model_name = "unsloth/gemma-3-12b-it-bnb-4bit"
adapter_name = "ewideplus/gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load and apply LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_name)

# For inference, merge adapter with base model for better performance
model = model.merge_and_unload()

# Use Alpaca format for instructions
def format_prompt(instruction, input_text=""):
    if input_text:
        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input_text}

### Response:
"""
    else:
        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:
"""

# Generate response
instruction = "Explain the concept of machine learning in simple terms."
prompt = format_prompt(instruction)

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response.split("### Response:")[-1].strip())

Merging with Base Model (Alternative)

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/gemma-3-12b-it-bnb-4bit",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load adapter
model = PeftModel.from_pretrained(base_model, "ewideplus/gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7")

# Merge and save as standalone model
merged_model = model.merge_and_unload()
merged_model.save_pretrained("./merged_model")

# Load tokenizer and save
tokenizer = AutoTokenizer.from_pretrained("unsloth/gemma-3-12b-it-bnb-4bit")
tokenizer.save_pretrained("./merged_model")

Adapter Details

—Base Model: unsloth/gemma-3-12b-it-bnb-4bit
—Dataset: Pustekhan-ITB/new-alpaca-edu-indonesian-v7 (version: main)
—Training Method: LoRA (Low-Rank Adaptation) with Unsloth
—Training Epochs: 1
—Adapter Status: 🔧 LoRA Adapter (requires base model)
—Training Completed: 2025-08-17T16:53:26.198321

Training Configuration

Parameter	Value
Learning Rate	0.0002
Batch Size	2
Gradient Accumulation Steps	4
Max Sequence Length	2048
LoRA Rank (r)	16
LoRA Alpha	32
LoRA Dropout	0.1

Model Statistics

Metric	Value
Total Parameters	6,671,051,376
Trainable Parameters	68,456,448
Trainable %	1.03%

Dataset Information

—Number of Training Samples: 8,450
—Dataset Features: instruction, input, output

Usage Notes

🔧 This is a LoRA Adapter

—Requires base model (unsloth/gemma-3-12b-it-bnb-4bit) to function
—Use with PEFT library for efficient loading and inference
—Can be merged with base model for standalone use
—Small file size - only contains the trained adapter weights

Instruction Format

This adapter was trained on the Alpaca instruction format:

### Instruction:
{Your instruction here}

### Input:
{Optional input context}

### Response:
{Model response}

Performance Tips

—Use merge_and_unload() for faster inference when using PEFT
—Consider merging adapter with base model if deploying to production
—Use torch.float16 or torch.bfloat16 for better memory efficiency
—For multi-adapter setups, you can load multiple LoRA adapters simultaneously

Training Framework

This adapter was fine-tuned using:

—[Unsloth](https://github.com/unslothai/unsloth): 2x faster training framework
—LoRA (Low-Rank Adaptation): Parameter-efficient fine-tuning
—Alpaca Format: Instruction-following dataset format
—PEFT: Parameter-Efficient Fine-Tuning library

Installation Requirements

bash

pip install transformers peft torch

For faster training and inference:

bash

pip install unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git

Limitations and Considerations

—This adapter inherits biases from the base model and training data
—Performance may vary on tasks not represented in the training dataset
—Always validate outputs for your specific use case
—Consider additional safety measures for production deployment
—Requires the specific base model version for optimal performance

Converting to Merged Model

To create a standalone merged model:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load and merge
base_model = AutoModelForCausalLM.from_pretrained("unsloth/gemma-3-12b-it-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "ewideplus/gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7")
merged_model = model.merge_and_unload()

# Save merged model
merged_model.save_pretrained("./merged_model")
tokenizer = AutoTokenizer.from_pretrained("unsloth/gemma-3-12b-it-bnb-4bit")
tokenizer.save_pretrained("./merged_model")

Citation

bibtex

@misc{ewideplus_gemma_3_12b_it_bnb_4bit_new_alpaca_edu_indonesian_v7,
  author = {Community},
  title = {gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/ewideplus/gemma-3-12b-it-bnb-4bit_new-alpaca-edu-indonesian-v7}
}

Acknowledgments

—Base Model: unsloth/gemma-3-12b-it-bnb-4bit
—Training Framework: Unsloth
—Dataset: Pustekhan-ITB/new-alpaca-edu-indonesian-v7
—Method: LoRA (Low-Rank Adaptation)
—Library: PEFT

This adapter card was automatically generated during the upload process.