ruhil6789/solana-llama-3.2-3b-lora

license: llama3.2 basemodel: meta-llama/Llama-3.2-3B libraryname: peft pipeline_tag: text-generation tags:

• —llama
• —llama-3.2
• —lora
• —peft
• —transformers
• —solana
• —blockchain ---

Solana Llama-3.2-3B LoRA

📖 Project Description

This repository contains a LoRA fine-tuned version of Meta Llama-3.2-3B specialized for the Solana blockchain ecosystem.

The model was instruction-tuned to answer questions related to Solana development, smart contracts, Anchor framework, SPL Tokens, Program Derived Addresses (PDAs), Cross Program Invocation (CPI), accounts, transactions, and other Solana concepts.

🚀 Base Model

• —Model: meta-llama/Llama-3.2-3B
• —Fine-tuning Method: LoRA (PEFT)

📚 Dataset

The model was fine-tuned on a custom Solana instruction dataset consisting of:

• —Solana documentation
• —Solana developer guides
• —Smart contract explanations
• —Anchor framework examples
• —Question-answer pairs
• —Solana programming concepts

🏋️ Training Method

This model was trained using:

• —Hugging Face Transformers
• —TRL SFTTrainer
• —PEFT (LoRA)
• —BitsAndBytes 8-bit Optimizer

⚙️ Hyperparameters

💻 Example Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "meta-llama/Llama-3.2-3B"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL)

model = PeftModel.from_pretrained(
    base_model,
    "ruhil6789/solana-llama-3.2-3b-lora"
)

prompt = "Explain Program Derived Addresses (PDAs) in Solana."

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🎯 Intended Use

This model is intended for:

• —Solana developers
• —Blockchain learners
• —Smart contract development
• —Educational purposes

⚠️ Limitations

• —Specialized for Solana-related tasks.
• —Not a general-purpose LLM.
• —Verify generated code before production use.

📜 License

This LoRA adapter follows the Meta Llama-3.2 license.

👨‍💻 Author

Sachin Ruhil

Hugging Face: https://huggingface.co/ruhil6789