wvnvwn/Mistral-7B-Instruct-v0.3-flora-v0

Mistral-7B-Instruct-v0.3-flora-v0

This repository contains a full merged model produced by applying a PEFT LoRA adapter to its base model. It is intended for reproducible evaluation without requiring a separate adapter loading path.

Quick Start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "wvnvwn/Mistral-7B-Instruct-v0.3-flora-v0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Run the bundled FSL evaluation wrapper:

bash fsl/src/evaluation/overall_eval.sh \
    --model_name wvnvwn/Mistral-7B-Instruct-v0.3-flora-v0 \
    --model_type instruct \
    --max_samples 100 \
    --output_root fsl/results/evaluation/Mistral-7B-Instruct-v0.3-flora-v0

Training Procedure

• —Procedure: Federated LoRA fine-tuning followed by adapter aggregation. The resulting PEFT LoRA adapter was merged into the base model for reproducible evaluation.
• —Algorithm: flora
• —Training data: data_hetero_with_4_tasks
• —Number of clients: 8
• —Communication round/checkpoint: 3
• —Local epochs: 3
• —Local batch size: 256
• —Local micro batch size: 32
• —Local learning rate: 0.0003

Original training command:

Not specified

Merge Metadata

• —Base model: mistralai/Mistral-7B-Instruct-v0.3
• —Adapter source: /NHNHOME/0226010080_A/BASE/jongbokwon/FLS/fsl/outputs/flora/8/2
• —PEFT type: LORA
• —Task type: CAUSAL_LM
• —LoRA rank: 128
• —LoRA alpha: 128
• —Target modules: k_proj, v_proj, down_proj, up_proj, gate_proj, o_proj, q_proj
• —Merged at UTC: 2026-05-21T13:31:22.398478+00:00

Framework Versions

• —python: 3.10.20
• —platform: Linux-6.8.0-100-generic-x86_64-with-glibc2.39
• —torch: 2.11.0+cu130
• —cuda: 13.0
• —transformers: 4.57.1
• —peft: 0.19.1
• —accelerate: 1.13.0
• —huggingface_hub: 0.36.2
• —safetensors: 0.7.0
• —vllm: 0.20.2
• —wandb: 0.27.0

Notes

• —This is a merged full-weight model, not an adapter-only checkpoint.
• —Redistribution/access should follow the base model license and access policy.
• —merge_info.json in this repository records the adapter and merge configuration.