CoolFace
Modelpublic

Meerkat-AI/Meerkat-TRIZ-v1-Qwen3.8-27B

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face
0likes19downloads
Model Card

Meerkat-TRIZ-v1 (Qwen3.8-27B)

LoRA adapter fine-tuned on TRIZ innovation methodology dataset.

Base Model

  • —Base: Qwen3.8-27B
  • —Architecture: Qwen3_5ForConditionalGeneration
  • —Precision: BF16

Adapter Config

  • —PEFT Type: LoRA
  • —Rank (r): 64
  • —Alpha: 128
  • —Dropout: 0.0
  • —Target Modules: qproj, kproj, vproj, oproj, inprojqkv, inprojz, inprojb, inproja, outproj, gateproj, upproj, downproj
  • —Trainable Params: 466,911,232 (1.7064% of base)

Training

  • —Framework: TRL SFTTrainer
  • —Dataset: v5a (11,096 train / 1,050 val)
  • —Max Length: 2048
  • —Horizon: 2,774 steps (cosine decay)
  • —Best Eval Loss: 1.505834 @ step 1300
  • —Early Stopping: patience=3, threshold=0.002

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "Meerkat-AI/Meerkat-TRIZ-v1-Qwen3.8-27B")
tokenizer = AutoTokenizer.from_pretrained("Meerkat-AI/Meerkat-TRIZ-v1-Qwen3.8-27B")

Related Models