sttjr/paganini-qwen35-27b-grpo-lora

Paganini GRPO LoRA — Qwen3.5-27B

<p align="center"> <img src="https://img.shields.io/badge/Base%20Model-Qwen3.5--27B-blue" /> <img src="https://img.shields.io/badge/Method-GRPO%20%2B%20LoRA-purple" /> <img src="https://img.shields.io/badge/Language-PT--BR%20%7C%20EN-green" /> <img src="https://img.shields.io/badge/Domain-FIDC%20%7C%20Code-orange" /> <img src="https://img.shields.io/badge/License-Apache%202.0-lightgrey" /> </p>

Paganini is a dual-domain LoRA adapter trained via GRPO (Group Relative Policy Optimization) on top of Qwen3.5-27B. It serves as the intelligence backbone for 9 specialized FIDC agents in the Paganini AIOS platform, with deep expertise in Brazilian investment fund regulation (CVM 175) and software architecture.

🧠 Model Overview

🏗️ Training Pipeline

Paganini follows a two-stage alignment pipeline:

Qwen3.5-27B (base)
       │
       ▼
  ┌─────────────────────────────────────────┐
  │  Stage 1: Supervised Fine-Tuning (SFT)  │
  │  Platform: RunPod A100 80GB             │
  │  Accuracy: 87.75% | Loss: 0.454         │
  └─────────────────────────────────────────┘
       │
       ▼  sttjr/paganini-qwen35-27b-sft-lora
       │
  ┌─────────────────────────────────────────┐
  │  Stage 2: GRPO RL Alignment (this)      │
  │  Platform: Tinker API (TML Cloud GPUs)  │
  │  23 training runs | ~3 hours            │
  │  Dual-domain reward optimization        │
  └─────────────────────────────────────────┘
       │
       ▼  sttjr/paganini-qwen35-27b-grpo-lora  ← you are here

SFT Predecessor

The GRPO run was initialized from the SFT checkpoint:

• —SFT Model: sttjr/paganini-qwen35-27b-sft-lora
• —Platform: RunPod A100 80GB
• —Accuracy: 87.75%
• —Final Loss: 0.454

📦 Dataset

Name: dual-dataset-v2.jsonl

Difficulty distribution:

Sources:

• —Finance: FIDC (Fundo de Investimento em Direitos Creditórios) regulatory corpus under CVM Resolution 175 — covering eligibility, concentration limits, covenants, PLD/AML procedures, compliance gates, and risk management
• —Code: Software architecture patterns, pipeline compliance, TDD practices, and spec adherence for AIOS agent development

🎯 Reward Function (Dual-Domain)

The GRPO training uses a composite reward function:

R(x) = λ · R_code + (1 - λ) · R_fin + R_shared

Where λ = 1.0 for code samples and λ = 0.0 for finance samples.

R_code — Code Domain Rewards

R_finance — Finance Domain Rewards

R_shared — Shared Penalty/Bonus

🤖 Use Case: Paganini AIOS

This model is the intelligence backbone for 9 specialized FIDC domain agents in the Paganini AIOS platform:

6-Gate Guardrail Pipeline

Each query passes through a sequential compliance chain:

Input → [Eligibility] → [Concentration] → [Covenant] → [PLD/AML] → [Compliance] → [Risk] → Output

All 6 gates must pass before a response is delivered to end users. This ensures CVM 175-compliant, hallucination-free outputs across all agent types.

🚀 Usage

Installation

pip install transformers peft accelerate

Load and Run

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-27B",
    device_map="auto",
    torch_dtype="auto"
)

# Load GRPO LoRA adapter
model = PeftModel.from_pretrained(base, "sttjr/paganini-qwen35-27b-grpo-lora")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-27B")

# Finance domain example (PT-BR)
prompt = "Explique os requisitos de PDD mínima para FIDC conforme CVM 175."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Merge Adapter (Optional)

# Merge LoRA weights into base model for faster inference
merged_model = model.merge_and_unload()
merged_model.save_pretrained("paganini-27b-merged")
tokenizer.save_pretrained("paganini-27b-merged")

📊 Checkpoints

⚠️ Intended Use & Limitations

Intended Use

• —FIDC regulatory Q&A in Portuguese (Brazil)
• —Software architecture guidance for AIOS agents
• —Compliance-first financial analysis aligned with CVM 175
• —Internal enterprise use within the Paganini AIOS platform

Out-of-Scope Use

• —General-purpose chatbot (use base Qwen3.5-27B instead)
• —Non-Brazilian regulatory domains (model is specialized for CVM/BACEN frameworks)
• —Real-time trading decisions or autonomous financial transactions

Limitations

• —Finance knowledge is bounded by CVM 175 regulatory corpus at training cutoff
• —PT-BR outputs are prioritized; EN responses may be less fluent
• —Requires at least 2× A100 80GB GPUs or equivalent for full-precision inference
• —LoRA adapter requires the base Qwen3.5-27B model (~54 GB in fp16)

🔗 Project Links

📄 Citation

@misc{paganini-grpo-lora-2026,
  title        = {Paganini GRPO LoRA -- Qwen3.5-27B: Dual-Domain RL Alignment for FIDC Regulatory Intelligence},
  author       = {sttjr},
  year         = {2026},
  publisher    = {HuggingFace},
  howpublished = {\url{https://huggingface.co/sttjr/paganini-qwen35-27b-grpo-lora}},
  note         = {GRPO-aligned LoRA adapter for Brazilian investment fund regulation and software architecture}
}

📜 License

Apache 2.0 — See LICENSE for details.

Paganini AIOS — Built for the Brazilian FIDC ecosystem.