Modelpublic

ayushozha/replicalab-scientist-grpo-lora

sourceHugging Facemitupdated 7mo agoView on Hugging Face

0likes24downloads

Model Card

ReplicaLab Scientist — GRPO LoRA Adapter

A LoRA adapter fine-tuned on unsloth/Qwen3.5-0.8B using Group Relative Policy Optimization (GRPO) for multi-agent scientific negotiation.

What is ReplicaLab?

ReplicaLab is a multi-agent constraint-aware planning environment that trains an AI Scientist agent to negotiate feasible scientific replication plans under realistic resource constraints. A Lab Manager enforces budgets, schedules, and equipment limits while a deterministic Judge scores every plan on rigor, feasibility, and fidelity.

Live demo: ayushozha-replicalab.hf.space

Training Details

—Method: GRPO (Group Relative Policy Optimization) via TRL
—Base model: unsloth/Qwen3.5-0.8B
—LoRA config: rank=16, alpha=32, dropout=0.0
—Target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
—Hardware: NVIDIA H100 80GB HBM3 (Northflank)
—Steps: 200 (checkpoints at 100, 150, 200)
—Training framework: Unsloth + TRL 0.24.0 + PEFT 0.18.1

Reward Formula

total_reward = 10 × rigor × feasibility × fidelity × parsimony
             + efficiency_bonus + communication_bonus − penalties

The multiplicative core prevents fake wins: a theoretically strong but impossible plan scores low.

Training Curves

Overview

[image]

Reward Over Training

[image]

Training Loss

[image]

KL Divergence

[image]

Completion Length

[image]

Evaluation Results

Improvement Over Baseline

[image]

Side-by-Side Comparison

[image]

Metric	Baseline Scientist	Trained Scientist	Change
Average reward	4.25	7.10	+67%
Rounds to agreement	4.1	2.8	−32%
Invalid action rate	15%	4%	−73%
Agreement rate	50%	80%	+60%
Avg rigor score	0.55	0.72	+31%
Avg feasibility score	0.52	0.78	+50%
Avg fidelity score	0.58	0.71	+22%

Scenario Families

Template	Domain	Example Task
`math_reasoning`	Mathematics	Proof planning under deadline and review constraints
`ml_benchmark`	Machine Learning	Model replication with compute and time budgets
`finance_trading`	Finance	Backtest design under capital and risk limits

Quick Start

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen3.5-0.8B")
model = PeftModel.from_pretrained(base_model, "ayushozha/replicalab-scientist-grpo-lora")
tokenizer = AutoTokenizer.from_pretrained("ayushozha/replicalab-scientist-grpo-lora")

# Use within the ReplicaLab environment for scientific negotiation

Framework Versions

—PEFT: 0.18.1
—TRL: 0.24.0
—Transformers: 5.2.0
—PyTorch: 2.8.0+cu128
—Datasets: 4.3.0
—Tokenizers: 0.22.2

Citation

bibtex

@misc{replicalab2026,
    title        = {ReplicaLab: Multi-Agent Constraint-Aware Planning for Scientific Replication},
    author       = {Ayush Ojha and Kian and Max and Kush},
    year         = 2026,
    url          = {https://github.com/Ayush10/replicalab-ai}
}

License

MIT