mehmetdavut/ruby3.4-qwen2.5-7b-instruct-5k-all-8bit-gemini

ruby3.4-qwen2.5-7b-instruct-5k-all-8bit-gemini

This model is a part of the RubyCraft-3.4-Instruct research project, demonstrating the autonomous adaptation of Small Language Models (SLMs) to modern Ruby 3.4 syntax.

🏆 Model Details

• —Experiment ID: exp-112
• —Base Model: Qwen/Qwen2.5-7B-Instruct
• —Model Tier: Big
• —Training Data: 5K samples (Split: ALL) from the RubyCraft-3.4-Instruct Dataset
• —Teacher Model: Gemini-2.5-Flash
• —Quantization Strategy: 8-bit
• —Adapter Type: LoRA (Low-Rank Adaptation)

🧪 Performance & Evaluation

In our comprehensive evaluation covering 164 unique configurations, base models frequently suffered from "Formatting Hallucinations" (e.g., wrapping outputs in Markdown tags), resulting in zero scores within strict execution environments.

By applying our Diagnostic Sanitization Procedure (DSP) and fine-tuning on high-quality synthetic data, this adapter successfully recovers the model's Intrinsic Capability (IC) and achieves Extrinsic Compliance (EC) with Ruby 3.4 standards.

For detailed evaluation logs, pass rates, and the full experimental matrix, please refer to our Evaluation Logs Dataset.