umjunsik1323/Qwen2.5-Coder-1.5B-roblox

🎮 Qwen2.5 Coder 1.5B Roblox

A specialized code generation model fine-tuned for Roblox Luau programming

⚡ Run in Google Colab (Recommended)


No setup required Click the badge above and run the chatbot instantly in your browser.

• —🎯 Pre-configured environment
• —🔥 GPU-accelerated inference
• —💬 Interactive chat interface
• —⏱️ Ready in ~3 minutes

📖 Overview

Qwen2.5 Coder 1.5B Roblox is a parameter-efficient fine-tuned model specifically designed for Roblox Luau development. Built on top of Qwen2.5-Coder-1.5B-Instruct, this model excels at generating, completing, and understanding Luau code patterns commonly used in Roblox game development.

🎯 What Makes This Special?

• —🎮 Roblox-Native: Trained exclusively on authentic Luau code from the official Roblox corpus
• —🧠 Context-Aware: Understands Roblox-specific APIs, patterns, and best practices

🏗️ Model Architecture

📚 Training Details

Dataset

• —Source: Roblox/luau_corpus
• —Filtering: Quality-filtered for code length (20-5000 chars) and Luau keyword presence
• —Split: 90% train / 10% validation

Training Configuration

{
    "max_length": 1024,
    "batch_size": 4,
    "gradient_accumulation_steps": 32,
    "learning_rate": 3e-5,
    "scheduler": "cosine_annealing",
    "epochs": 1,
    "optimizer": "AdamW"
}

🚀 Quick Start

Installation

pip install transformers peft torch

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-Coder-1.5B-Instruct",
    torch_dtype="auto",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "umjunsik1323/Qwen2.5-Coder-1.5B-roblox")

# Generate Luau code
messages = [
    {"role": "system", "content": "You are a Roblox Luau programming expert."},
    {"role": "user", "content": "Create a function to make a part glow"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Advanced: Merge and Export

# Merge LoRA weights into base model
merged_model = model.merge_and_unload()
merged_model.save_pretrained("./qwen-luau-merged")

💡 Features

Supported Tasks

• —✨ Code Completion: Finish partial Luau scripts intelligently
• —🔧 Function Generation: Create Roblox-specific functions from descriptions
• —📝 Code Explanation: Understand and document existing Luau code
• —🐛 Error Fixing: Suggest corrections for common Luau mistakes
• —🎯 API Usage: Generate proper Roblox API calls

Example Prompts

-- Completion
"local function teleportPlayer(player, position)"
→ Generates complete teleportation logic

-- Generation
"Create a tween that smoothly moves a part to a new position"
→ Generates TweenService implementation

-- Context-Aware
"Handle player damage with a cooldown system"
→ Generates debounce pattern with Humanoid health management

🎯 Use Cases

Game Development

• —Quick prototyping of Roblox mechanics
• —Learning Luau programming patterns
• —Code review and suggestions

Education

• —Teaching Roblox development
• —Demonstrating best practices
• —Interactive coding assistance

Productivity

• —Accelerating development workflows
• —Reducing boilerplate code
• —Standardizing team coding styles

⚠️ Limitations

• —Scope: Specialized for Luau only, not general-purpose programming
• —Context Window: Limited to 1024 tokens
• —Recency: Training data may not include latest Roblox API updates
• —Validation: Always test generated code in Roblox Studio

📄 Citation

@misc{youngseong_kim_2025,
	author       = { Youngseong Kim },
	title        = { Qwen2.5-Coder-1.5B-roblox (Revision 63e9452) },
	year         = 2025,
	url          = { https://huggingface.co/umjunsik1323/Qwen2.5-Coder-1.5B-roblox },
	doi          = { 10.57967/hf/7093 },
	publisher    = { Hugging Face }
}

📜 License

This LoRA adapter is released under Apache 2.0 License, maintaining compatibility with the base Qwen2.5-Coder model.

🤝 Acknowledgments

• —Qwen Team at Alibaba Cloud for the base model
• —Roblox for providing the Luau corpus dataset
• —Kaggle for providing the computational resources

Made with ❤️ for the Roblox Developer Community