joshuajoshy/brickstyle-gen

🧱 BrickStyle-Gen

Text-to-LEGO-Style Image Generator

Generate stunning brick-style images using Stable Diffusion XL + LEGO LoRA. Built with research foundation from Image2Lego (2021) for future 2D�3D brick conversion.

Project Status

Code Integration: Complete ✅ Testing: Requires dependency installation and model download (~7GB) Deployment: Ready for HuggingFace Spaces - see DEPLOY.md Examples: Documentation added - actual generations pending first run

The codebase is fully integrated and ready to deploy. Full testing requires installing PyTorch and diffusers, then downloading SDXL and LEGO LoRA weights. The HuggingFace Space deployment will handle this automatically.

( Features

Current (MVP)

• —<� Text-to-Brick Generation: Transform text prompts into LEGO-style images using SDXL + LEGO LoRA
• —<� Style Presets: 10+ curated presets (Minifig Portrait, Micro-City, BrickHeadz, etc.)
• —=� Gradio Web UI: Clean, intuitive interface with adjustable parameters
• —>� Research Stub: API framework for future 2D�3D brick conversion (based on Image2Lego paper)
• —=� CPU/GPU Support: Runs on CPU (with patience), MPS (M-series Mac), or CUDA

Planned (Future)

• —<� Reference Style (IP-Adapter): Generate images matching a reference style
• — Sketch-to-Brick (ControlNet): Convert sketches/doodles to brick-style
• —=� 2D�3D Conversion: Full Image2Lego pipeline integration (voxels � bricks � instructions)

=� Example Generations

(Coming soon - generate your first images and share!)

Sample Prompts:

• —"a medieval castle on a cliff" + Preset: Castle/Medieval
• —"a futuristic spaceship" + Preset: Space/Sci-Fi
• —"cute cat portrait" + Preset: BrickHeadz Style

See assets/sample_prompts.md for 18+ curated examples.

=� Quick Start

Prerequisites

• —Python 3.10 or higher
• —8GB+ RAM (16GB+ recommended)
• —GPU optional (CPU works, just slower)

Installation

# Clone the repository
git clone https://github.com/yeungjosh/brickstyle-gen.git
cd brickstyle-gen

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run the app
python app.py

The app will launch at http://localhost:7860

First Run Note: Model weights (~7GB) will download automatically. This may take 5-15 minutes depending on your connection.

Quick Run Scripts

# Using run.sh (Linux/Mac)
./run.sh

# Using Makefile
make run

⚙️ Configuration

BrickStyle-Gen supports environment variable configuration for advanced features and customization.

Core Settings

Optional Features

Safety & Moderation

Example Usage

# Enable ControlNet for sketch-to-brick
export BRICKSTYLE_ENABLE_CONTROLNET=1
python app.py

# Enable IP-Adapter for reference styling
export BRICKSTYLE_ENABLE_IP_ADAPTER=1
python app.py

# Use a custom LoRA
export BRICKSTYLE_LORA_REPO=/path/to/custom/lora
python app.py

Note: ControlNet and IP-Adapter require additional dependencies. Tabs will auto-disable with helpful setup instructions when dependencies are missing.

Usage

Basic Generation

1. 1.Enter a prompt in the "Prompt � Brick-Style" tab
2. 2.Select a preset (optional) for themed styles
3. 3.Adjust settings:
4. 4.Steps: 20-30 for fast, 40-50 for quality
5. 5.CFG Scale: 7-9 for balanced results
6. 6.Seed: 0 for random, or set for reproducibility
7. 7.Click Generate and wait 30-60 seconds (CPU) or 5-10 seconds (GPU)

Advanced Settings

• —LoRA Scale (0-1): Controls brick-style intensity
• —1.0 = Full LEGO style
• —0.5 = Blend with realistic style
• —Batch Size (1-4): Generate multiple variations at once
• —Resolution: 768�768 (fast) or 1024�1024 (quality)

Presets

Choose from 10 built-in presets:

• —Minifig Portrait: Classic yellow minifigure faces
• —Micro-City Diorama: Tiny architectural scenes
• —BrickHeadz Style: Cute cubic characters
• —Mech/Robot: Technic-style mechanical builds
• —Castle/Medieval: Knights and castles
• —Space/Sci-Fi: Classic space theme
• —Vehicle/Car: Sleek brick cars and trucks
• —Architecture: Detailed modular buildings
• —Nature/Organic: Plants and animals
• —Pixel Art: Flat mosaic designs

>� Research: Image2Lego Integration

This project is built on the research foundation of Image2Lego: Customized LEGO Set Generation from Images (2021).

Paper Summary

"We implement a system that generates a LEGO brick model from 2D images. Our system uses an octree-structured autoencoder to convert images into 3D voxel grids, then applies an algorithmic approach to convert voxels into physically realizable LEGO brick layouts, and finally generates step-by-step building instructions."

Citation:

@article{lennon2021image2lego,
  title={Image2Lego: Customized LEGO Set Generation from Images},
  author={Lennon, Kyle and Terry, Michael and Gordon, Mitchell R.},
  journal={arXiv preprint arXiv:2108.08477},
  year={2021}
}

Paper: arXiv:2108.08477

Current Status

The Research Tab in the app demonstrates the planned pipeline:

1. 1.Image � Voxels: Octree-structured VAE encoding
2. 2.Voxels � Bricks: Greedy placement algorithm with stability constraints
3. 3.Bricks � Instructions: Layer-by-layer assembly guide

See research/NOTES.md for detailed implementation roadmap and technical challenges.

=� Citations & Acknowledgments

Research

• —Image2Lego (2021): Kyle Lennon, Michael Terry, Mitchell R. Gordon (arXiv)

Models

• —LeLo - LEGO LoRA for SDXL: Trained with ~900 LEGO images (HuggingFace)
• —Stable Diffusion XL: Stability AI (HuggingFace)

Tools

• —IP-Adapter Docs: HuggingFace
• —ControlNet: HuggingFace

=� Development

Project Structure

brickstyle-gen/
 app.py                     # Main Gradio application
 pipelines/
    sdxl_lego_lora.py     # Core SDXL + LoRA generator
    ip_adapter.py         # IP-Adapter (planned)
    controlnet_canny.py   # ControlNet (planned)
 research/
    image2lego_stub.py    # 2D�3D API stubs
    NOTES.md              # Implementation roadmap
 assets/
    presets.json          # Style presets
    sample_prompts.md     # Example prompts
 requirements.txt          # Python dependencies
 README.md                 # This file

Running Tests

# Smoke test (generates one image on CPU)
python smoke_test.py

# Check installation
python -c "import torch; print(f'PyTorch: {torch.__version__}')"
python -c "import gradio; print(f'Gradio: {gradio.__version__}')"

Contributing

We welcome contributions! Key areas:

1. 1.2D�3D Pipeline: Implement Image2Lego research
2. 2.UI/UX: Improve Gradio interface
3. 3.Presets: Add new style presets
4. 4.Documentation: Tutorials and guides
5. 5.Testing: More robust test coverage

See research/NOTES.md for detailed research roadmap.

� Configuration

Environment Variables

Create a .env file for custom settings:

# Model cache directory
HF_HOME=/path/to/huggingface/cache

# GPU settings
CUDA_VISIBLE_DEVICES=0

# Development
GRADIO_SERVER_NAME=0.0.0.0
GRADIO_SERVER_PORT=7860

Performance Tuning

CPU (slow but works):

• —Reduce steps to 20-25
• —Use 768�768 resolution
• —Batch size = 1

M-series Mac (decent):

• —30-40 steps
• —1024�1024 resolution
• —Batch size = 1-2

NVIDIA GPU (fast):

• —40-50 steps
• —1024�1024 resolution
• —Batch size = 2-4
• —Enable xformers (uncomment in requirements.txt)

=� Deployment

Hugging Face Spaces

1. 1.Create a new Space at huggingface.co/spaces
2. 2.Choose Gradio as SDK
3. 3.Push repository:

   git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/brickstyle-gen
   git push hf main

1. 1.Space will auto-deploy (first build takes ~10 minutes)

Note: Free Spaces run on CPU. Consider upgrading to GPU for better performance.

See .huggingface/README.md for Space-specific configuration.

Local Server

# Run with external access
python app.py

# Custom port
GRADIO_SERVER_PORT=8080 python app.py

S FAQ

Q: How long does generation take? A: CPU: 30-90 seconds | M-series Mac: 15-30 seconds | NVIDIA GPU: 5-15 seconds

Q: Can I use my own LoRA? A: Yes! Modify pipelines/sdxl_lego_lora.py and change lora_repo parameter.

Q: Why are my images not brick-style? A: Check that LoRA downloaded successfully. Look for "LoRA loaded" in console logs.

Q: Can I generate 3D models yet? A: Not in MVP. See Research Tab for planned pipeline. Contributions welcome!

Q: Is this affiliated with LEGO? A: No. This generates brick-style art and is unaffiliated with the LEGO Group.

� Legal & Trademarks

This tool generates brick-style artwork and is not affiliated with or endorsed by the LEGO Group.

LEGO� is a trademark of the LEGO Group of companies which does not sponsor, authorize, or endorse this project.

Generated images are your own creations - use them freely for personal or commercial projects. Please respect trademark law and don't imply official LEGO affiliation.

=� License

MIT License - see LICENSE file for details.

Summary:

• — Use commercially
• — Modify and distribute
• — Private use
• —L No warranty or liability

=O Acknowledgments

• —Image2Lego research team for pioneering 2D�3D LEGO generation
• —LeLo LEGO LoRA trainer for the brick-style model
• —Stability AI for Stable Diffusion XL
• —HuggingFace for hosting and diffusers library
• —Gradio team for the amazing UI framework
• —Open-source community for inspiration and tools

= Links

• —Repository: github.com/yeungjosh/brickstyle-gen
• —Issues: Report bugs or request features
• —Discussions: Share your creations!

Made with >� by the open-source community

"Everything is awesome when you're part of a team!"