twarner/dcode-sd-gcode-v3

1---2license: mit3library_name: diffusers4pipeline_tag: text-to-image5tags:6  - gcode7  - cnc8  - plotter9  - polargraph10  - stable-diffusion11  - text-to-gcode12  - diffusion13base_model: runwayml/stable-diffusion-v1-514datasets:15  - twarner/dcode-imagenet-sketch16---17 18# dcode: Text-to-Gcode Diffusion Model19 20An end-to-end diffusion model that converts **text prompts directly into G-code** for CNC machines, plotters, and polargraph drawing robots.21 22## Overview23 24dcode is a fine-tuned Stable Diffusion model with a custom G-code decoder head. It takes a text description (e.g., "a sketch of a horse") and outputs machine-executable G-code.25 26| Component | Description |27|-----------|-------------|28| Base Model | [Stable Diffusion v1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5) |29| Decoder | 200M param transformer (12 layers, 1024 hidden, 16 heads) |30| Tokenizer | Custom BPE tokenizer for G-code |31| Training Data | [dcode-imagenet-sketch](https://huggingface.co/datasets/twarner/dcode-imagenet-sketch) |32 33## Architecture34 35```36Text Prompt37    ↓38[CLIP Text Encoder] ← frozen39    ↓40[UNet Diffusion] ← frozen41    ↓42Latent (4×64×64)43    ↓44[CNN Projector] ← trained45    ↓46[Transformer Decoder] ← trained47    ↓48G-code Tokens49    ↓50G-code Text51```52 53## Usage54 55### With Diffusers56 57```python58import torch59from diffusers import StableDiffusionPipeline60from huggingface_hub import hf_hub_download61from transformers import PreTrainedTokenizerFast62 63# Load components64pipe = StableDiffusionPipeline.from_pretrained(65    "runwayml/stable-diffusion-v1-5",66    torch_dtype=torch.float1667).to("cuda")68 69# Download decoder weights70weights = hf_hub_download("twarner/dcode-sd-gcode-v3", "pytorch_model.bin")71tokenizer_path = hf_hub_download("twarner/dcode-sd-gcode-v3", "gcode_tokenizer/tokenizer.json")72 73# Load custom gcode tokenizer74gcode_tokenizer = PreTrainedTokenizerFast(tokenizer_file=tokenizer_path)75 76# Generate latent from text77with torch.no_grad():78    latent = pipe("a sketch of a horse", output_type="latent").images79 80# ... decode with GcodeDecoderV3 (see repo for full inference code)81```82 83### Interactive Demo84 85Try the model live: **[huggingface.co/spaces/twarner/dcode](https://huggingface.co/spaces/twarner/dcode)**86 87## Training88 89- **Dataset**: 50,000 ImageNet-Sketch images → 200,000 G-code files90- **Hardware**: 8× NVIDIA H100 80GB91- **Epochs**: 5092- **Batch Size**: 256 effective (32 × 8 GPUs)93- **Learning Rate**: 1e-4 with cosine schedule94- **Regularization**: Label smoothing (0.1), weight decay (0.05)95 96## G-code Output97 98The model generates G-code compatible with:99- Polargraph/drawbot machines100- Pen plotters101- Any G-code compatible CNC102 103Example output:104```gcode105G21 ; mm106G90 ; absolute107M280 P0 S90 ; pen up108G28 ; home109 110G0 X-200.00 Y100.00 F1000111M280 P0 S40 ; pen down112G1 X-180.00 Y120.00 F500113G1 X-160.00 Y115.00 F500114...115```116 117## Machine Specs118 119Default work area (configurable):120- Width: 841mm121- Height: 1189mm (A0 paper)122- Pen servo: 40° down, 90° up123 124## Project125 126Full project documentation, hardware build guide, and source code:127 128**🔗 [teddywarner.org/Projects/Polargraph/#dcode](https://teddywarner.org/Projects/Polargraph/#dcode)**129 130**GitHub**: [github.com/Twarner491/dcode](https://github.com/Twarner491/dcode)131 132## Citation133 134```bibtex135@misc{dcode2024,136  author = {Teddy Warner},137  title = {dcode: Text-to-Gcode Diffusion Model},138  year = {2026},139  url = {https://teddywarner.org/Projects/Polargraph/#dcode}140}141```142 143## License144 145MIT License146