apple/DiffuCoder-7B-Instruct

1---2base_model:3- apple/DiffuCoder-7B-Base4tags:5- code6- text-diffusion-model7- diffusion large language model8license: apple-amlr9---10### DiffuCoder-7B-Instruct 11 12The DiffuCoder-7B-Instruct model builds on the DiffuCoder-7B-Base checkpoint with instruction-tuning to better follow code-related prompts.13 14- Training recipe: with a newly introduced pad token, we train this model with fixed length conditionally on [OpenCoder-SFT](https://huggingface.co/datasets/OpenCoder-LLM/opc-sft-stage2) data for 5 epochs.15 16- Benchmarks: Demonstrates stronger instruction-following capabilities than the Base model.17 18 19#### More details and usage examples:20 21- Paper: [DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation](https://arxiv.org/abs/2506.20639)22 23- GitHub: https://github.com/apple/ml-diffucoder24 25```26import torch27from transformers import AutoModel, AutoTokenizer28 29model_path = "apple/DiffuCoder-7B-Instruct"30model = AutoModel.from_pretrained(model_path, torch_dtype=torch.bfloat16, trust_remote_code=True)31tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)32model = model.to("cuda").eval()33 34query = "Write a function to find the shared elements from the given two lists."35prompt = f"""<|im_start|>system36You are a helpful assistant.<|im_end|>37<|im_start|>user38{query.strip()}39<|im_end|>40<|im_start|>assistant41""" ## following the template of qwen; you can also use apply_chat_template function42 43TOKEN_PER_STEP = 1 # diffusion timesteps * TOKEN_PER_STEP = total new tokens44 45inputs = tokenizer(prompt, return_tensors="pt")46input_ids = inputs.input_ids.to(device="cuda")47attention_mask = inputs.attention_mask.to(device="cuda")48 49output = model.diffusion_generate(50    input_ids,51    attention_mask=attention_mask,52    max_new_tokens=256,53    output_history=True,54    return_dict_in_generate=True,55    steps=256//TOKEN_PER_STEP,56    temperature=0.3,57    top_p=0.95,58    alg="entropy",59    alg_temp=0.,60)61generations = [62    tokenizer.decode(g[len(p) :].tolist())63    for p, g in zip(input_ids, output.sequences)64]65 66print(generations[0].split('<|dlm_pad|>')[0])67```68 69#### Acknowledgement70To power this HuggingFace model release, we reuse [Dream](https://huggingface.co/Dream-org/Dream-v0-Base-7B)'s modeling architecture and generation utils.71